Volumes: refactor volume grid storage
This refactors how volume grids are stored with the following new goals in mind:
* Get a **stand-alone volume grid** data structure that can be used by geometry nodes.
Previously, the `VolumeGrid` data structure was tightly coupled with the `Volume` data block.
* Support **implicit sharing of grids and trees**. Previously, it was possible to share data
when multiple `Volume` data blocks loaded grids from the same `.vdb` files but this was
not flexible enough.
* Get a safe API for **lazy-loading and unloading** of grids without requiring explicit calls
to some "load" function all the time.
* Get a safe API for **caching grids from files** that is not coupled to the `Volume` data block.
* Get a **tiered API** for different levels of `openvdb` involvement:
* No `OpenVDB`: Since `WITH_OPENVDB` is optional, it's helpful to have parts of the API that
still work in this case. This makes it possible to write high level code for volumes that does
not require `#ifdef WITH_OPENVDB` checks everywhere. This is in `BKE_volume_grid_fwd.hh`.
* Shallow `OpenVDB`: Code using this API requires `WITH_OPENVDB` checks. However, care
is taken to not include the expensive parts of `OpenVDB` and to use forward declarations as
much as possible. This is in `BKE_volume_grid.hh` and uses `openvdb_fwd.hh`.
* "Full" `OpenVDB`: This API requires more heavy `OpenVDB` includes. Fortunately, it turned
out to be not necessary for the common API. So this is only used for task specific APIs.
At the core of the new API is the `VolumeGridData` type. It's a wrapper around an
`openvdb::Grid` and adds some features on top like implicit sharing, lazy-loading and unloading.
Then there are `GVolumeGrid` and `VolumeGrid` which are containers for a volume grid.
Semantically, each `VolumeGrid` has its own independent grid, but this is cheap due to implicit
sharing. At highest level we currently have the `Volume` data-block which contains a list of
`VolumeGrid`.
```mermaid
flowchart LR
Volume --> VolumeGrid --> VolumeGridData --> openvdb::Grid
```
The loading of `.vdb` files is abstracted away behind the volume file cache API. This API makes
it easy to load and reuse entire files and individual grids from disk. It also supports caching
simplify levels for grids on disk.
An important new concept are the "tree access tokens". Whenever some code wants to work
with an openvdb tree, it has to retrieve an access token from the corresponding `VolumeGridData`.
This access token has to be kept alive for as long as the code works with the grid data. The same
token is valid for read and write access. The purpose of these access tokens is to make it possible
to detect when some code is currently working with the openvdb tree. This allows freeing it if it's
possible to reload it later on (e.g. from disk). It's possible to free a tree that is referenced by
multiple owners, but only no one is actively working with. In some sense, this is similar to the
existing `ImageUser` concept.
The most important new files to read are `BKE_volume_grid.hh` and `BKE_volume_grid_file_cache.hh`.
Most other changes are updates to existing code to use the new API.
Pull Request: https://projects.blender.org/blender/blender/pulls/116315
2023-12-20 15:32:52 +01:00
|
|
|
/* SPDX-FileCopyrightText: 2023 Blender Foundation
|
|
|
|
*
|
|
|
|
* SPDX-License-Identifier: GPL-2.0-or-later */
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
/** \file
|
|
|
|
* \ingroup bke
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <string>
|
|
|
|
|
|
|
|
#include "BKE_volume_enums.hh"
|
|
|
|
|
|
|
|
#include "BLI_math_matrix_types.hh"
|
|
|
|
|
|
|
|
/**
|
|
|
|
* This header gives contains declarations for dealing with volume grids without requiring
|
|
|
|
* including any OpenVDB headers (which can have a significant impact on compile times).
|
|
|
|
*
|
|
|
|
* These functions are available even if `WITH_OPENVDB` is false, but they may just be empty.
|
|
|
|
*/
|
|
|
|
|
|
|
|
namespace blender::bke::volume_grid {
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Wraps an OpenVDB grid and adds features like implicit sharing and lazy-loading.
|
|
|
|
*/
|
|
|
|
class VolumeGridData;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Owning container for a #VolumeGridData instance.
|
|
|
|
*/
|
|
|
|
class GVolumeGrid;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Same as #GVolumeGrid but means that the contained grid is of a specific type.
|
|
|
|
*/
|
|
|
|
template<typename T> class VolumeGrid;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Access token required to use the tree stored in a volume grid. This allows detecting whether a
|
|
|
|
* tree is currently used or not, for the purpose of safely freeing unused trees.
|
|
|
|
*/
|
|
|
|
class VolumeTreeAccessToken;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Compile time check to see of a type is a #VolumeGrid. This is false for e.g. `float` or
|
|
|
|
* `GVolumeGrid` and true for e.g. `VolumeGrid<int>` and `VolumeGrid<float>`.
|
|
|
|
*/
|
|
|
|
template<typename T> static constexpr bool is_VolumeGrid_v = false;
|
|
|
|
template<typename T> static constexpr bool is_VolumeGrid_v<VolumeGrid<T>> = true;
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Get the name stored in the volume grid, e.g. "density".
|
|
|
|
*/
|
|
|
|
std::string get_name(const VolumeGridData &grid);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Get the data type stored in the volume grid.
|
|
|
|
*/
|
|
|
|
VolumeGridType get_type(const VolumeGridData &grid);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Get the number of primitive values stored per voxel. For example, for a float-grid this is 1 and
|
|
|
|
* for a vector-grid it is 3 (for x, y and z).
|
|
|
|
*/
|
|
|
|
int get_channels_num(VolumeGridType type);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Unloads the tree data if no one is using it right now and it could be reloaded later on.
|
|
|
|
*/
|
|
|
|
void unload_tree_if_possible(const VolumeGridData &grid);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Get the transform of the grid as an affine matrix.
|
|
|
|
*/
|
|
|
|
float4x4 get_transform_matrix(const VolumeGridData &grid);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Replaces the transform matrix with the given one.
|
|
|
|
*/
|
|
|
|
void set_transform_matrix(VolumeGridData &grid, const float4x4 &matrix);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Clears the tree data in the grid, but keeps meta-data and the transform intact.
|
|
|
|
*/
|
|
|
|
void clear_tree(VolumeGridData &grid);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Makes sure that the volume grid is loaded afterwards. This is necessary to call this for
|
2024-01-08 01:24:37 +01:00
|
|
|
* correctness, because the grid will be loaded on demand anyway. Sometimes it may be beneficial
|
Volumes: refactor volume grid storage
This refactors how volume grids are stored with the following new goals in mind:
* Get a **stand-alone volume grid** data structure that can be used by geometry nodes.
Previously, the `VolumeGrid` data structure was tightly coupled with the `Volume` data block.
* Support **implicit sharing of grids and trees**. Previously, it was possible to share data
when multiple `Volume` data blocks loaded grids from the same `.vdb` files but this was
not flexible enough.
* Get a safe API for **lazy-loading and unloading** of grids without requiring explicit calls
to some "load" function all the time.
* Get a safe API for **caching grids from files** that is not coupled to the `Volume` data block.
* Get a **tiered API** for different levels of `openvdb` involvement:
* No `OpenVDB`: Since `WITH_OPENVDB` is optional, it's helpful to have parts of the API that
still work in this case. This makes it possible to write high level code for volumes that does
not require `#ifdef WITH_OPENVDB` checks everywhere. This is in `BKE_volume_grid_fwd.hh`.
* Shallow `OpenVDB`: Code using this API requires `WITH_OPENVDB` checks. However, care
is taken to not include the expensive parts of `OpenVDB` and to use forward declarations as
much as possible. This is in `BKE_volume_grid.hh` and uses `openvdb_fwd.hh`.
* "Full" `OpenVDB`: This API requires more heavy `OpenVDB` includes. Fortunately, it turned
out to be not necessary for the common API. So this is only used for task specific APIs.
At the core of the new API is the `VolumeGridData` type. It's a wrapper around an
`openvdb::Grid` and adds some features on top like implicit sharing, lazy-loading and unloading.
Then there are `GVolumeGrid` and `VolumeGrid` which are containers for a volume grid.
Semantically, each `VolumeGrid` has its own independent grid, but this is cheap due to implicit
sharing. At highest level we currently have the `Volume` data-block which contains a list of
`VolumeGrid`.
```mermaid
flowchart LR
Volume --> VolumeGrid --> VolumeGridData --> openvdb::Grid
```
The loading of `.vdb` files is abstracted away behind the volume file cache API. This API makes
it easy to load and reuse entire files and individual grids from disk. It also supports caching
simplify levels for grids on disk.
An important new concept are the "tree access tokens". Whenever some code wants to work
with an openvdb tree, it has to retrieve an access token from the corresponding `VolumeGridData`.
This access token has to be kept alive for as long as the code works with the grid data. The same
token is valid for read and write access. The purpose of these access tokens is to make it possible
to detect when some code is currently working with the openvdb tree. This allows freeing it if it's
possible to reload it later on (e.g. from disk). It's possible to free a tree that is referenced by
multiple owners, but only no one is actively working with. In some sense, this is similar to the
existing `ImageUser` concept.
The most important new files to read are `BKE_volume_grid.hh` and `BKE_volume_grid_file_cache.hh`.
Most other changes are updates to existing code to use the new API.
Pull Request: https://projects.blender.org/blender/blender/pulls/116315
2023-12-20 15:32:52 +01:00
|
|
|
* for performance to load the grid eagerly though.
|
|
|
|
*/
|
|
|
|
void load(const VolumeGridData &grid);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Returns a non-empty string if there was some error when the grid was loaded.
|
|
|
|
*/
|
|
|
|
std::string error_message_from_load(const VolumeGridData &grid);
|
|
|
|
|
|
|
|
/**
|
|
|
|
* True if the full grid (including meta-data, transform and the tree) is already available and
|
|
|
|
* does not have to be loaded lazily anymore.
|
|
|
|
*/
|
|
|
|
bool is_loaded(const VolumeGridData &grid);
|
|
|
|
|
|
|
|
} // namespace blender::bke::volume_grid
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Put the most common types directly into the `blender::bke` namespace.
|
|
|
|
*/
|
|
|
|
namespace blender::bke {
|
|
|
|
using volume_grid::GVolumeGrid;
|
|
|
|
using volume_grid::is_VolumeGrid_v;
|
|
|
|
using volume_grid::VolumeGrid;
|
|
|
|
using volume_grid::VolumeGridData;
|
|
|
|
using volume_grid::VolumeTreeAccessToken;
|
|
|
|
} // namespace blender::bke
|