Avoid copying the positions array into the evaluated edit hints array
that's used to support editing with deformed positions when there is
a topology-changing procedural operation. In a simple test in sculpt
mode with 706k curve points, memory usage went from 78 to 70 MB.
This adds more duplication would be ideal, mainly because retrieving
the data with write access and making implicit sharing info for arbitrary
arrays aren't abstracted by implicit sharing utilities. It may be possible
to improve both of those aspects, either now or in the future.
Pull Request: https://projects.blender.org/blender/blender/pulls/120146
With this patch, materials are kept intact in simulation zones and bake nodes
without any additional user action.
This implements the design proposed in #108410 to support referencing
data-blocks (only materials for now) in the baked data. The task also describes
why this is not a trivial issue. A previous attempt was implemented in #109703
but it didn't work well-enough.
The solution is to have an explicit `name (+ library name) -> data-block`
mapping that is stored in the modifier for each bake node and simulation zone.
The `library name` is necessary for it to be unique within a .blend file. Note
that this refers to the name of the `Library` data-block and not a file path.
The baked data only contains the names of the used data-blocks. When the baked
data is loaded, the correct material data-block is looked up from the mapping.
### Automatic Mapping Generation
The most tricky aspect of this approach is to make it feel mostly automatic.
From the user point-of-view, it should just work. Therefore, we don't want the
user to have to create the mapping manually in the majority of cases. Creating
the mapping automatically is difficult because the data-blocks that should
become part of the mapping are only known during depsgraph evaluation. So we
somehow have to gather the missing data blocks during evaluation and then write
the new mappings back to the original data.
While writing back to original data is something we do in some cases already,
the situation here is different, because we are actually creating new relations
between data-blocks. This also means that we'll have to do user-counting. Since
user counts in data-blocks are *not* atomic, we can't do that from multiple
threads at the same time. Also, under some circumstances, it may be necessary to
trigger depsgraph evaluation again after the write-back because it actually
affects the result.
To solve this, a small new API is added in `DEG_depsgraph_writeback_sync.hh`. It
allows gathering tasks which write back to original data in a synchronous way
which may also require a reevaluation.
### Accessing the Mapping
A new `BakeDataBlockMap` is passed to geometry nodes evaluation by the modifier.
This map allows getting the `ID` pointer that should be used for a specific
data-block name that is stored in baked data. It's also used to gather all the
missing data mappings during evaluation.
### Weak ID References
The baked/cached geometries may have references to other data-blocks (currently
only materials, but in the future also e.g. instanced objects/collections).
However, the pointers of these data-blocks are not stable over time. That is
especially true when storing/loading the data from disk, but also just when
playing back the animation. Therefore, the used data-blocks have to referenced
in a different way at run-time.
This is solved by adding `std::unique_ptr<bake::BakeMaterialsList>` to the
run-time data of various geometry data-blocks. If the data-block is cached over
a longer period of time (such that material pointers can't be used directly), it
stores the material name (+ library name) used by each material slot. When the
geometry is used again, the material pointers are restored using these weak name
references and the `BakeDataBlockMap`.
### Manual Mapping Management
There is a new `Data-Blocks` panel in the bake settings in the node editor
sidebar that allows inspecting and modifying the data-blocks that are used when
baking. The user can change what data-block a specific name is mapped to.
Pull Request: https://projects.blender.org/blender/blender/pulls/117043
Some common headers were including this. Separating the includes
will ideally lead to better conceptual separation between CustomData
and the attribute API too. Mostly the change is adding the file to
places where it was included indirectly before. But some code is
shuffled around to hopefully better places as well.
Remove most includes of this header inside other headers, to remove unnecessary
indirect includes which can have a impact on compile times. In the future we may
want more dedicated "_fwd.hh" headers, but until then, this sticks with the
solution in existing code.
Unfortunately it isn't yet possible to remove the include from `BKE_geometry_set.hh`.
Each value is now out of the global namespace, so they can be shorter
and easier to read. Most of this commit just adds the necessary casting
and namespace specification. `enum class` can be forward declared since
it has a specified size. We will make use of that in the next commit.
This PR implements an initial drawing tool that can already be used for testing.
While this is not fully feature complete (compared to the current grease pencil draw tool) the following is already implemented:
* Pressure support for radius and opacity.
* Material color and vertex color support.
* New active smoothing algorithm based on curve fitting.
* Simplify algorithm as a post-process step.
Some deliberate limitations include:
* The drawing plane is always the front plane. Drawing on surfaces is also not supported.
*
The current approach has not been optimized for performance yet. The goal was to have a straightforward implementation
first and then focus on performance later.
There are numerous parameters in the code that are hard-coded for now. These should be exposed at some point, potentially as user settings.
Pull Request: https://projects.blender.org/blender/blender/pulls/110093
This PR adds vertex groups to `CurvesGeometry` as well as an attribute read/write accessor.
This is also in preparation for GPv3. Since the goal is to have full compatibility with the current grease pencil features, vertex groups need to be supported.
Grease Pencil allows filtering by vertex group for modifiers.To support this, it also makes sense to have read/write access for vertex groups in the attributes API.
In the future, vertex groups should be just another custom attribute on meshes/curves/grease pencil. There are some more issues to be solved before that can happen. This step gets us a bit closer since the vertex weight data is stored in `CustomData`.
Pull Request: https://projects.blender.org/blender/blender/pulls/106944
Since 34b4487844, attributes are always made mutable when
accessed from the RNA API. This can result in unnecessary copies, which
increases memory usage and reduces performance.
Cycles is the only user of the C++ RNA API, which we'd like to remove
in the future since it doesn't really make sense in the big picture.
Hydra is now a better alternative for external render engines.
To start that change and fix the unnecessary copies, this commit
moves to use Blender headers directly for accessing attribute and
other geometry data. This also removes the few places that still had
overhead from the RNA API after the changes ([0]) in 3.6. In a simple
test with a large grid, I observed a 1.76x performance improvement,
from 1.04 to 0.59 seconds to extract the mesh data to Cycles.
[0]: https://wiki.blender.org/wiki/Reference/Release_Notes/3.6/Cycles#Performance
Pull Request: https://projects.blender.org/blender/blender/pulls/112306
In 2788fa915b the `CurvesGeometry::blend_write` and `CurvesGeometry::blend_read` functions were added. Unfortunately, the commit also altered the writing logic and introduced a bug where loading a file with a converted `Curves` object would crash. See https://projects.blender.org/blender/blender/issues/112068.
This PR fixes the issue by making sure that `CustomData_blend_write_prepare` is called before `BKE_id_blend_write`, which is the root cause of the crash that happens on load.
The `CurvesGeometry::blend_write` function is split into `CurvesGeometry::blend_write_prepare` and `CurvesGeometry::blend_write`.
Pull Request: https://projects.blender.org/blender/blender/pulls/112280
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
More consistently return geometry bounds with the `Bounds` type that
holds the min and max in one variable. This simplifies some code and
reduces the need to initialize separate min and max variables first.
Meshes now use the same `bounds_min_max()` function as curves and
point clouds, though the wrapper mesh isn't affected yet.
The motivation is to make some of the changes for #96968 simpler.
A lot of files were missing copyright field in the header and
the Blender Foundation contributed to them in a sense of bug
fixing and general maintenance.
This change makes it explicit that those files are at least
partially copyrighted by the Blender Foundation.
Note that this does not make it so the Blender Foundation is
the only holder of the copyright in those files, and developers
who do not have a signed contract with the foundation still
hold the copyright as well.
Another aspect of this change is using SPDX format for the
header. We already used it for the license specification,
and now we state it for the copyright as well, following the
FAQ:
https://reuse.software/faq/
Pass the curves and points to keep instead of delete. In the same test
file as the previous commit, this gave an increase from 50 to 60 FPS
when deleting curves.
Goals of this refactor:
* Reduce memory consumption of `IndexMask`. The old `IndexMask` uses an
`int64_t` for each index which is more than necessary in pretty much all
practical cases currently. Using `int32_t` might still become limiting
in the future in case we use this to index e.g. byte buffers larger than
a few gigabytes. We also don't want to template `IndexMask`, because
that would cause a split in the "ecosystem", or everything would have to
be implemented twice or templated.
* Allow for more multi-threading. The old `IndexMask` contains a single
array. This is generally good but has the problem that it is hard to fill
from multiple-threads when the final size is not known from the beginning.
This is commonly the case when e.g. converting an array of bool to an
index mask. Currently, this kind of code only runs on a single thread.
* Allow for efficient set operations like join, intersect and difference.
It should be possible to multi-thread those operations.
* It should be possible to iterate over an `IndexMask` very efficiently.
The most important part of that is to avoid all memory access when iterating
over continuous ranges. For some core nodes (e.g. math nodes), we generate
optimized code for the cases of irregular index masks and simple index ranges.
To achieve these goals, a few compromises had to made:
* Slicing of the mask (at specific indices) and random element access is
`O(log #indices)` now, but with a low constant factor. It should be possible
to split a mask into n approximately equally sized parts in `O(n)` though,
making the time per split `O(1)`.
* Using range-based for loops does not work well when iterating over a nested
data structure like the new `IndexMask`. Therefor, `foreach_*` functions with
callbacks have to be used. To avoid extra code complexity at the call site,
the `foreach_*` methods support multi-threading out of the box.
The new data structure splits an `IndexMask` into an arbitrary number of ordered
`IndexMaskSegment`. Each segment can contain at most `2^14 = 16384` indices. The
indices within a segment are stored as `int16_t`. Each segment has an additional
`int64_t` offset which allows storing arbitrary `int64_t` indices. This approach
has the main benefits that segments can be processed/constructed individually on
multiple threads without a serial bottleneck. Also it reduces the memory
requirements significantly.
For more details see comments in `BLI_index_mask.hh`.
I did a few tests to verify that the data structure generally improves
performance and does not cause regressions:
* Our field evaluation benchmarks take about as much as before. This is to be
expected because we already made sure that e.g. add node evaluation is
vectorized. The important thing here is to check that changes to the way we
iterate over the indices still allows for auto-vectorization.
* Memory usage by a mask is about 1/4 of what it was before in the average case.
That's mainly caused by the switch from `int64_t` to `int16_t` for indices.
In the worst case, the memory requirements can be larger when there are many
indices that are very far away. However, when they are far away from each other,
that indicates that there aren't many indices in total. In common cases, memory
usage can be way lower than 1/4 of before, because sub-ranges use static memory.
* For some more specific numbers I benchmarked `IndexMask::from_bools` in
`index_mask_from_selection` on 10.000.000 elements at various probabilities for
`true` at every index:
```
Probability Old New
0 4.6 ms 0.8 ms
0.001 5.1 ms 1.3 ms
0.2 8.4 ms 1.8 ms
0.5 15.3 ms 3.0 ms
0.8 20.1 ms 3.0 ms
0.999 25.1 ms 1.7 ms
1 13.5 ms 1.1 ms
```
Pull Request: https://projects.blender.org/blender/blender/pulls/104629
Generally, one does not know if the sharing info is currently shared
and should therefore be const. Better keep it const almost all the
time and only remove the constness when absolutely necessary
and the code has checked that it is valid.
Similar to 7eee378ecc, this change decreases memory usage and
improves performance when copying curves and meshes without changing
their topology. The same change used for custom data layers is applied
to face and curve offset indices, which aren't stored as a custom data
layer.
The implicit sharing info for the offsets is stored in the mesh and
curve runtime structs, since it doesn't need to be written to files
directly. When changing the offsets pointer directly, the sharing info
must be updated accordingly. To make that easier, a few utility
functions take care of common operations like making an array mutable,
resizing an array, and creating sharing info for allocated data.
This commit also clarifies the intention to not allocate the offsets
at all when there are no curves/faces. That slightly complicates some
of the logic, but there's no reason for the single `0` integer to be
allocated.
Pull Request: https://projects.blender.org/blender/blender/pulls/106907
For example
```
OIIOOutputDriver::~OIIOOutputDriver()
{
}
```
becomes
```
OIIOOutputDriver::~OIIOOutputDriver() {}
```
Saves quite some vertical space, which is especially handy for
constructors.
Pull Request: https://projects.blender.org/blender/blender/pulls/105594
This add the`CurvesGeometry::blend_read` and `CurvesGeometry::blend_write` methods
and uses them in the `curves_blend_read_data` and `curves_blend_write` functions.
No functional changes.
Pull Request: https://projects.blender.org/blender/blender/pulls/105737
The evaluated positions cache can live longer than a specific
`CurvesGeometry`, but for only-poly curves, it pointed to the positions,
which are freed when the curves are. Instead, use the same pattern
as the evaluated offsets and don't store the positions span, just return
it when retrieving evaluated positions.
Straightforward port. I took the oportunity to remove some C vector
functions (ex: copy_v2_v2).
This makes some changes to DRWView to accomodate the alignement
requirements of the float4x4 type.
Use the `SharedCache` concept introduced in D16204 to share lazily
calculated evaluated data between original and multiple evaluated
curves data-blocks. Combined with D14139, this should basically remove
most costs associated with copying large curves data-blocks (though
they add slightly higher constant overhead). The caches should
interact well with undo steps, limiting recalculations on undo/redo.
Options for avoiding the new overhead associated with the shared
caches are described in T104327 and can be addressed separately.
Simple situations affected by this change are using any of the following data
on an evaluated curves data-block without first invalidating it:
- Evaluated offsets (size of evaluated curves)
- Evaluated positions
- Evaluated tangents
- Evaluated normals
- Evaluated lengths (spline parameter node)
- Internal Bezier and NURBS caches
In a test with 4m points and 170k curves, using curve normals in a
procedural setup that didn't change positions procedurally gave 5x
faster playback speeds. Avoiding recalculating the offsets on every
update saved about 3 ms for every sculpt update for brushes that don't
change topology.
Differential Revision: https://developer.blender.org/D17134
Straightforward port. I took the oportunity to remove some C vector
functions (ex: `copy_v2_v2`).
This makes some changes to DRWView to accomodate the alignement
requirements of the float4x4 type.
Also minor changes in comments:
- Reference BLENDER_HISTORY_FILE instead of the literal file-name
(simplifies looking up usage).
- Use usernames in tags, as noted in code-style.
The same logic from D17025 is used in other places in the curve code.
This patch uses the class for the evaluated point offsets and the Bezier
control point offsets. This helps to standardize the behavior and make
it easier to read.
Previously the Bezier control point offsets used a slightly different standard
where the first point was the first offset, just so they could have the same
size as the number of points. However two nodes used a helper function
to use the same `OffsetIndices` system, so switch to that there too.
That requires removing the subtraction by one to find the actual offset.
Also add const when accessing data arrays from curves, for consistency.
Differential Revision: https://developer.blender.org/D17038
This changes how we access the points that correspond to each curve in a `CurvesGeometry`.
Previously, `CurvesGeometry::points_for_curve(int curve_index) -> IndexRange`
was called for every curve in many loops. Now one has to call
`CurvesGeometry::points_by_curve() -> OffsetIndices` before the
loop and use the returned value inside the loop.
While this is a little bit more verbose in general, it has some benefits:
* Better standardization of how "offset indices" are used. The new data
structure can be used independent of curves.
* Allows for better data oriented design. Generally, we want to retrieve
all the arrays we need for a loop first and then do the processing.
Accessing the old `CurvesGeometry::points_for_curve(...)` did not follow
that design because it hid the underlying offset array.
* Makes it easier to pass the offsets to a function without having to
pass the entire `CurvesGeometry`.
* Can improve performance in theory due to one less memory access
because `this` does not have to be dereferenced every time.
This likely doesn't have a noticable impact in practice.
Differential Revision: https://developer.blender.org/D17025
Previously, the lifetimes of anonymous attributes were determined by
reference counts which were non-deterministic when multiple threads
are used. Now the lifetimes of anonymous attributes are handled
more explicitly and deterministically. This is a prerequisite for any kind
of caching, because caching the output of nodes that do things
non-deterministically and have "invisible inputs" (reference counts)
doesn't really work.
For more details for how deterministic lifetimes are achieved, see D16858.
No functional changes are expected. Small performance changes are expected
as well (within few percent, anything larger regressions should be reported as
bugs).
Differential Revision: https://developer.blender.org/D16858
Use the same `".selection"` attribute for both curve and point domains,
instead of a different name for each. The attribute can now have
either boolean or float type. Some tools create boolean selections.
Other tools create float selections. Some tools "upgrade" the attribute
from boolean to float.
Edit mode tools that create selections from scratch can create boolean
selections, but edit mode should generally be able to handle both
selection types. Sculpt mode should be able to read boolean selections,
but can also and write float values between zero and one.
Theoretically we could just always use floats to store selections,
but the type-agnosticism doesn't cost too much complexity given the
existing APIs for dealing with it, and being able to use booleans is
clearer in edit mode, and may allow future optimizations like more
efficient ways to store boolean attributes.
The attribute API is usually used directly for accessing the selection
attribute. We rely on implicit type conversion and domain interpolation
to simplify the rest of the code.
Differential Revision: https://developer.blender.org/D16057
Bounding box calculation can be a large in some situations, especially
instancing. This patch caches the min and max of the bounding box in
runtime data of meshes, point clouds, and curves, implementing part of
T96968.
Bounds are now calculated lazily-- only after they are tagged dirty.
Also, cached bounds are also shared when copying geometry data-blocks
that have equivalent data. When bounds are calculated on an evaluated
data-block, they are also accessible on the original, and the next
evaluated ID will also share them. A geometry will stop sharing bounds
as soon as its positions (or radii) are changed.
Just caching the bounds gave a 2-3x speedup with thousands of mesh
geometry instances in the viewport. Sharing the bounds can eliminate
recalculations entirely in cases like copying meshes in geometry nodes
or the selection paint brush in curves sculpt mode, which causes a
reevaluation but doesn't change the positions.
**Implementation**
The sharing is achieved with a `shared_ptr` that points to a cache mutex
(from D16419) and the cached bounds data. When geometries are copied,
the bounds are shared by default, and only "un-shared" when the bounds
are tagged dirty.
Point clouds have a new runtime struct to store this data. Functions
for tagging the data dirty are improved for added for point clouds
and improved for curves. A missing tag has also been fixed for mesh
sculpt mode.
**Future**
There are further improvements which can be worked on next
- Apply changes to volume objects and other types where it makes sense
- Continue cleanup changes described in T96968
- Apply shared cache design to more expensive data like triangulation
or normals
Differential Revision: https://developer.blender.org/D16204
This patch introduces a new `CacheMutex` which makes it easy to implement
lazily computed caches in e.g. `Curves`. For more details see `BLI_cache_mutex.hh`.
Differential Revision: https://developer.blender.org/D16419
This patch contains an initial set of nodes to access basic
mesh topology information, as explored in T100020.
The nodes allow six direct topology mappings for meshes:
- **Corner -> Face** The face a corner is in, the index in the face
- **Vertex -> Edge** Choose an edge attached to the vertex
- **Vertex -> Corner** Choose a corner attached to the vertex
- **Corner -> Edge** The next and previous edge at each face corner
- **Corner -> Vertex** The vertex associated with a corner
- **Corner -> Corner** Offset a corner index within a face
And two new topology mappings for curves:
- **Curve -> Points** Choose a point within a curve
- **Point -> Curve** The curve a point is in, the index in the curve
The idea is that some of the 16 possible mesh mappings are more
important, and that this is a useful set of nodes to start exploring
this area. For mappings with an arbitrary number of connections, we
must sort them and use an index to choose a single element, because
geometry nodes does not support list fields. Note that the sort
index has repeating behavior as it goes over the "Total" number of
connections, and negative sort indices choose from the end.
Currently which of the "start" elements is used is determined by the
field context, so the "Field at Index" and "Interpolate Domain" nodes
will be quite important. Also, currently the "Sort Index" inputs are
clamped to the number of connections.
One important feature that isn't implemented here is using the winding
order for the output elements. This can be a separate mode for some
of these nodes. It will be optional because of the performance impact.
There are several todos for separate commits after this:
- Rename "Control Point Neighbors" to be consistent with this naming
- Version away the "Vertex Neighbors" node which is fully redundant now
- Implement a special case for when no weights are used for performance
- De-duplicating some of the sorting logic between the nodes
- Improve performance and memory use of topology mappings
- Look into caching some of the mappings on meshes
Differential Revision: https://developer.blender.org/D16029
Correct interpolation of integer POD types for Catmull Rom
interpolation as implemented in eaf416693d.
**Problem description**
`attribute_math::DefaultMixer<T>::mix_in()` assumes/asserts positive
weights but the basis function for Catmull-Rom splines generates
negative weights (see image in revision). Passing negative weights will
yield correct result as sum(weights) = 1 (after multiplication by 0.5)
but the assert is still triggered in debug builds. This patch adjusts
the behavior by extending the mix functions with mix4(). The benefit
of using mix#() over a DefaultMixer is that the result no longer needs
to be divided by the weight sum, instead utilizing that the basis weight
sum is constant (see plot).
**Changes**
* Added mix4() and updated catmull_rom::interpolate() to use it.
* Removed TODOs from catmull_rom functions.
* Moved mix definitions to be ordered as 2, 3, 4 in the header.
**Implementation specifics**
`catmull_rom::interpolate()` uses a constexpr to differentiate between
POD types which multiplies the result with 0.5 after weighting the
values, this reduces the number of multiplications for 1D, 2D, 3D
vectors (https://godbolt.org/z/8M1z9Pxx6). While this could be
considered unnecessary, I didn't want to change the original behavior
as it could influence performance (did not measure performance here
as this should ensure the logic is ~identical for FP types).
Differential Revision: https://developer.blender.org/D15997
While this worked, the result for curves with a resolution of zero was
just a single evaluated point, which isn't useful or intuitive. Using
the attribute validation from 8934f00ac5, make sure users
can't set values 0 or less.
We expect some builtin attributes to have positive values or values
within a certain range, but currently there some cases where users
can set attributes to arbitrary values: the store named attribute node,
and the output attributes of the geometry nodes modifier. The set
material index node also needs validation.
This patch adds an `AttributeValidator` to the attribute API, which
can be used to correct values from these untrusted inputs if necessary.
As an alternative to D15548, this approach makes it much easier to
understand when validation is being applied, without the need to add
arguments to every attribute API method or complicate the virtual
array system.
Currently validation is provided with a multi-function. That integrates
well with the field evaluations that set these values now, but it could
be wrapped to be friendlier to other areas of Blender in the future.
The Python API is not handled here either. Currently I would prefer to
wait until we can integrate the C++ and C attribute APIs better before
addressing that.
Fixes T100952
Differential Revision: https://developer.blender.org/D15990
The trim functionality is implemented in the geometry module, and
generalized a bit to be potentially useful for bisecting in the future.
The implementation is based on a helper type called `IndexRangeCyclic`
which allows iteration over all control points between two points on a
curve.
Catmull Rom curves are now supported-- trimmed without resampling first.
However, maintaining the exact shape is not possible. NURBS splines are
still converted to polylines using the evaluated curve concept.
Performance is equivalent or faster then a 3.1 build with regards to
node timings. Compared to 3.3 and 3.2, it's easy to observe test cases
where the node is at least 3 or 4 times faster.
Differential Revision: https://developer.blender.org/D14481
Hans Goudey
Jacques Lucke
Campbell Barton
Falk David
Hans Goudey
Sergey Sharybin
Clément Foucault
Ray Molenkamp
Chris Blackbourn
Mattias Fredriksson