The depsgraph CoW mechanism is a bit of a misnomer. It creates an
evaluated copy for data-blocks regardless of whether the copy will
actually be written to. The point is to have physical separation between
original and evaluated data. This is in contrast to the commonly used
performance improvement of keeping a user count and copying data
implicitly when it needs to be changed. In Blender code we call this
"implicit sharing" instead. Importantly, the dependency graph has no
idea about the _actual_ CoW behavior in Blender.
Renaming this functionality in the despgraph removes some of the
confusion that comes up when talking about this, and will hopefully
make the depsgraph less confusing to understand initially too. Wording
like "the evaluated copy" (as opposed to the original data-block) has
also become common anyway.
Pull Request: https://projects.blender.org/blender/blender/pulls/118338
Bundling many tests in a single binary reduces build time and disk space
usage, but is less convenient for running individual tests command line
as filter flags need to be used.
This adds WITH_TESTS_SINGLE_BINARY to generate one executable file per
source file. Note that enabling this option requires a significant amount
of disk space.
Due to refactoring, the resulting ctest names are a bit different than
before. The number of tests is also a bit different depending if this
option is used, as one uses gtests discovery and the other is organized
purely by filename, which isn't always 1:1.
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/114604
Along with the 4.1 libraries upgrade, we are bumping the clang-format
version from 8-12 to 17. This affects quite a few files.
If not already the case, you may consider pointing your IDE to the
clang-format binary bundled with the Blender precompiled libraries.
This refactors `SocketValueVariant` with the following goals in mind:
* Support type erasure so that not all users of `SocketValueVariant` have
to know about all the types sockets can have.
* Move towards supporting "rainbow sockets" which are sockets whoose
type is only known at run-time.
* Reduce complexity when dealing with socket values in general. Previously,
one had to use `SocketValueVariantCPPType` a lot to manage uninitialized
memory. This is better abstracted away now.
One related change that I had to do that I didn't see coming at first was that
I had to refactor `set_default_remaining_outputs` because now the default value
of a `SocketValueVariant` would not contain any value. Previously, it was
initialized the zero-value of the template parameter. Similarly, I had to change
how implicit conversions are created, because comparing the `CPPType` of linked
sockets was not enough anymore to determine if a conversion is necessary.
We could potentially use `SocketValueVariant` for the remaining socket types in the
future as well. Not entirely sure if that helps yet. `SocketValueVariant` can easily be
adapted to make that work though. That would also justify the name
"SocketValueVariant" better.
Pull Request: https://projects.blender.org/blender/blender/pulls/116231
Due to changes in the build environment shader_builder wasn't able to
compile on macOs. This patch reverts several recent changes to CMake files.
* dbb2844ed9
* 94817f64b9
* 1b6cd937ff
The idea is that in the near future shader_builder will run on the buildbot as
part of any regular build to ensure that changes to the CMake doesn't break
shader_builder and we only detect it after a few days.
Pull Request: https://projects.blender.org/blender/blender/pulls/115929
NDEBUG is part of the C standard and disables asserts. Only this will
now be used to decide if asserts are enabled.
DEBUG was a Blender specific define, that has now been removed.
_DEBUG is a Visual Studio define for builds in Debug configuration.
Blender defines this for all platforms. This is still used in a few
places in the draw code, and in external libraries Bullet and Mantaflow.
Pull Request: https://projects.blender.org/blender/blender/pulls/115774
This struct is currently defined in the `functions` module but not actually used there. It's only used by the geometry nodes module, with an indirect dependency from blenkernel via simulation zone baking. This scope is problematic when adding grids as socket data, which should not be part of the functions module.
The `ValueOrField` struct is now moved to blenkernel, so it can be more easily extended to other kinds of data that might be passed around by geometry nodes sockets in future. No functional changes.
Pull Request: https://projects.blender.org/blender/blender/pulls/115087
Nodes that are scheduled can be executed in any order in theory. So when
there are many scheduled nodes, it can be benefitial to start evaluating
them in parallel.
Note that it is not very common that many nodes are scheduled at the
same time in typical setups because the evaluator uses a depth-first heuristic
to decide in which order to evaluate nodes. It can happen more easily in
generated node trees though.
Also, this change only has an affect in practice if none of the scheduled nodes
uses multi-threading internally, as this would also trigger the user of multiple
threads in the graph executor.
This idea is of remapping parameters of lazy-functions is useful
not only for the repeat zone. For example, it could be used for the
for-each zone as well.
Also, moving it to a more general place indicates that there is no
repeat-zone specific stuff in it.
The main goal of this refactor is to simplify how a geometry node group is executed.
Previously, there was duplicated logic that turned the lazy-function graph of a node
group into a single lazy-function. Now this is done only in one place and others can
just execute the lazy-function directly, without having to worry about the underlying graph.
Pull Request: https://projects.blender.org/blender/blender/pulls/112482
Goals of the refactor:
* Simplify adding (named) graph inputs and outputs.
* Add ability to refer to a graph input or output with an index.
* Get rid of the "dummy" terminology which doesn't really help.
Previously, one would add "dummy nodes" which can then serve as input
and output nodes of the graph. Now one directly adds input and outputs
using `Graph.add_input` and `Graph.add_output`. There is one interface
node that contains all inputs and another one that contains all outputs.
Being able to refer to a graph input or output with an index makes it
more efficient to implement some algorithms. E.g. one could have a
bit span for a socket that contains all the information what graph
inputs this socket depends on.
Pull Request: https://projects.blender.org/blender/blender/pulls/112474
This is a light weight solution to passing in some extra context into
a lazy-function that is invoked by the graph executor.
The new functionality is used by #112421.
By storing a raw pointer instead of a `Span`, we save 16 bytes
per node state. I measured a ~5% speedup in my setup with
a simple repeat zone.
5c450aea05 added some additional asserts to check for valid
indices. Generally, index-errors in this area lead to wrong
behaviors of geometry nodes very quickly.
There are many small allocations when the graph executor is
initialized (e.g. all the node/sockets have to be allocated). Those
were already combined into a few allocations by making use
of `LinearAllocator`. However, even better performance can be
achieved by making one larger allocation and then using
preprocessed offsets into that buffer.
I measured up to 20% speedup in geometry nodes with a simple
repeat zone.
Including <iostream> or similar headers is quite expensive, since it
also pulls in things like <locale> and so on. In many BLI headers,
iostreams are only used to implement some sort of "debug print",
or an operator<< for ostream.
Change some of the commonly used places to instead include <iosfwd>,
which is the standard way of forward-declaring iostreams related
classes, and move the actual debug-print / operator<< implementations
into .cc files.
This is not done for templated classes though (it would be possible
to provide explicit operator<< instantiations somewhere in the
source file, but that would lead to hard-to-figure-out linker error
whenever someone would add a different template type). There, where
possible, I changed from full <iostream> include to only the needed
<ostream> part.
For Span<T>, I just removed print_as_lines since it's not used by
anything. It could be moved into a .cc file using a similar approach
as above if needed.
Doing full blender build changes include counts this way:
- <iostream> 1986 -> 978
- <sstream> 2880 -> 925
It does not affect the total build time much though, mostly because
towards the end of it there's just several CPU cores finishing
compiling OpenVDB related source files.
Pull Request: https://projects.blender.org/blender/blender/pulls/111046
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
There's quite a few libraries that depend on dna_type_offsets.h
but had gotten to it by just adding the folder that contains it to
their includes INC section without declaring a dependency to
bf_dna in the LIB section.
which occasionally lead to the lib building before bf_dna and the
header being missing, while this generally gets fixed in CMake by
adding bf_dna to the LIB section of the lib, however until last
week all libraries in the LIB section were linked as INTERFACE so
adding it in there did not resolve the build issue.
To make things still build, we sprinkled add_dependencies wherever
we needed it to force a build order.
This diff :
Declares public include folders for the bf_dna target so there's
no more fudging the INC section required to get to them.
Removes all dna related paths from the INC section for all
libraries.
Adds an alias target bf:dna to signify it has been updated to
modern cmake
Declares a dependency on bf::dna for all libraries that require it
Removes (almost) all calls to add_dependencies for bf_dna
Future work:
Because of the manual dependency management that was done, there is
now some "clutter" with libs depending on bf_dna that realistically
don't. Example bf_intern_opencolorio itself has no dependency on
bf_dna at all, doesn't need it, doesn't use it. However the
dna include folder had been added to it in the past since bf_blenlib
uses dna headers in some of its public headers and
bf_intern_opencolorio does use those blenlib headers.
Given bf_blenlib now correctly declares the dependency on bf_dna
as public bf_intern_opencolorio will get the dna header directory
automatically from CMake, hence some cleanup could be done for
bf_intern_opencolorio
Because 99% of the changes in this diff have been automated, this diff
does not seek to address these issues as there is no easy way to
determine why a certain dependency is in place. A developer will have
to make a pass a this at some later point in time. As I'd rather not
mix automated and manual labour.
There are a few libraries that could not be automatically processed
(ie bf_blendthumb) that also will need this manual look-over.
Pull Request: https://projects.blender.org/blender/blender/pulls/109835
Add a quaternion attribute type that will be used in combination with
rotation sockets for geometry nodes to give a more intuitive experience
and better performance when using rotations.
The most interesting part is probably the interpolation, the rest is
the same as the last attribute type addition, 988f23cec3.
We need to interpolate multiple values with different weights.
Based on Sybren's suggestion, this uses the `expmap` methods from
4805a54525 for that.
This also refactors `SimpleMixerWithAccumulationType` to use a
function rather than a cast to convert to the accumulation type.
See #92967
Pull Request: https://projects.blender.org/blender/blender/pulls/108678
A lot of files were missing copyright field in the header and
the Blender Foundation contributed to them in a sense of bug
fixing and general maintenance.
This change makes it explicit that those files are at least
partially copyrighted by the Blender Foundation.
Note that this does not make it so the Blender Foundation is
the only holder of the copyright in those files, and developers
who do not have a signed contract with the foundation still
hold the copyright as well.
Another aspect of this change is using SPDX format for the
header. We already used it for the license specification,
and now we state it for the copyright as well, following the
FAQ:
https://reuse.software/faq/
Goals of this refactor:
* Reduce memory consumption of `IndexMask`. The old `IndexMask` uses an
`int64_t` for each index which is more than necessary in pretty much all
practical cases currently. Using `int32_t` might still become limiting
in the future in case we use this to index e.g. byte buffers larger than
a few gigabytes. We also don't want to template `IndexMask`, because
that would cause a split in the "ecosystem", or everything would have to
be implemented twice or templated.
* Allow for more multi-threading. The old `IndexMask` contains a single
array. This is generally good but has the problem that it is hard to fill
from multiple-threads when the final size is not known from the beginning.
This is commonly the case when e.g. converting an array of bool to an
index mask. Currently, this kind of code only runs on a single thread.
* Allow for efficient set operations like join, intersect and difference.
It should be possible to multi-thread those operations.
* It should be possible to iterate over an `IndexMask` very efficiently.
The most important part of that is to avoid all memory access when iterating
over continuous ranges. For some core nodes (e.g. math nodes), we generate
optimized code for the cases of irregular index masks and simple index ranges.
To achieve these goals, a few compromises had to made:
* Slicing of the mask (at specific indices) and random element access is
`O(log #indices)` now, but with a low constant factor. It should be possible
to split a mask into n approximately equally sized parts in `O(n)` though,
making the time per split `O(1)`.
* Using range-based for loops does not work well when iterating over a nested
data structure like the new `IndexMask`. Therefor, `foreach_*` functions with
callbacks have to be used. To avoid extra code complexity at the call site,
the `foreach_*` methods support multi-threading out of the box.
The new data structure splits an `IndexMask` into an arbitrary number of ordered
`IndexMaskSegment`. Each segment can contain at most `2^14 = 16384` indices. The
indices within a segment are stored as `int16_t`. Each segment has an additional
`int64_t` offset which allows storing arbitrary `int64_t` indices. This approach
has the main benefits that segments can be processed/constructed individually on
multiple threads without a serial bottleneck. Also it reduces the memory
requirements significantly.
For more details see comments in `BLI_index_mask.hh`.
I did a few tests to verify that the data structure generally improves
performance and does not cause regressions:
* Our field evaluation benchmarks take about as much as before. This is to be
expected because we already made sure that e.g. add node evaluation is
vectorized. The important thing here is to check that changes to the way we
iterate over the indices still allows for auto-vectorization.
* Memory usage by a mask is about 1/4 of what it was before in the average case.
That's mainly caused by the switch from `int64_t` to `int16_t` for indices.
In the worst case, the memory requirements can be larger when there are many
indices that are very far away. However, when they are far away from each other,
that indicates that there aren't many indices in total. In common cases, memory
usage can be way lower than 1/4 of before, because sub-ranges use static memory.
* For some more specific numbers I benchmarked `IndexMask::from_bools` in
`index_mask_from_selection` on 10.000.000 elements at various probabilities for
`true` at every index:
```
Probability Old New
0 4.6 ms 0.8 ms
0.001 5.1 ms 1.3 ms
0.2 8.4 ms 1.8 ms
0.5 15.3 ms 3.0 ms
0.8 20.1 ms 3.0 ms
0.999 25.1 ms 1.7 ms
1 13.5 ms 1.1 ms
```
Pull Request: https://projects.blender.org/blender/blender/pulls/104629
The main goal here is to reduce the number of times thread-local data has
to be looked up using e.g. `EnumerableThreadSpecific.local()`. While this
isn't a bottleneck in many cases, it is when the action performed on the local
data is very short and that happens very often (e.g. logging used sockets
during geometry nodes evaluation).
The solution is to simply pass the thread-local data as parameter to many
functions that use it, instead of looking it up in those functions which
generally is more costly.
The lazy-function graph executor now only looks up the local data if
it knows that it might be on a new thread, otherwise it uses the local data
retrieved earlier.
Alongside with `UserData` there is `LocalUserData` now. This allows users
of the lazy-function evaluation (such as geometry nodes) to have custom
thread-local data that is passed to all the lazy-functions automatically.
This is used for logging now.
This type will be used to store mesh edges in #106638, but it could
be used for anything else too. This commit adds support for:
- The new type in the Python API
- Editing the type in the edit mode "Attribute Set" operator
- Rendering the type in EEVEE and Cycles for all geometry types
- Geometry nodes attribute interpolation and mixing
- Viewing the type in the spreadsheet and using row filters
The attribute uses the `blender::int2` type in most code, and
the `vec2i` DNA type in C code when necessary. The enum names
are based on `INT32_2D` for consistency with `INT8` and `INT32`.
Pull Request: https://projects.blender.org/blender/blender/pulls/106677
Hans Goudey
Jacques Lucke
Brecht Van Lommel
Jeroen Bakker
Ray molenkamp
Lukas Tönne
Iliya Katueshenock
Campbell Barton
Iliya Katueshenock
Aras Pranckevicius
Ray Molenkamp
Hans Goudey
Sergey Sharybin