This patch adds support for full precision compositing for the Realtime
Compositor. A new precision option was added to the compositor to change
between half and full precision compositing, where the Auto option uses
half for the viewport compositor and the interactive render compositor,
while full is used for final renders.
The compositor context now need to implement the get_precision() method
to indicate its preferred precision. Intermediate results will be stored
using the context's precision, with a number of exceptions that can use
a different precision regardless of the context's precision. For
instance, summed area tables are always stored in full float results
even if the context specified half float. Conversely, jump flooding
tables are always stored in half integer results even if the context
specified full. The former requires full float while the latter has no
use for it.
Since shaders are created for a specific precision, we need two variants
of each compositor shader to account for the context's possible
precision. However, to avoid doubling the shader info count and reduce
boilerplate code and development time, an automated mechanism was
employed. A single shader info of whatever precision needs to be added,
then, at runtime, the shader info can be adjusted to change the
precision of the outputs. That shader variant is then cached in the
static cache manager for future processing-free shader retrieval.
Therefore, the shader manager was removed in favor of a cached shader
container in the static cache manager.
A number of utilities were added to make the creation of results as well as
the retrieval of shader with the target precision easier. Further, a
number of precision-specific shaders were removed in favor of more
generic ones that utilizes the aforementioned shader retrieval
mechanism.
Pull Request: https://projects.blender.org/blender/blender/pulls/113476
Design task: #93551
This PR replaces the auto smooth option with a geometry nodes modifier
that sets the sharp edge attribute. This solves a fair number of long-
standing problems related to auto smooth, simplifies the process of
normal computation, and allows Blender to automatically choose between
face, vertex, and face corner normals based on the sharp edge and face
attributes.
Versioning adds a geometry node group to objects with meshes that had
auto-smooth enabled. The modifier can be applied, which also improves
performance.
Auto smooth is now unnecessary to get a combination of sharp and smooth
edges. In general workflows are changed a bit. Separate procedural and
destructive workflows are available. Custom normals can be used
immediately without turning on the removed auto smooth option.
**Procedural**
The node group asset "Smooth by Angle" is the main way to set sharp
normals based on the edge angle. It can be accessed directly in the add
modifier menu. Of course the modifier can be reordered, muted, or
applied like any other, or changed internally like any geometry nodes
modifier.
**Destructive**
Often the sharp edges don't need to be dynamic. This can give better
performance since edge angles don't need to be recalculated. In edit
mode the two operators "Select Sharp Edges" and "Mark Sharp" can be
used. In other modes, the "Shade Smooth by Angle" controls the edge
sharpness directly.
### Breaking API Changes
- `use_auto_smooth` is removed. Face corner normals are now used
automatically if there are mixed smooth vs. not smooth tags. Meshes
now always use custom normals if they exist.
- In Cycles, the lack of the separate auto smooth state makes normals look
triangulated when all faces are shaded smooth.
- `auto_smooth_angle` is removed. Replaced by a modifier (or operator)
controlling the sharp edge attribute. This means the mesh itself
(without an object) doesn't know anything about automatically smoothing
by angle anymore.
- `create_normals_split`, `calc_normals_split`, and `free_normals_split`
are removed, and are replaced by the simpler `Mesh.corner_normals`
collection property. Since it gives access to the normals cache, it
is automatically updated when relevant data changes.
Addons are updated here: https://projects.blender.org/blender/blender-addons/pulls/104609
### Tests
- `geo_node_curves_test_deform_curves_on_surface` has slightly different
results because face corner normals are used instead of interpolated
vertex normals.
- `bf_wavefront_obj_tests` has different export results for one file
which mixed sharp and smooth faces without turning on auto smooth.
- `cycles_mesh_cpu` has one object which is completely flat shaded.
Previously every edge was split before rendering, now it looks triangulated.
Pull Request: https://projects.blender.org/blender/blender/pulls/108014
Currently object bounds (`object.runtime.bb`) are lazily initialized
when accessed. This access happens from arbitrary threads, and
is unprotected by a mutex. This can cause access to stale data at
best, and crashes at worst. Eager calculation is meant to keep this
working, but it's fragile.
Since e8f4010611, geometry bounds are cached in the geometry
itself, which makes this object-level cache redundant. So, it's clearer
to build the `BoundBox` from those cached bounds and return it by
value, without interacting with the object's cached bounding box.
The code change is is mostly a move from `const BoundBox *` to
`std::optional<BoundBox>`. This is only one step of a larger change
described in #96968. Followup steps would include switching to
a simpler and smaller `Bounds` type, removing redundant object-
level access, and eventually removing `object.runtime.bb`.
Access of bounds from the object for mesh, curves, and point cloud
objects should now be thread-safe. Other object types still lazily
initialize the object `BoundBox` cache since they don't have
a data-level cache.
Pull Request: https://projects.blender.org/blender/blender/pulls/113465
The experimental GPU compositor is leaking memory in any setup.
This is because the current implementation of the render texture pool
always created a new texture and only freed the textures upon deletion.
This was a temporary implementation until a proper implementation that
uses the DRW textures pool was used.
This patch implements a small texture pool as a temporary fix until the
aforementioned DRW texture pool implementation is done.
The GPU compositor crops the viewed images to the render resolution.
While the original size and content of the input to the viewer should be
retained as is.
This patch fixes that by specializing compositors that can use composite
outputs to be able to view images of any arbitrary size. This is still
missing the translation offset of the viewer, but this shall be tackled
separately.
The experimental GPU compositor always returned the first view in
multi-view rendering. This patch fixes that by also checking for the
view name of the context when searching for the appropriate pass.
Don't assume existence of GPU backend in (background) preview rendering.
Also add null pointer checks and rely on assert instead to detect
invalid usage of GPU_render_begin/end, so that potential future mistakes
don't cause crashes.
Pull Request: https://projects.blender.org/blender/blender/pulls/112971
This adds initial support for rendering Cycles and EEVEE shaders in Hydra
render engines that support MaterialX. Not all nodes are currently
supported, see the detailed compatibility list in #112864.
Co-authored-by: Georgiy Markelov <georgiy.m.markelov@gmail.com>
Co-authored-by: Vasyl Pidhirskyi <vpidhirskyi@gmail.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/111765
This simplifies running built-in IO tests with:
ctest -R bf_io_
Also use "bf_io_" prefix for the libraries since it was already used
by some and it's a useful hint the libraries are used for IO.
A mistake in some of the previous refactor which was aimed to make the
byte buffer to be stored as uint8_t. One of the array size calculation
was missing multiplication by 4 channels.
Pull Request: https://projects.blender.org/blender/blender/pulls/112508
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
Also semantically separate draw_lock and draw_unlock, as it
is more clear than a single method with a boolean argument.
Should be no functional changes.
Use appropriate function for converting paths. This adds an ugly
dependency on a function that is private to python/, similar to how it
is done for Freestyle. Solving that will be for a separate change.
Ref #110765
Using ClangBuildAnalyzer on the whole Blender build, it was pointing
out that BLI_math.h is the heaviest "header hub" (i.e. non tiny file
that is included a lot).
However, there's very little (actually zero) source files in Blender
that need "all the math" (base, colors, vectors, matrices,
quaternions, intersection, interpolation, statistics, solvers and
time). A common use case is source files needing just vectors, or
just vectors & matrices, or just colors etc. Actually, 181 files
were including the whole math thing without needing it at all.
This change removes BLI_math.h completely, and instead in all the
places that need it, includes BLI_math_vector.h or BLI_math_color.h
and so on.
Change from that:
- BLI_math_color.h was included 1399 times -> now 408 (took 114.0sec
to parse -> now 36.3sec)
- BLI_simd.h 1403 -> 418 (109.7sec -> 34.9sec).
Full rebuild of Blender (Apple M1, Xcode, RelWithDebInfo) is not
affected much (342sec -> 334sec). Most of benefit would be when
someone's changing BLI_simd.h or BLI_math_color.h or similar files,
that now there's 3x fewer files result in a recompile.
Pull Request #110944
Add a High Dynamic Range option in the Color Management > Display panel.
This enables display of extended color ranges above 1.0 for the 3D
viewport, image editor and render previews.
This requires a monitor that can display HDR colors, and a view
transform designed for HDR output. The Standard view transform works,
but Filmic does not as it was designed to bring values into the 0..1
range for SDR displays.
This patch is limited to allowing the display to visualize extended
colors, but does not include future looking work to better integrate HDR
into the full workflow.
It is implemented by rendering to high bit-depth texture formats for
the user interface, and uncapping the color range in color management.
Authored by Apple: Michael Parkin-White
Pull Request: https://projects.blender.org/blender/blender/pulls/105662
First implementation of node previews in the shader node editor. Using
the same user interface as compositor node previews, most shader nodes
can now be previewed (except group in/output and material output).
This is currently still an experimental feature, as polishing of the
user experience and performance improvements are planned. These will
be easier to do as incremental changes on this implementation.
See #110353 for details on the work that remains to be done and known
limitations.
Implementation notes:
We take advantage of the `RenderResult` available as `ImBuf` images to
store a `Render` for every viewed nested node tree present in a
`SpaceNode`. The computation is initiated at the moment of drawing nodes
overlays.
One render is started for the current nodetree, having a `ViewLayer`
associated with each previewed node. We separate the previewed nodes in
two categories: the shader ones and the non-shader ones.
- For non-shader nodes, we use AOVs which highly speed up the rendering
process by rendering every non-shader nodes at the same time. They are
rendered in the first `ViewLayer`.
- For shader nodes, we render them each in a different `ViewLayer`, by
rerouting the node to the output of the material in the preview scene.
The preview scene takes the same aspect as the Material preview scene,
and the same preview object is used.
At the moment of drawing the node overlay, we take the `Render` of the
viewed node tree and extract the `ImBuf` of the wanted viewlayer/pass
for each previewed node.
Pull Request: https://projects.blender.org/blender/blender/pulls/110065
This is currently meant mainly for testing, when "Developer Extras" is
enabled. The goal is to make interactive Hydra export and USD file export
identical. We are not there yet, and having the ability to compare both
in the viewport and automated tests should help us get and stay there.
Ref #110765
Hydra is a rendering architecture part of USD, designed to abstract the
host application from the renderer. A renderer implementing a Hydra
render delegate can run in any host application supporting Hydra, which
now includes Blender.
For external renderers this means less code to be written, and improved
performance due to a using a C++ API instead of a Python API.
Add-ons need to subclass bpy.types.HydraRenderEngine. See the example in
the Python API docs for details.
An add-on for Hydra Storm will be included as well. This is USD's
rasterizing renderer, used in other applications like usdview. For users
it can provide a preview of USD file export, and for developers it
serves a reference.
There are still limitations and missing features, especially around
materials. The remaining to do items are tracked in #110765.
This feature was contributed by AMD.
Ref #110765
Co-authored-by: Georgiy Markelov <georgiy.m.markelov@gmail.com>
Co-authored-by: Vasyl-Pidhirskyi <vpidhirskyi@gmail.com>
Co-authored-by: Brian Savery <brian.savery@gmail.com>
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/104712
The C-style API is still in place, but the implementation
is moved to a virtual method.
This is part of unification of viewport and final render
structure.
Should be no functional changes.
Ref #108618
Pull Request: https://projects.blender.org/blender/blender/pulls/110696
Historically, the OCIO based color management implementation in Blender
had exceptions to treat specific configurations differently. It was a
compatibility with the legacy "No color management" option.
With time and more development in the area there are better ways of
achieving this goal, if needed.
This commit removes the named-based exception, which also solves confusion
about why certain similar configurations (from OCIO stand point) give
different results. As well as allows to create a cleaner plate for an
upcoming additions in the OCIO configuration such as AgX.
Quite simple and technical change which constant-folds the check for
whether the scene color management enabled or not with "true" value.
Ref #110685
Pull Request: https://projects.blender.org/blender/blender/pulls/110580
This change fixes crash which happens when a viewer node is used for
backdrop, and the scene render size is modified. After the modification
the render size and the texture size gets out of sync since the texture
was never adapting for the size change.
Pull Request: https://projects.blender.org/blender/blender/pulls/110590
Implements the rest of #101689, after 5e9ea9243b.
- `vdata` -> `vert_data`
- `edata` -> `edge_data`
- `pdata` -> `face_data`
- `ldata` -> `loop_data`
A deeper rename of `loop` to `corner` will be proposed as a next
step, and renaming `totvert` and `totedge` can be done separately.
Pull Request: https://projects.blender.org/blender/blender/pulls/110432
The cleanup of blenkernel last weeks , caused the house of cards to
collapse on top of bf_gpu's shader_builder, which is off by default
but used on a daily basis by the rendering team.
Given the fixes forward in #110394 ran into a ODR violation in OSL that
was hiding there for years, I don't see another way forward without
impeding the rendering teams productivity for "quite a while" as there
is no guarantee the OSL issue would be the end of it.
the only way forward appears to be back.
this reverts :
19422044eda670b53abe0f541db97cbe516e8c813e88a2f44c4e64b772f59547e7a31707fe6c5a57
The problematic commit was 07fe6c5a57
as blenkernel links most of blender, it's a bit of a link order issue
magnet. Given all these commits stack, it's near impossible to revert
just that one without spending a significant amount of time resolving
merge conflicts. 99% of that work was automated, so easier to just
revert all of them, and re-do the work, than it is to deal with the
merge conflicts.
Pull Request: https://projects.blender.org/blender/blender/pulls/110438
Implements part of #101689.
The "poly" name was chosen to distinguish the `MLoop` + `MPoly`
combination from the `MFace` struct it replaced. Those two structures
persisted together for a long time, but nowadays `MPoly` is gone, and
`MFace` is only used in some legacy code like the particle system.
To avoid unnecessarily using a different term, increase consistency
with the UI and with BMesh, and generally make code a bit easier to
read, this commit replaces the `poly` term with `poly`. Most variables
that use the term are renamed too. `Mesh.totface` and `Mesh.fdata` now
have a `_legacy` suffix to reduce confusion. In a next step, `pdata`
can be renamed to `face_data` as well.
Pull Request: https://projects.blender.org/blender/blender/pulls/109819
This change replaces a bare RenderEngine owned by a viewport
with a VeiwRender. This unlocks a possibility of accessing
RenderResult for viewport renders. Currently it is not done,
but it will be needed for an upcoming work towards unification
of the render passes handling.
Ref #108618
Pull Request: https://projects.blender.org/blender/blender/pulls/110244
Currently no functional changes.
Preparing for introduction of a Render structure for the viewport
render which will hold both engine and the render result for
passes access.
Use proper allocation and destruction for it.
Since the allocation is no longer zero-initialized make sure the
fields are explicitly zeroed out. This also allows to use an easier
way to initialize mutexes.
Currently no functional changes, preparing for a bigger refactor.
Used for the global list of the Render structures.
Using C++ container helps moving towards reliable "real" C++
structure for the Render, without worrying about the offset of
the next/prev fields.
Should be no functional changes on the user side.
This formats code that is disabled using `#if 0`. Formatting was achieved
by temporarily changing `#if 0` to `#if 1 /*something*/`, then formatting,
and then changing it back to `#if 0`.
Doing so avoids having duplicated logic for working with pixel
data which is being passed throughout the render pipeline.
Notable changes:
- ImBug can now store GPU texture.
This is not very finished part of the API, which will be
worked further to support tiling for very-high-res images.
- Implicit sharing is removed from the image buffer, as it is
no longer needed.
There should be no functional changes on user level with this
change.
Ref #108618
Pull Request: https://projects.blender.org/blender/blender/pulls/109788
There's quite a few libraries that depend on dna_type_offsets.h
but had gotten to it by just adding the folder that contains it to
their includes INC section without declaring a dependency to
bf_dna in the LIB section.
which occasionally lead to the lib building before bf_dna and the
header being missing, while this generally gets fixed in CMake by
adding bf_dna to the LIB section of the lib, however until last
week all libraries in the LIB section were linked as INTERFACE so
adding it in there did not resolve the build issue.
To make things still build, we sprinkled add_dependencies wherever
we needed it to force a build order.
This diff :
Declares public include folders for the bf_dna target so there's
no more fudging the INC section required to get to them.
Removes all dna related paths from the INC section for all
libraries.
Adds an alias target bf:dna to signify it has been updated to
modern cmake
Declares a dependency on bf::dna for all libraries that require it
Removes (almost) all calls to add_dependencies for bf_dna
Future work:
Because of the manual dependency management that was done, there is
now some "clutter" with libs depending on bf_dna that realistically
don't. Example bf_intern_opencolorio itself has no dependency on
bf_dna at all, doesn't need it, doesn't use it. However the
dna include folder had been added to it in the past since bf_blenlib
uses dna headers in some of its public headers and
bf_intern_opencolorio does use those blenlib headers.
Given bf_blenlib now correctly declares the dependency on bf_dna
as public bf_intern_opencolorio will get the dna header directory
automatically from CMake, hence some cleanup could be done for
bf_intern_opencolorio
Because 99% of the changes in this diff have been automated, this diff
does not seek to address these issues as there is no easy way to
determine why a certain dependency is in place. A developer will have
to make a pass a this at some later point in time. As I'd rather not
mix automated and manual labour.
There are a few libraries that could not be automatically processed
(ie bf_blendthumb) that also will need this manual look-over.
Pull Request: https://projects.blender.org/blender/blender/pulls/109835
Also fix some incorrect usages of `N_` macro instead of `TIP_` one
(these error messages typically need to get translated explicitely, not
only marked for extraction).
This introduces an alias target `bf::intern::atomic` for
`bf_intern_atomic`. This has the following benefits:
- Any target name with `::` in it will be recognized as an actual
target by cmake, rather than a library name it may not know about.
and will be validated by cmake to exist. Which means if you make
a typo in the LIB section, CMake will error out telling you it
doesn't know about this specific target rather than passing it on
to the build system, where you'll either get build or linker errors
because of said typo.
- Given there is quite a cleanup still to do in the build system,
it won't always be obvious which targets have been updated to
modern targets and which still need to be done. Having a namespaced
target name is a good indicator there.
Pull Request: https://projects.blender.org/blender/blender/pulls/109784
The Texture margin 'adjacent faces' algorithm always used the
full UV map, even if not all polygons were actually part of the
baking. Remedy this by checking the mask if passed in.
Pull Request: https://projects.blender.org/blender/blender/pulls/109500
It was only used by OpenEXR and Iris images, and saving the Z Buffer
in those formats was disabled by default. This option comes from the
times prior to the addition of the Multilayer EXR.
It also worth noting that it was not possible to save Iris with Depth
pass from Blender as internally it is called IRIZ format and it was
not exposed. But even after exposing this format option something still
was missing as saving and loading ITIZ did not show up the Depth pass.
The reason of removal is to make it a more clear match of the ImBuf
with a render pass, and use it instead of a custom type in the render
result and render pass API. This will simplify the API and also avoid
stealing buffers and making shallow copies when showing the render
result.
For the cases when Depth is needed a Multilayer EXR is to be used,
as most likely more than just the Depth will be needed.
On a user level this change:
- Removes the "Z Buffer" option from the interface.
- It preserves existing sockets in compositor nodes, but it will
output black image. Also changing the image data-block will
remove the socket unless a Multilayer EXR with Depth pass image
is selected.
- Removes "Depth" socket of the Viewer and Composite nodes.
Ref #108618
Pull Request: https://projects.blender.org/blender/blender/pulls/109687
Since UDIM baking support in 6787cc13d4, the normalization of
diplacement heights was always based on the min/max height detected in
the _last_ tile, which could lead to clipping if the last tile had very
subtle (or no) displacement.
Now getting the min/max is spread across all images.
This also takes the first thread into account for getting the min/max (which for some reason was skipped).
Pull Request: https://projects.blender.org/blender/blender/pulls/109409
This is cached in Render, and gets cleared along with render pass GPU
textures when there is no editor open using it, or a new final render is
started.
The context and texture pool are cached. But the evaluator is re-created
every time as this only runs on compositing node changes, which require
recreating it anyway (unlike the viewport where e.g. camera navigation
does not need a new evaluator).
Pull Request: https://projects.blender.org/blender/blender/pulls/108909
Before this change the ImBuf struct had dedicated fields for the
buffer data. Now the color space is stored inside of the struct
which wraps around the buffer information.
This only changes the field placement, without changing the way
it is handled. In the future one might imagine that operations
like stealing buffer data should null-ify the buffer colorspace
pointer. Such changes would need to have more accurate thinking
before implementation.
Should be no functional changes.
Pull Request: https://projects.blender.org/blender/blender/pulls/109291
This patch adds support for node previews in the realtime compositor.
Only node operations have previews for now. Shader nodes likes the
MixRGB node does not have previews implemented yet due to required
sizable changes in the node compiler.
Depends on: #108900.
Pull Request: https://projects.blender.org/blender/blender/pulls/108904
`AUDIO_MUTE` flag of `scene->audio.flag` works as intended and tooltip
for this property is not correct. However muting scene audio for
playback and rendering is not be very useful, since users can render
with no audio explicitly or adjust scene volume.
Only mute audio if `DAG_EVAL_VIEWPORT` mode is used.
* Store per RenderPass in RenderResult.
* Caches are cleared when starting rendering, to make more memory available
to GPU rendering.
* Caches are cleared on UI changes, when no compositing node editor and no
image editor with a render result or viewer node image is visible.
* Store 3 channel RGB passes as such, and set alpha 1 in shader.
This is an intermediate step before implementing GPU backed ImBuf, to
improve performance and figure out cache eviction.
Pull Request: https://projects.blender.org/blender/blender/pulls/108818
This patch adds support for Viewer and File Output nodes to the realtime
compositor. The experimental render GPU compositor was also extended to
support viewers. While support for File Output nodes was added, it
remains unimplemented.
This is just an experimental implementation, the logic for viewers will
probably be changed once #108656 is agreed upon. Furthermore, the recalc
NODE_DO_OUTPUT_RECALC flags need to be taken into account to avoid
superfluous computations.
Pull Request: https://projects.blender.org/blender/blender/pulls/108804
* opengl_context -> system_gpu_context. This is the operating system OpenGL,
Metal or Vulkan context provided by GHOST.
* gpu_context -> blender_gpu_context. This is the GPUContext provided by
the Blender GPU module, which wraps the GHOST context and adds some state.
* Various functions create/destroy/enable/disable both contexts, these have
just gpu_context in the name now.
Pull Request: https://projects.blender.org/blender/blender/pulls/108723
* Enable "Experimental Compositors" in preferences, then choose
Realtime GPU execution mode in node editor sidebar.
* Only supports combined pass input and Render Result combined output.
* No viewer nodes, no file output nodes, and no node previews yet.
Pull Request: https://projects.blender.org/blender/blender/pulls/108629
A lot of files were missing copyright field in the header and
the Blender Foundation contributed to them in a sense of bug
fixing and general maintenance.
This change makes it explicit that those files are at least
partially copyrighted by the Blender Foundation.
Note that this does not make it so the Blender Foundation is
the only holder of the copyright in those files, and developers
who do not have a signed contract with the foundation still
hold the copyright as well.
Another aspect of this change is using SPDX format for the
header. We already used it for the license specification,
and now we state it for the copyright as well, following the
FAQ:
https://reuse.software/faq/
Combine the newer less efficient C++ implementations and the older
less convenient C functions. The maps now contain one large array of
indices, split into groups by a separate array of offset indices.
Though performance of creating the maps is relatively unchanged, the
new implementation uses 4 bytes less per source element than the C
maps, and 20 bytes less than the newer C++ functions (which also
had more overhead with larger N-gons). The usage syntax is simpler
than the C functions as well.
The reduced memory usage is helpful for when these maps are cached
in the near future. It will also allow sharing the offsets between
maps for different domains like vertex to corner and vertex to face.
A simple `GroupedSpan` class is introduced to make accessing the
topology maps much simpler. It combines offset indices and a separate
span, splitting it into chunks in an efficient way.
Pull Request: https://projects.blender.org/blender/blender/pulls/107861