Exceptions:
* Links to personal wiki pages
* Pages that are not in the new developer docs yet (like Human Interface Guidelines)
* tools\check_wiki\check_wiki_file_structure.py needs a refactor
Using pooljobs with default settings should never have any significant impact
on the build speed, and it makes building full debug with sanitizer builds
safe on (almost) all machines.
Quick test showed no significant difference in Release build time with or
without Ninja pooljobs (on linux, with a 16 cores, 64GBb machine).
Update the regular jobs amount computation to follow the same logic as
for the heavy ones. the main difference is that it uses a '2Gb of RAM
per job' base value.
This change is mainly targetted at machines with a relatively low
RAM/cores ratio, since even regular compile jobs can end up using quite
a lot of RAM if many are running in parallel, previous defaults would
likely not work well on machines with e.g. 16Gb of RAM and 16 cores.
Also fix a typo in previous commit (6493d0233c), sorry about that.
This commit simplifies and makes more generic the computation of the
maximum number of parallel heavy build jobs. Essentially, it allows 1
heavy job per 8Gb of RAM.
It also systematically sets the amount of heavy jobs, since we are going
to get more of these in the future (like the 'unity build' units), the
previous heuristic had some loose ends (e,g for a 40Gb RAM, 16 threads
machine, it would not set any limit to heavy jobs, yet said machine
would likely not be able to run 16 3.5+Gb heavy jobs in parallel...).
This is some initial step towards a better handling of 'sanitizer' builds
on the Blender buildbot.
Bundling many tests in a single binary reduces build time and disk space
usage, but is less convenient for running individual tests command line
as filter flags need to be used.
This adds WITH_TESTS_SINGLE_BINARY to generate one executable file per
source file. Note that enabling this option requires a significant amount
of disk space.
Due to refactoring, the resulting ctest names are a bit different than
before. The number of tests is also a bit different depending if this
option is used, as one uses gtests discovery and the other is organized
purely by filename, which isn't always 1:1.
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/114604
This change solves the following linker error:
ld: warning: __eh_frame section too large (max 16MB) to encode dwarf unwind offsets in compact unwind table
Proposed solution is to disable unwind table when building Blender with
address sanitizer enabled.
It is stated in the comment in the code, but to make it explicit there
could some side-effects of code which relies on frame walking:
- backtrace()
- __attribute__((__cleanup__(f)))
- __builtin_return_address(n), for n > 0
- pthread_cleanup_push when it is implemented using
__attribute__((__cleanup__(f)))
From the local tests it seems that backtrace() provides the same
output as prior to this change (but with classic linker used, as prior
to this change it is not possible to link Blender).
The rest of the possibly functionality is not used by Blender, but
it is a bit hard to tell if it is used by any of the dependent
libraries. However, if the libraries are compiled dynamically, there
will be no affect on them with this change.
I am unable to run the full test suit as some of the tests are
failing prior to this change with classic linker. Overall it seems
to be no unwanted side effects on Blender development.
Note that the change only affects debug builds with ASAN enabled,
so it is a low risk of causing some real problem so might as well
just give it a whirl and see if some unpredicted issue arises.
Pull Request: https://projects.blender.org/blender/blender/pulls/116745
The build scripts are still referring to gpu tests as being opengl.
Although they can also use Metal or Vulkan. This PR only replaces
the work `opengl` with `gpu` for build options.
Special note is that the windows argument `with_opengl_tests` is
also replaced with `with_gpu_tests` for consistency.
Pull Request: https://projects.blender.org/blender/blender/pulls/116030
Asserts historically only have been on in debug builds, however
running the tests in a debug configuration takes a while especially
if Cycles is enabled.
This adds WITH_ASSERT_RELEASE to enable asserts in release
configurations. It is enabled for "make developer" , and will also
be used for CI.
Co-authored-by: Brecht Van Lommel <brecht@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/115071
NDEBUG is part of the C standard and disables asserts. Only this will
now be used to decide if asserts are enabled.
DEBUG was a Blender specific define, that has now been removed.
_DEBUG is a Visual Studio define for builds in Debug configuration.
Blender defines this for all platforms. This is still used in a few
places in the draw code, and in external libraries Bullet and Mantaflow.
Pull Request: https://projects.blender.org/blender/blender/pulls/115774
Harfbuzz and FriBiDi are included in our external libraries for all
platforms. This PR adds the glue to make them available as optional
build components (off by default).
Pull Request: https://projects.blender.org/blender/blender/pulls/114947
Some tests like cycles, sequencer and compositor batch together multiple
tests in a single Blender invocation. This makes them run faster, but
makes debugging harder. This is an option to disable that batching.
Pull Request: https://projects.blender.org/blender/blender/pulls/114603
User a Blender wrapper `tests/utils/blender_headless.py` to runs a
graphical instance of Blender within a headless weston compositor.
Currently only WAYLAND is supported as a back-end, support for other
platforms is possible. The tests can run from X11 since the tests don't
depend on existing instances of X11 or WAYLAND.
- Each test runs a separate headless instance of WESTON
since the overhead is minimal, this allows tests to run in parallel
without interfering with each other.
- There is a CMake option WESTON_BIN, when left empty the weston
from LIBDIR is used. Otherwise this can point to the weston binary
installed on the users system.
- In most cases simulated events are needed to implement these tests
(running blender with `--enable-event-simulate`).
- This commit adds 14 undo tests - simulating user interaction as well
as undo/redo actions, ensuring the desired result is reached.
Other kinds of UI tests could be added in the future.
Ref !114164
The last good commit was 8474716abb.
After this commits from main were pushed to blender-v4.0-release. These are
being reverted.
Commits a4880576dc from to b26f176d1a that happend afterwards were meant for
4.0, and their contents is preserved.
With current main code, trying to build a full debug build with ASAN
enabled will fail at linking, due to out-of-bound memory references
within the binary file, e.g.:
```
mold: error: /usr/lib/gcc/x86_64-linux-gnu/13/crtbegin.o:(.text):
relocation R_X86_64_32S against .tm_clone_table out of range:
2147741344 is not in [-2147483648, 2147483648)
```
This commit works around the issue by disabling ASAN by default for the
`extern` dependencies. A new `WITH_COMPILER_ASAN_EXTERN` CMake
option is added to enable it if needed - some other components need to be
disabled then.
NOTE: This is more of an emergency band-aid to the general binary size issue,
see #113892 for on-going discussions about better solutions in the long term.
Pull Request: https://projects.blender.org/blender/blender/pulls/113891
Tested with IBUS on GNOME 45.
Added a capabilities flag to GHOST since support for IME works on
Wayland but not on X11, so runtime detection is needed.
CYCLES_ONEAPI_SPIR64_GEN_DEVICES -> CYCLES_ONEAPI_INTEL_BINARIES_ARCH
so it's more aligned with the name for the other backends, such as
CYCLES_CUDA_BINARIES_ARCH.
This was only used for accessing cursor themes which only worked
with gnome and wasn't used in official releases.
Use the default theme or the theme defined by XCURSOR_THEME.
Eventually wp_cursor_shape_manager_v1 can be supported which avoids
having to access the theme.
When WITH_GHOST_SDL or WITH_HEADLESS were used, the message didn't make
much sense, especially since the features warned about weren't
necessarily enabled or even supported by the platform.
Replace with a `set_and_warn_incompatible` macro which only reports
configuration changes based on incompatible features.
Printing that a library is found every time CMake runs isn't helpful.
Restrict these messages for the first execution so messages are limited
to information developers may need to know such as features being
disabled because of incompatible configurations.
This PR enables vulkan backend as experimental option.
It will only be available in alpha builds on Linux and Windows.
This option is highly experimental and enabled to get some insight
on supported platforms. Don't expect a fully working Blender
yet. Also don't expect it to have usable performance.
**What is known to not work?**
* OCIO textures are not supported on Intel and AMD GPUs. sRGB/Standard is supported
on those platforms.
* AMD Polaris based GPUs on Linux will generate a crash when drawing the 3d cursor as it
doesn't support the needed vertex format. Comment out `DRW_draw_cursor` in `DRW_draw_region_info`.
* The colors in the node editor and sequencer are of as sRGB viewports aren't detected correctly.
* The image / UV editor isn't working as many texture formats haven't been tested yet. Some
tweaks are also needed to do correct depth testing.
* 3D Viewport is known to be flickering. Sometimes workbench doesn't display anything.
* 3D Viewport wireframe will crash as it uses a framebuffer with gaps between color attachments,
which isn't supported yet. (#113141)
* Rotate the view widget is partially drawn due to incompatible depth clipping.
* GPU Selection isn't working. It is expected to be solved when Overlay-Next will become the
default engine. For now disable GPU depth picking in the preferences.
* Cycles/EEVEE are known to not work with Vulkan yet. Cycles requires Vulkan Pixel Buffer.
Cuda <-> Vulkan interop might require a different approach than OpenGL as Vulkan doesn't allow
importing memory from a Cuda context. EEVEE uses features that aren't available yet in the backend
* Workbench is working, except Workbench shadows.
* EEVEE-Next basics are working. Shadows, lights are known to be not working. Materials/Shading
works in simple scenes. Changes are expected in EEVEE-Next that will break Vulkan compatibility
in the near future.
* Systems with multiple GPUs is not expected to work.
* Wayland support is in development and requires some iterations. You can start Blender, but
the protocols are not aligned yet.
* OpenXR hasn't been modified and is expected to fail.
* The backend is very strict when mis-using the GPU module. In debug builds it may crash
on asserts.
* Older drivers/GPUs might not have all the features that we require. The workarounds
for the missing features still need to be implemented.
**A word about performance**
In the project planning we focus first on stability and platform support. The performance of Vulkan is
around 20% of what we want to achieve. The reason is that each command sent to the
GPU is done one at a time. The implementation even waits until we have feedback that the GPU
is idle again.
Geometry is currently stored in System RAM. The GPU will read and cache the data when
accessing geometry. This slows down when using objects with much geometry.
Some performance features like MDI (Multi-Draw-Indirect) hasn't been implemented and
falls back to Single Draw Indirect.
**Why enable it is an experimental option?**
* Ensures that new features are being tested with Vulkan
* Ensure that building with Vulkan is possible on supported platforms
* Get feedback from developers if Vulkan can run on their system or that
there are special cases that we are not aware of. Main development
environment has been Linux/X11 with occasionally testing using Windows.
* Validate Add-ons that use the `gpu` module.
* Possible to enable GLSL validation on the buildbot. (Needs more work).
* Does it compile on all machines or does it require more changes to cmake
config. We expect it to be able to compile without installing the Vulkan SDK.
The Vulkan SDK is a very powerful tool, but only when actually doing GPU
development. Otherwise it is an overhead which slows down other
activities.
**How can the backend be enabled?**
Currently the Vulkan backend can be enabled per Blender session by starting
using the command line argument `--gpu-backend vulkan`. In the future, when
the backend is more mature, we will add a user preference to switch between
OpenGL and Vulkan.
Pull Request: https://projects.blender.org/blender/blender/pulls/113057
The new Clang which came with XCode 15 introduced the new
warning: -Wsingle-bit-bitfield-constant-conversion. It is a bit
too noisy in the Bullet code.
Since the warning is in the code which we do not maintain suppress
it similar to other similar warnings.
Pull Request: https://projects.blender.org/blender/blender/pulls/113136
Tests must be enabled manually using the CMake flag `WITH_COMPOSITOR_REALTIME_TEST`. Reasons are F12 for realtime compositor is experimental and buildbots have no GPU. Failing tests are excluded.
Pull Request: https://projects.blender.org/blender/blender/pulls/109878
This renames the WITH_WINDOWS_PDB and WITH_WINDOWS_STRIPPED_PDB cmake
options to WITH_WINDOWS_RELEASE_PDB WITH_WINDOWS_RELEASE_STRIPPED_PDB
The Stripped PDB isn't cost free to generate, and is only needed for
builds that are distributed to end users. There is no benefit in making
one for a debug build as the debugger locally will prefer to use the
bigger un-stripped PDB anyhow.
This also stops the copy/install of the PDB for anything but a release
build, this file is about 1.6G for a debug build, and there is really
no need to do this for local development, as the debugger will find/use
the PDB from its original location.
This brings down the time needed for an incremental link on a debug
build by about 30% (10->7 seconds on my local system)
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.
While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.
Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.
Some directories in `./intern/` have also been excluded:
- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.
An "AUTHORS" file has been added, using the chromium projects authors
file as a template.
Design task: #110784
Ref !110783.
Campbell Barton
Thomas Dinges
Bastien Montagne
Brecht Van Lommel
Sergey Sharybin
Jeroen Bakker
Ray Molenkamp
Harley Acheson
bsavery
Xavier Hallade
Habib Gahbiche