Solves the issue of the script potentially sitting for a long time
without having any progress reported. Confusingly, such behavior
depends on Git version.
In older versions (< 2.33) there will be progress reported, but it
then got changed by the commit in Git:
7a132c628e
The delayed checkout is exactly how Git LFS integrates into the Git
process. Another affecting factor for the behavior is that submodule
configured to use "checkout" update policy is forced to have quite
flag passed to the "git checkout":
https://github.com/git/git/blob/v2.43.2/builtin/submodule--helper.c#L2258
This is done to avoid the long message at the end of checkout about
the detached state of HEAD, with instructions how to resolve that.
There are two possible solutions: either use "rebase" update policy
for submodules, or skip Git LFS download during the submodule update.
Changing the update policy is possible, but it needs to be done with
a bit of care, and possible revised process for updating/merging
tests data.
This change follows the second idea of delaying LFS download for a
later step, so the process is the following:
- Run `git submodule update`, but tell Git LFS to not resolve the links
by using GIT_LFS_SKIP_SMUDGE=1 environment variable.
- Run `git lfs pull` for the submodule, to resolve the links.
Doing so bypasses hardcoded silencing in the Git. It also potentially
allows to recover from an aborted download process.
The `git lfs pull` seems to be a nominal step to resolve the LFS links
after the smudging has been skipped. It is also how in earlier Git
versions some Windows limitations were bypassed:
https://www.mankier.com/7/git-lfs-faq
The submodule update now also receives the "--progress" flag, which
logs the initial Git repository checkout process, which further
improves the feedback.
The byproduct of this change is that an error during precompiled
libraries and tests data update is not considered to be fatal.
It seems to be more fitting with other update steps, and allows
more easily reuse some code.
There is also a cosmetic change: the messages at the end of the
update process now have their own header, allowing more easily
see them in the wall-of-text.
Pull Request: https://projects.blender.org/blender/blender/pulls/118673
Likewise, skip tests update when --use-tests is not provided.
It was a bit of ambiguous situation because libraries and tests
are technically submodules. After some feedback it seems that it
is better to ignore submodule for libraries and tests unless
requested explicitly.
Pull Request: https://projects.blender.org/blender/blender/pulls/118631
git on windows is giving no status updates while git lfs
is downloading the libs, causing people to assume its
stuck and interrupt the download. This change adds a
rather large visual message telling people not to do
that.
also it was not respecting people selecting n on the lib
download so some extra messaging has been added there
as well.
Git escaped a quote and used all of the cli arguments as the
folder name.
Also fixes:
- Progress on the initial clone so it doesn't appear stuck
- re-detect python after updating libs, so you don't have to run
make update twice on a fresh clone.
- Hide python not found warning unless the lib folder exists (this
seemingly got lost in one of the merges)
If LFS hooks are installed manually the automatic installation would
fail.
This change makes it so `lfs` is a valid command of `git`, ensuring
that the package is installed. If the installation fails assume it
is due to tricky local setup, and do not fail.
Pull Request: https://projects.blender.org/blender/blender/pulls/118618
- git lfs install was called a bit too late, after the libraries
has been checked out, leaving the checkout in a non-resolved
state.
- Update blender code first, allowing proper submodule hash to
be pulled.
Pull Request: https://projects.blender.org/blender/blender/pulls/118615
This change makes it so build system and update utilities for Blender builds
are using pre-compiled libraries and other resources attached as Git modules
instead of using checkout of SVN repositories in the parent folder.
The directory layout:
```
* release/datafiles/
* assets/ -> blender-assets.git
* publish/
* ...
* README.txt
* lib/
* darwin_x64/ -> lib-darwin_x64.git
* darwin_arm64/ -> lib-darwin_arm64.git
* linux_x64/ -> lib-linux_x64.git
* windows_x64/ -> lib-windows_x64.git
* tests/
* data/ -> blender-test-data.git
```
The changes about configuring the actual Git sub-modules are not included
into this patch, as those require repository to actually exist before it
can be used.
The assets submodule is enabled by default, and the rest of them are
disabled. This means that if someone runs `git submodule update --init`
they will not get heavy libraries. The platform-specific and tests
related submodules are enabled when using `make update` or `make test`.
All the submodules are tracked: this means that when new commits are
done to the submodule, the blender.git repository is to be updated to
point them to the new hash. This causes some extra manual work, but it
allows to more easily update Blender and its dependencies to known good
state when performing operations like bisect.
Ref #108978
Pull Request: https://projects.blender.org/blender/blender/pulls/117946
People running `make update` first time, will not have a lib
folder yet, and get a warning about python being missing, this
sends some of them off onto somewhat of a wild goose chase
installing python when this is really not needed since it's
included in our libraries.
This change, changes the warning to only emit when the lib
folder exists, but python is missing in the lib folder and
there actually is likely an issue with the lib folder.
Blender had a very limited (only uncompressed or MJPEG frames) .avi file
support, for both reading and writing. This is something that ffmpeg can
fully do.
This removes all of that. 3500 lines of code gone, primary motivations being:
- ffmpeg can read and write .avi files just fine, including ones with
uncompressed or MJPEG frames.
- Blender's ffmpeg integration could also be taught to produce uncompressed or
MJPEG .avi files, but TBH I don't see a particular reason to do that. Modern
formats like H264 are better in every way, and already support "lossless"
option if needed.
- The "Lite" blender build configuration was excluding both ffmpeg and avi
anyway, so that config is something that can't read nor write any movies.
User visible changes:
- In scene image output type, under Video section now there's only Ffmpeg Video
(AVI Raw and AVI JPEG are gone)
- Whenever loading an existing file, if output was one of AVI Raw / AVI JPEG,
it is set to Ffmpeg Video.
Pull Request: https://projects.blender.org/blender/blender/pulls/118409
Main improvements:
* Spatial structure (Kd-tree) build is now multithreaded.
* Kd-tree switched to use cache-friendlier TreeLets.
* Field fixed some non-deterministic behavior when spatial cache does
not receive any training data during a training iteration due to a
large number of training iterations.
* Fixed build problems on (non-Mac) ARM systems.
Pull Request: https://projects.blender.org/blender/blender/pulls/118328
This was already the minimum requirement for Intel and Apple Silicon
GPUs. It is required for the Metal backend to work correctly.
Previously the minimum for AMD GPUs was 10.15.
Pull Request: https://projects.blender.org/blender/blender/pulls/118287
Turns out we were not building OSL with OptiX enabled anymore.
Also check now if the OSL builds has OptiX support and if not
disable it in Cycles.
Building OSL with support for this (still) does not require
either the OptiX SDK or CUDA, it only needs LLVM.
Pull Request: https://projects.blender.org/blender/blender/pulls/118234
- Commands which have arguments split over multiple lines use
indented lines.
- Wrap lines where multiple commands run using "&&".
- Blank lines between multiple commands helps the text from becoming
too dense.
This is not a hard requirement to be able to build the libraries, but
these versions should be written down somewhere. So compare compiler
versions as part of make deps setup.
Pull Request: https://projects.blender.org/blender/blender/pulls/117457
Flex's bundled configure depended on aclocal-1.15 which has been
updated to 1.16.
Resolve by regenerating configure files on Linux which in turn adds a
new dependency on texinfo.
Flex's bundled configure depended on aclocal-1.15 which has been
updated to 1.16.
Resolve by regenerating configure files on Linux which in turn adds a
new dependency on texinfo.
This is supported on Apple Silicon GPUs and macOS 13.0+.
Co-authored-by: Stefan Werner <stefan.werner@intel.com>
Co-authored-by: Attila Afra <attila.t.afra@intel.com>
Pull Request: https://projects.blender.org/blender/blender/pulls/116124
NOTE: Blender does not seem to build with older OpenImageIO versions
(2.4.x and before) anymore. Since this library is now mandatory, it
means that Blender cannot be built without using the pre-compiled libs
on most Debian system, currently.
Ref. #113157.
Since the addition of Meteor Lake binaries, prebuilt GPU binaries
are now stored as fatbinaries. When running on a platform for which
prebuilt binaries are lacking or considered incompatible, the DPC++
SYCL runtime caching logic failed storing the (re)compiled
compatible version. This patch to DPC++ SYCL runtime fixes it.
Pull Request: https://projects.blender.org/blender/blender/pulls/117844
Add silently fail option to GPU based render tests. This is a pre-requisite to enable
render tests on the buildbot. By default these render tests will pass silently.
* Test will pass when using the `--pass-silently` arguments.
* Only crashes will be reported as failed tests.
* To find out failing test, review the test reports.
`WITH_GPU_RENDER_TESTS_SILENT` compile option can be used to let tests pass (default)
or fail (default for developers).
Although some tests fail, they still passed. In the generated render report,
the silently passed failures are correctly reported to be failures.
Pull Request: https://projects.blender.org/blender/blender/pulls/117629
This change fixes confusion situation when the render output
is an RGBA image: the difference in color was not visible in
the report because alpha channel was all zeros. This is due
to idiff performing per-channel difference.
The solution to this problem is to have separate images for
color and alpha difference, which makes it clear where the
difference actually is coming from.
It's not clear what these are used for as they seem like recent
additions and are set to "NOT-FOUND" even when tiff/freetype are found.
Marking as advanced instead of removing so as not to interfere with
freetype/tiff finding logic.
Exceptions:
* Links to personal wiki pages
* Pages that are not in the new developer docs yet (like Human Interface Guidelines)
* tools\check_wiki\check_wiki_file_structure.py needs a refactor
Previous code was using the 'nuclear' SAN weapon `exitcode=0`, which
essentially makes testing ignore all sanitizer errors.
This commit removes this option, and instead uses the suppression
list from Blender source repository (`tools/config/analysis/lsan.supp`)
for the leak sanitizer.
It also enables the `allocator_may_return_null` for the address sanitizer.
This should only be needed for one test (the `guardedalloc` one), but for
now it is simpler to define it for all tests. This can be refined at a
later time.
Pull Request: https://projects.blender.org/blender/blender/pulls/116635
Along with regular fixes and improvements,
this removes a warning on startup for Linux, see:
https://github.com/kcat/openal-soft/issues/554
Some deprecated CMake variables were removed and needed to be renamed.
Note that the previous URL from http://openal-soft.org/
is no longer responsive, using the GITHUB URL instead.
Ref !116877
The current handling had a fairly bad issue: multiple calls to
`set_tests_properties` to set envvars of a same test.
This does not work, only the last call is effective, all previous
ones have absolutely no effect.
This has been addressed by moving all 'set envvar for test' logic into a
single CMake function, `blender_test_set_envvars`.
This function takes optional extra envvars if needed, and define a set
of default ones (currently, `PATH` from `PLATFORM_ENV_INSTALL` if
defined, and the 'nuke' `exitcode=0` `LSAN_OPTIONS` if relevant).
NOTE: The way `blender_test_set_envvars` handles extra envvars passed to
it as parameter is fairly basic and unsafe, in that there is no check
whether a same envvar is defined more than once. Think for now this is
an acceptable limitation.
NOTE: Although this commit _should_ be a non-functional change one, the
unification of the handling of all envvars makes it hard to ensure there is no
side effects.
The `PATH` envvar e.g. was set to either `PLATFORM_ENV_INSTALL` if defined,
or a copy of that variable's definition, but only in Windows case. So technically,
the behavior for this envvar is changed.
This commit adds a new helper to define expected properties when a
target needs to use the unity build feature.
That new helper does what was already done for existing cases, and in
addition add the target to the Ninja 'heavy' pooljobs if relevant.
Pull Request: https://projects.blender.org/blender/blender/pulls/116791
Bundling many tests in a single binary reduces build time and disk space
usage, but is less convenient for running individual tests command line
as filter flags need to be used.
This adds WITH_TESTS_SINGLE_BINARY to generate one executable file per
source file. Note that enabling this option requires a significant amount
of disk space.
Due to refactoring, the resulting ctest names are a bit different than
before. The number of tests is also a bit different depending if this
option is used, as one uses gtests discovery and the other is organized
purely by filename, which isn't always 1:1.
Co-authored-by: Sergey Sharybin <sergey@blender.org>
Pull Request: https://projects.blender.org/blender/blender/pulls/114604
The warning was introduced with XCode 15 and conflicted with some
assumptions in the CMake. Even without actual cyclic dependency
between targets CMake might decide to pass library multiple times
to the linker, to ensure all its users find symbols from it.
This behavior is expected to be tweaked in the upcoming CMake
version, but until it is released and became widely used by all
Blender macOS developers silence the warning.
Pull Request: https://projects.blender.org/blender/blender/pulls/116718
* Different fix for Mantaflow linker warnings that works with new OpenVDB.
* Use new linker for arm64 as it no longer produces warnings with latest
Xcode. Still use the old one for x86_64 as some warnings remain.
* Fix wrong x86_64 build target in deps builder.
For the upcoming 4.1 libraries.
Ref #113157
Pull Request: https://projects.blender.org/blender/blender/pulls/116708
OIIO was including imaths half.h when it did not
need to, leading to build errors in lite builds.
This issue had been fixed already upstream but
was not yet included in the version we use.
Brecht Van Lommel
Sergey Sharybin
Bastien Montagne
Ray Molenkamp
Falk David
Aras Pranckevicius
bartus
Campbell Barton
Harley Acheson
Thomas Dinges
Stefan Werner
Xavier Hallade
Nikita Sirgienko
Jeroen Bakker
Hans Goudey