By default boost::python in debug configuration links the release
python libraries. Which leads to loading issues with the produced
modules in a debug blender (which does use the debug python libs)
Bjam has an option to switch this to the debug libraries but when
you switch that on it changes the library names for *all* boost
libraries, even ones that don't have anything to do with python
and even the release libraries.
therefore an alternative way has been chosen and we're dealing
with this by adding a define rather than asking bjam to do it.
This patch changes openvdb from a static to a dynamic library.
this is in preparation for enabling pyopenvdb at some point
in the future.
Differential Revision: https://developer.blender.org/D8282
Reviewed by: brecht
This this adds the option of building boost::python
in the libs builder, in preparation for future
dependencies that require it (ie pyopenvdb)
disabled by default, can be enabled with the
`WITH_BOOST_PYTHON` cmake option.
Differential Revision: https://developer.blender.org/D8212
Reviewed by: brecht
This commits:
- Updates some libraries to latest officially supported versions:
** Numpy: 1.17.5
** OCIO: 1.1.1
** OIIO: 2.1.15
** OSL: 1.10.10
** OIDN: 1.2.1
- Re-enables some distro packages (like OSL, OIIO, OCIO...).
- Add missing 'CMake cleanup commands' for generated CMake update
command, for Embree, OIDN and OpenXR.
- Generalizes using min/max versions of accepted libraries, if no
package can be found in specified range then it is built from sources.
The later point should help keeping things a bit in better conditions,
although current maximal accepted versions are somewhat arbitrary guess
currently.
Fix package name missmatch in a few module files. IE "ALEMBIC" was
defined in the file but the find_package commands used "Alembic"
Some modules state that they set and use the _LIBRARY variable but the
do in fact not do this. Removed these comments from those files.
Required for the new boolean code, disabled by default
until all platforms have landed the libs and the boolean
code actually lands in master.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D8384
Add our own copy of the gtest discovery scripts from CMake a few reasons:
* Use the very latest version which supports PRE_TEST for Windows
* Fix usage of [] symbols in file paths that fail with the zsh shell
* Disable asan leak checker when discovering tests
This means Windows also no longer requires the very latest CMake 3.18.
This bumps the minimum requirement for cmake from 3.10 to 3.18 on windows
if `WITH_GTESTS` is enabled.
Reviewed By: sergey brecht sybren campbellbarton
Differential Revision: https://developer.blender.org/D8405
Description of `USER_HEADER_SEARCH_PATHS` build setting:
"
This is a list of paths to folders to be searched by the compiler
for included or imported user header files (those headers listed
in quotes) when compiling C, Objective-C, C++, or Objective-C++.
Paths are delimited by whitespace, so any paths with spaces in
them need to be properly quoted. See Always Search User Paths
(Deprecated) (ALWAYS_SEARCH_USER_PATHS) for more details
on how this setting is used. If the compiler doesn't support the
concept of user headers, then the search paths are prepended to
the any existing header search paths defined in Header Search
Paths (HEADER_SEARCH_PATHS).
"
http://help.apple.com/xcode/mac/current/#/itcaec37c2a6
Xcode doesn't use `HEADER_SEARCH_PATHS` for auto-complete. Only the
header files in the same directory as the current file are suggested.
CMake as of now correctly sets `SYSTEM_HEADER_SEARCH_PATHS` and lumps the
rest in `HEADER_SEARCH_PATHS`. The standard way is to use
`USER_HEADER_SEARCH_PATHS` & `SYSTEM_HEADER_SEARCH_PATHS` and let
`HEADER_SEARCH_PATHS` be used as a fallback for compilers which do not
distinguish between `<*.h>` and `"*.h"` syntax.
So set `USER_HEADER_SEARCH_PATHS` to the include paths specified
in the `CMakeLists.txt` files of all targets.
In the situation where the precompiled libraries are used on Linux +
GCC, a version of GCC older than 9.3 is guaranteed to cause problems.
This just implents a fatal error message when we know it doesn't make
sense to continue. We could do more checks and add some warnings, but
it's very likely that these will be ignored amongst the other noise.
Reviewed By: sergey, brecht
Differential Revision: https://developer.blender.org/D8396
This patch changes the discovery of pre-compiled kernels, to look for any PTX, even if
it does not match the current architecture version exactly. It works because the driver can
JIT-compile PTX generated for architectures less than or equal to the current one.
This e.g. makes it possible to render on a new GPU architecture even if no pre-compiled
binary kernel was distributed for it as part of the Blender installation.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D8332
This commit introduces a new way to build unit tests. It is now possible
for each module to generate its own test library. The tests in these
libraries are then bundled into a single executable.
The test executable can be run with `ctest`. Even though the tests
reside in a single executable, they are still exposed as individual
tests to `ctest`, and thus can be selected via its `-R` argument.
Not yet ported tests still build & run as before.
The following rules apply:
- Test code should reside in the same directory as the code under test.
- Tests that target functionality in `somefile.{c,cc}` should reside in
`somefile_test.cc`.
- The namespace for tests is the `tests` sub-namespace of the code under
test. For example, tests for `blender::bke` should be in
`blender::bke:tests`.
- The test files should be listed in the module's `CMakeLists.txt` in a
`blender_add_test_lib()` call. See the `blenkernel` module for an
example.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D7649
Enabling all `make deps` dependencies with the exception of Embree and OIDN.
After that, Blender can be compiled on an Apple Silicon Mac just like on any
Intel based Mac. There are still compiler warnings that need to be
investigated and there are probably a couple of bug still to be discovered
and to be fixed.
Most patches to the dependencies are simple and are about disabling SSE and
setting the proper architecture to compiile for. Notable exception is Python,
where I back ported a yet to be accepted PR for upstream Python:
https://github.com/python/cpython/pull/21249
Cross compiling or buliding a Universal Binary is not supported yet.
The minimum macOS target version for x86_64 remains at 10.13, the target
for arm64 is 11.00.
Differential Revision: https://developer.blender.org/D8236
There were two issues.
First is related on ISPC's CMake configuration forcing C and C++
compilers to be clang and clang++. This goes against of desired
behavior when we use our own compiled clang compilers.
The second issue was related on linker failure: CLang libraries
are linked statically, and they need some of C++ 11 STL symbols
which are coming from libstdc++.
Differential Revision: https://developer.blender.org/D8258
The ff_cfhd_init_vlcs() function was using a lot of stack space, which
made linker on macOS unhappy. Using heap allocation allows to silence
the warning without causing other side-effects.
Kept the patch enabled for all platforms to avoid difference in behavior
and performance on different platforms, which could make certain types
of investigation very tricky.
Differential Revision: https://developer.blender.org/D8248
C++17 does not work on 10.12, and Apple extended support ended for 10.12 in
October 2019.
Maniphest Tasks: T76783, T76184
Differential Revision: https://developer.blender.org/D8179
The spelling and capitalization of package name passed to find_package()
and find_package_handle_standard_args() needs to match.
Silences CMake warning about mismatch.
Differential Revision: https://developer.blender.org/D8247
The upstream version of nasm does not put version information to the
generated object files, which makes linker to show the following
warning:
building for macOS, but linking in object file
Using own patched version of nasm which puts required information to
the object file, making linker happy.
The plan is to either streamline the patch and provide it to the
upstream, or, it that takes too long, get an independent fix from the
upstream.
We don't need it and it was optionally enabled, causing Blender to fail
to link on certain configuration (when Brotli is installed via Homebrew
for example).
The configuration was confused about gettext installed via Homebrew
and isysroot passed to Python's compilation but not to test programs.
After this change `import gettext` still works, but it is unclear how
to test it further,
Differential Revision: https://developer.blender.org/D8231
Set of fixes which had to be made in order to have dependencies built
on own laptop:
- Require bison as a dependent software. It is required by ISPC.
On macOS it is required to be installed via Homebrew. This is because
Bison from Xcode toolchain is too old.
- Made sure Boost is compiled using clang.
Without this gcc was used, and some unsupported command line argument
was passed to it.
- Modify OGG in a way which does in fact pull fixed sized types.
They are defined in stdint.h.
Without this fix FFmpeg will not detect presence of OGG because the
test program fails to compile.
- Force disable zstd compression and make wepb optional for the TIFF
library. Without this TIFF might pick up development libraries from
Homebrew.
Differential Revision: https://developer.blender.org/D8221
Clang Tidy is a Clang based "linter" tool which goal is to help
fixing typical programming errors.
It is run as a separate compile step of every file, which slows
compilation down but allows to fully analyze the file the same
way as compiler does and catch non-trivial bugprone cases.
This change includes:
- CMake option called `WITH_CLANG_TIDY` which enables Clang Tidy
linter tool on all source in the `source/` directory.
This option is only available on Linux, as it is currently the
easiest platform to get the Clang Tidy toolchain to work.
- CMake module which is aimed to find latest available Clang Tidy.
- Set of rules which allows to have Blender fully compiled without
extra issues.
The goal of this change is to provide a base ground so that solving
all the warnings can happen later on, as a team effort.
It should be possible to use Clang Tidy side-by-side with both GCC
and Clang, but there seems to be some tweaks to be done in CMake to
make it really work for Blender. For now use Clang toolchain if
there are issues with GCC+Clang Tidy.
It will be worked on in the nearest future to bring seamless
experience for all configurations.
Currently there is no official way of getting Clang Tidy on macOS,
and on Windows there are some difficulties of hooking up Clang Tidy
from LLVM package to the MSVC compiler toolchain.
The actual warnings in the code will be addressed as a part of the
Code Quality Days, task T78535.
Differential Revision: https://developer.blender.org/D7937
Solves problem with different order of codesign server startup and
mount of network shares: avoids exception happening when server is
started prior to the mounts are ready.
When there is no system python OSL will fail to build the documentation.
Given we don't ship the documentation, this is safe to disable.
Originally part of D8123
When doing a release build the TBB debug libs are not
set which was causing an error during the configure
phase of USD, so always set them even if not used.
This requires ISPC for building OpenImageDenoise, so that is now added as
a dependency as well. Blender itself does not need ISPC for building so it
is not included as part of the precompiled libraries.
Differential Revision: https://developer.blender.org/D7641
The Blender USD code didn't have to change for this upgrade. Pixar's USD
did include a change that we had in the patch, so that's been removed
from our patch now. Some of the USD code that we patched changed as
well.
Is achieved by replacing hard-coded signed/unsigned file names with
"<uuid>" which acts as a "request ID". This way multiple workers can
put their requests into a single directory without collisions. The
code sign server will handle the requests sequentially in an unknown
order.
The directory layout on worker goes as following:
<Worker>
<Builder Name>
blender.git/
build/
install/
lib/
Adding an extra <Builder Name> after build is redundant.
Differential Revision: https://developer.blender.org/D8045
The old URL did have a Git commit hash in it, but apparently the server
was ignoring it and only used the `master` that was also mentioned in the
URL. As a result, every new download would get the latest version from
the `master` branch, invalidating the SHA256 checksum.
I replaced `master` with the actual commit hash. This should make the
situation stable.
No functional changes.
embree marks a few of its functions with a dll_export macro
forcibly exporting these symbols from whatever binary links
them. Given we link embree statically and we do not want these
exports in the blender binary, the macro needs to be a no-op.
This updates python to the latest patch level available for 3.7
also updates some of the packages we rely on:
idna 2.9
urllib3 1.25.9
cerifi 2020.4.5.2
requests 2.23.0
numpy 1.17.5
This upgrade required a few changes:
- Some parts of our patch are no longer necessary, as the USD library
now includes those changes.
- The rest of the patch needed adjustment as the `pxr/base/lib/*`
directories in USD's source code have moved to `pxr/base/*`.
- Updated library names on Windows -- thanks @LazyDodo.
Note that this does not enable the USD Python API for inclusion in
Blender. It just aims at being an as-simple-as-possible version upgrade
of the USD library.
now that we stick to some outdated py version, some distro (like current
debian testing) will feature several python3 dev package, but other
dependant libs like numpy are only built against current default version
of python (3.8 now in deb testing)...
In order to be able to use distro packages we need to allow using higher
versions of python, and set relevant CMake option accordingly.
This is the cluster of OIIO and friends , since they are all kinda tangled best to deal with this as a single unit
OIIO 2.1.15.0
png 1.6.37
jpeg 2.0.4
opencolorio 1.1.1
tiff 4.1.0
OSL 1.10.10
pugixml 1.10
openjpeg 2.3.1
Differential Revision: https://developer.blender.org/D7727
Reviewed by: brecht
The file subversion is no longer used in the Python API or user interface,
and is now internal to Blender.
User interface, Python API and file I/O metadata now use more consistent
formatting for version numbers. Official releases use "2.83.0", "2.83.1",
and releases under development use "2.90.0 Alpha", "2.90.0 Beta".
Some Python add-ons may need to lower the Blender version in bl_info to
(2, 83, 0) or (2, 90, 0) if they used a subversion number higher than 0.
https://wiki.blender.org/wiki/Reference/Release_Notes/2.83/Python_API#Compatibility
This change is in preparation of LTS releases, and also brings us more
in line with semantic versioning.
Fixes T76058.
Differential Revision: https://developer.blender.org/D7748
This diff updates:
FFmpeg : 4.3.2
libogg : 1.3.4
flac : 1.3.3
vpx : 1.8.2
xvid : 1.3.7
x264 : 33f9e1474613f59392be5ab6a7e7abf60fa63622
x264 seemingly has given up on even providing snapshots
and has been updated to the latest hash available at
this time.
faad has been removed since ffmpeg has not supported
it since 2010.
For a more detailed description of the issue see the commit
message for rB497cd3d7dd6e497be484eb78a8ddb23f53b20343
This change moves fftw to a shared library and reverts the bandaid
we did for 2.83.
Unsure what it is that upsets it so much, but when linking
both sndfile and fftw dynamically, the linker gets confused
and thinks that fftw is importing sf_close from the blender
binary (which makes *NO* sense) leading to a start-up error.
Generating the import library from the .def file using the
ms lib tool creates an import library that works fine.
This is not as much a fix as a work around, but given the real
involves replacing how we build fftw, it is not eligible for 2.83
which is in BCON3 already.
The root of the issue lies with (how we build) fftw3
The first issue is: fftw does not build with MSVC, there are other
dependencies that are not compatible with MSVC and for those we
build the libraries required with mingw64, same for fftw
The second issue is: for reasons unknown we really really really
liked all deps to link statically so wherever possible we did so.
Now during the building of the fftw it linked a few symbols from
libgcc (which we do not ship) like __chkstk_ms, for which we passed
some flags to stop generating calls to it. Problem solved! There
is no way this could possibly turn around and bite us in the rear.
fast forward to today mystery crashes that look like a race condition.
What is happening is, we tell the linker that each thread will require
a 2-megabyte stack, now if every thread immediately allocated 2 megs,
that be 'rough' on the memory usage. So, what happens is (for all apps
not just blender), 2 megs are reserved but not backed by any real memory
and the first page is allocated for use by the stack, now as the stack
grows, it will eventually grow out of that first page, and end up in
an area that has not been allocated yet, to deal with that the allocated
page is followed by a guard page, someone touches the guard page it's
time to grow the stack!
Meanwhile in FFTW is it's doing substantial allocation using alloca
(up to 64 kb) on the stack, jumping over the guard page, and ending
up in reserved but not yet committed memory, causing an access violation.
Now if you think, that doesn't sound right! something should have
protected us from that! You are correct! That thing was __chkstk_ms
which we disabled.
Given we do not want a dependency on libgcc while building with MSVC
the proper solution is to build fftw as a shared library which will
statically link any bits and pieces it needs, however that change
is a little bit too big to be doing in BCON3.
So as a work around, we change the size the stack grows from 8k to
68k which gives fftw a little bit more wiggle room to keep it out
of trouble most of the time.
Note this only sidesteps the issue, this may come up again if the
conditions are just right, and a proper solution will need to be
implemented for 2.90.
Search for all potential library names in each directory, otherwise e.g.
libImath-2_2.a from a system directory will be preferred over libImath.a even
if we specified a directory.
It was disabled in D7520 to keep the pdb's from growing out
of control however the increased link time is just not worth
it.
I'll keep an eye on the dailies and see if we have to come up
with a different solution.
The msvc linker had been warning about libcmt being dragged
in for a bit, finally tracked the issue down to the deps of
jpeg and pthreads which both ignored our cflags.
this diff changes them both to use the dynamic crt rather
than the static one so they'll be in line with all our
other libraries.
We now want to use fairly recent TBB (2018 at least?), so distro a bit
old will not have required package anymore...
Tested with Debian, other distro "should work" (c), but tests there are
much welcome! ;)
Previous code would forec built libs to always be rebuilt when one of
their deps was using a package. While this could be useful when said
package was actually updated, this generates way too much false
positives.
Now only rebuild a built lib when we actually switch from built to
package for one of its deps.
Ever since debug symbols were added for release builds the linker
has been on the chatty side about symbols being missing for our
binary libs.
There's currently no plans to supply those, so best for the linker
not to warn us about them.
Static tbb has always been frowned upon [1] sofar it has worked for us but
given our reliance on tbb is about to increase (D7475), I'd like to move the library
to more supported configuration. Which means moving it to be a dynamic library
The libs part of this change is in rBL62416
Reviewed By: Brecht
Differential Revision: https://developer.blender.org/D7570
This diff add supports for crash logs on windows for
release builds. This can be toggled on/off with the
`WITH_WINDOWS_PDB` cmake option. by default it is on.
Things to take into consideration:
Release builds are hightly optimized and the resulting
backtraces can be wrong/misleading, take the backtrace
as a general area where the problem resides rather than
an exact location.
By default we ship a minimized symbol file that can only
resolve the function names. This was chosen to strike
a balance between growth in size of the download vs
functionality gained. If more detailed information is
required such as source file + line number information
a full pdb can be shipped by setting `WITH_WINDOWS_STRIPPED_PDB`
to off.
Differential Revision: https://developer.blender.org/D7520
Reviewed by: brecht
This diff add supports for crash logs on windows for
release builds. This can be toggled on/off with the
`WITH_WINDOWS_PDB` cmake option. by default it is on.
Things to take into consideration:
Release builds are hightly optimized and the resulting
backtraces can be wrong/misleading, take the backtrace
as a general area where the problem resides rather than
an exact location.
By default we ship a minimized symbol file that can only
resolve the function names. This was chosen to strike
a balance between growth in size of the download vs
functionality gained. If more detailed information is
required such as source file + line number information
a full pdb can be shipped by setting `WITH_WINDOWS_STRIPPED_PDB`
to off.
The Release in the title of this diff refers to the
release build type, not the official blender releases.
Initially this will only be enabled for nightly build
bot versions of blender, official releases as of now
will not ship with symbols.
Differential Revision: https://developer.blender.org/D7520
Reviewed by: brecht
This patch enables TBB as the default task scheduler. TBB stands for Threading Building Blocks and is developed by Intel. The library contains several threading patters. This patch maps blenders BLI_task_* function to their counterpart. After this patch we can add more patterns. A promising one is TBB:graph that can be used for depsgraph, draw manager and compositor.
Performance changes depends on the actual hardware. It was tested on different hardwares from laptops to workstations and we didn't detected any downgrade of the performance.
* Linux Xeon E5-2699 v4 got FPS boost from 12 to 17 using Spring's 04_010_A.anim.blend.
* AMD Ryzen Threadripper 2990WX 32-Core Animation playback goes from 9.5-10.5 FPS to 13.0-14.0 FPS on Agent 327 , 10_03_B.anim.blend.
Reviewed By: brecht, sergey
Differential Revision: https://developer.blender.org/D7475
Usable with the CMake option 'WITH_GHOST_WAYLAND'
The following functionality is working:
- Building with X11 and Wayland at the same time,
wayland is used when available.
- Keyboard, pointer handling.
- Cursor handling.
- Dedicated off-screen windows.
- Drag & drop.
- Copy & paste.
- Pointer grabbing.
See D6567 for further details.
Add root usd library directory to build arguments, same as other libraries.
Also fix error/typo in compile_USD regarding _is_building.
Reviewed By: mont29
Differential Revision: https://developer.blender.org/D7563
sccache [1] is one of the few ccache like solutions that will
work on windows.
sccache support can be enabled with the `WITH_WINDOWS_SCCACHE`
cmake option however it will only will work with ninja as the
build system, msbuild is not supported currently.
Advanced option, developes are expected to obtain and configure
sccache on their own.
```
Full build no cache 1428.90s (100.00%)
Full build cached 434.34s ( 30.40%)
```
[1] https://github.com/mozilla/sccache
Reviewed By: nicholas_rishel, Brecht
Differential Revision: https://developer.blender.org/D7466
Otherwise file browsers do not recognize the Blender executable. This is
already done for official releases.
We leave it off for non-portable builds, since that's how Linux distribution
packages will typically build Blender and we can continue to follow the OS
default there. Using a file browser to launch executables from e.g. /usr/bin
would be rare as wel.
Differential Revision: https://developer.blender.org/D7363
Following up to b555b8d.
Building Blender with hidden symbols but using libraries with visible symbols
was giving linker warnings, specifically for USD. So revert that for now, as
it was not needed for the bugfix.
Hide USD symbols (some of which are not in the USD namespace) to avoid potential
conflicts. May potentially help with AMD OpenCL issues in T74262.
This means symbols from Blender itself and most external libraries. We can't
just hide all because that breaks some libraries. The better solution would
be to rebuild all library dependencies with hidden visibility.
Fixes T75223: Luxrender add-on failing to load on macOS
Happens on macOS 10.15.4 and Xcode 11.4.
The reason of failure is caused by following factors:
- xcodebuild reports full semantic macOS SDK version 10.15.4
- The actual SDK file path will only include major and minor part
of the version (10.15, MacOSX10.15.sdk)
- Previous CMake code of ours expected direct match between SDK
version and file path.
The solution is to make our detection code a bit more flexible and
additionally check for major.minor macOS SDK version in the path.
The specific goal of this change is to get rid of separate code paths
for older and newer Xcode versions.
The version 8.2 is picked since it's the latest version which runs on
macOS 10.11 (which is our current deployment target). If that turns
out too new for some reason the alternative would be to require Xcode
version 5.
The main CMakeLists already requires CMake 3.5, so there is no point of
requiring "newer" CMake on macOS.
This was a code from a while back where CMake 3 was not required on all
platforms.
The OpenXR-SDK contains utilities for using the OpenXR standard
(https://www.khronos.org/openxr/). Namely C-headers and a so called
"loader" to manage runtime linking to OpenXR platforms ("runtimes")
installed on the user's system.
The WITH_XR_OPENXR build option is disabled by default for now, as there
is no code using it yet. On macOS it will remain disabled for now, it's
untested and there's no OpenXR runtime in sight for it.
Some points on the OpenXR-SDK dependency:
* The repository is located at
https://github.com/KhronosGroup/OpenXR-SDK (Apache 2).
* Notes on updating the dependency:
https://wiki.blender.org/wiki/Source/OpenXR_SDK_Dependency
* It contains a bunch of generated files, for which the sources are in a
separate repository
(https://github.com/KhronosGroup/OpenXR-SDK-Source).
* We could use that other repo by default, but I'd rather go with the
simpler solution and allow people to opt in if they want advanced dev
features.
* We currently use the OpenXR loader lib from it and the headers.
* To use the injected OpenXR API-layers from the SDK (e.g. API
validation layers), the SDK needs to be compiled from this other
repository.
The extra "XR_" prefix in the build option is to avoid mix-ups of OpenXR
with OpenEXR.
Most of this comes from the 2019 GSoC project, "Core Support of Virtual
Reality Headsets through OpenXR"
(https://wiki.blender.org/wiki/User:Severin/GSoC-2019/).
Differential Revision: https://developer.blender.org/D6188
Reviewed by: Campbell Barton, Sergey Sharybin, Bastien Montagne, Ray
Molenkamp
Ray Molenkamp
Philipp Oeser
Brecht Van Lommel
Bastien Montagne
Sebastian Parborg
Campbell Barton
Ankit Meel
Sybren A. Stüvel
Patrick Mours
Stefan Werner
Sergey Sharybin
Stefan Werner
Tuomo Keskitalo
Brecht Van Lommel
Christian Rauch
Anthony Edlin
Nathan Craddock
Bastien Montagne
Mateusz Grzeliński
Julian Eisel
blender
Bastien Montagne
Julian Eisel