The dpcpp folder grew from 200M to 500M with the last update
due to lld being enabled and having 5 different copies in the bin
folder. We do not need to ship lld so it can be safely removed.
However previous harvest cleaned up the build folder before copying
the libs to their final destination in output, this will no longer
work, since we actually do need lld to build embree.
So copy to the full build folder to output first, then remove the
binaries we do not need. Embree will use the binaries in the build
folder so it will be unaffected by this.
There is a `KeyError` exception when the `install_linux_packages.py` build script is ran with `--distro-id`:
```console
$ ./build_files/build_environment/install_linux_packages.py --distro-id arch
INFO: Distribution identifier forced by user to arch.
Traceback (most recent call last):
File "file:///blender/./build_files/build_environment/install_linux_packages.py", line 1656, in <module>
main()
File "file:///blender/./build_files/build_environment/install_linux_packages.py", line 1646, in main
distro_package_installer = PackageInstaller(settings) if settings.show_deps else get_distro_package_installer(settings)
File "file:///blender/./build_files/build_environment/install_linux_packages.py", line 1570, in get_distro_package_installer
return DISTRO_IDS_INSTALLERS[get_distro(settings)](settings)
KeyError: None
```
This happens because the `get_distro` function returns `None` if the distribution ID is forced with the `--distro-id` option, when it should return the provided value.
Pull Request: https://projects.blender.org/blender/blender/pulls/106461
Use the latest Wayland version, resolves inverted track-pad scroll
direction #104272 by adding support for physical scroll direction.
Updates to GHOST/Wayland have already been made.
This tool now only supports installing packages, it won't build any
dependency anymore. This implies that on most distributions, it wont
make a complete Blender build possible, some features will likely be
misisng.
By default, it only install basic dev environment to enable building
Blender with the precompiled libraries (which is the recommended way).
Passing the `--all` option will make it try to install all known
dependencies from distro packages. This is a development option,
regular Blender build should be done with the precompiled libraries.
Tested on Debian Testing, Arch, Fedora and Suse Tumbleweed.
With the new design, it should be fairly trivial to add more
distributions if desired, and maintenance should also be way simpler.
NOTE: side effect is that a working python3 interpreter is expected on
system running this script. In practice this should not be an issue,
since Python3 is installed by default in almost all regular desktop
linux installations.
During the discussion for #101413 there was consensus that we could make
OIIO a mandatory dependency. This patch does just that.
The `idiff` testing tool remains optional.
Pull Request #105111
Contributed by Yulia Kuznetcova at Apple.
NanoVDB is patched to give add address spaces required by Metal. We hope that
in the future Metal will support the generic address space.
For AMD and Intel this is currently not available since it causes a performance
regression also on scenes without volumes.
Pull Request #104837
This reverts commit 19222627c6.
Something went wrong here, seems like this commit merged the main branch
into the release branch, which should never be done.
This reverts commit 68181c2560.
I merged 3.6 into 3.5 by mistake. Basically I had a PR against main,
then changed it in the last minute to be against 3.5 via the
web-interface unaware that I shouldn't do it without updating the
patch.
Original Pull Request: #104889
Note that the node group has its sockets names
translated, while the built-in nodes don't.
So we need to use data_ for the built-in nodes names,
and the sockets of the created node groups.
Pull Request #104889
* BLENDER_VERSION_CYCLE set to beta
* Update pipeline_config.yaml to point to 3.5 branches and svn tags
* Update and uncomment BLENDER_VERSION in download.cmake
Use a MAKE wrapper for 'make deps' on Linux that ensures dependencies
are built one at a time. This is preferable because building many
dependencies at once made troubleshooting impractical and had the
downside that large deps such as LLVM would bottleneck on a single core.
This may be used for macOS, so far it's only tested on Linux.
This is used for most Python release builds and has been reported to
give a modest 5-10% speedup (depending on the workload).
This could be enabled on macOS too but needs to be tested.
This was quite a fight, some resulting notes:
* We cannot use anymore Boost packages because of a weird bug breaking
Blender debug builds and ivolving Boost, TBB and USD (see also
rB019b930d6b9c).
* OCIO, OIIO, OpenVDB and USD build options where updated to match these
from cmake libs building.
** Most notably, USD now has imaging, OIIO, OpenVDB and GL support.
Ref. T99618.
Based on suggestion from Ray molenkamp (@LazyDodo), thanks.
Also fix patching of OSL code, would not work properly when using repo
checkout instead of archive download.
Re. T99618.
OIIO: 2.4.6.0
OpenVDB: 10.0.0
OSD: 3.5.0
OCIO: 2.2.0
NOTE: Had to fight OpenVDB to force it to use 'deprecated' TBB 2020, it
really wants to use oneTBB when it can find it.
Re. T99618.
Having all packages on one line made reviewing changes difficult.
Also note why Python modules are needed (with some TODO's where I wasn't
able to find any reason given for their inclusion).
The PKGCONFIG file exposes the OPUS include: requiring <opus/opus.h>
to be replaced with <opus.h>. Manipulate the PKGCONFIG file instead of
patching the source since the small change is only needed in one place.
Meson is built as part of external_python_site_packages,
without this dependency it would be called before being built.
Also remove Meson as a build requirement since the version is used.
Apple notarization rejects packages containing them.
Similar to rpaths we handle generically this as part of the harvest step rather
than patches individual library builds systems.
Ref T99618
- Added comment for additional tasks to do when bumping python
- updated sqlite to 3.39.4
- downgrade setuptools to 63.2.0 to avoid numpy build issues
- numpy 1.23.5
This updates the libraries dependencies for VFX platform 2023, and adds various
new libraries. It also enables Python bindings and switches from static to
shared for various libraries.
The precompiled libraries for all platforms will be updated to these new
versions in the coming weeks.
New:
Fribidi 1.0.12
Harfbuzz 5.1.0
MaterialX 1.38.6 (shared lib with python bindings)
Minizipng 3.0.7
Pybind11 2.10.1
Shaderc 2022.3
Vulkan 1.2.198
Updated:
Boost 1.8.0 (shared lib)
Cython 0.29.30
Numpy 1.23.2
OpenColorIO 2.2.0 (shared lib with python bindings)
OpenImageIO 2.4.6.0 (shared lib with python bindings)
OpenSubdiv 3.5.0
OpenVDB 10.0.0 (shared lib with python bindings)
OSL 1.12.7.1 (enable nvptx backend)
TBB (shared lib)
USD 22.11 (shared lib with python bindings, enable hydra)
yaml-cpp 0.8.0
Includes contributions by Ray Molenkamp, Brecht Van Lommel, Georgiy Markelov
and Campbell Barton.
Ref T99618
* BLENDER_VERSION_CYCLE set to beta
* Update pipeline_config.yaml to point to 3.2 branches and svn tags
* Update and uncomment BLENDER_VERSION in download.cmake
the windows builder has 2 different build folders for the debug and
release libs, hardcoding the path like this leads to an extra 1GB
of downloaded files. The windows builder supplies a single PACKAGE_DIR
for both but gets ignored.
This change promotes PACKAGE_DIR to be cached in CMakeCache so it can
be changed if needed.
Differential Revision: https://developer.blender.org/D16213
Reviewed by; brecht, dfelinto
The check was already there, but given it was not
exposed as an option it didn't show up in my cmakecache
and i got tired of having to lookup the exact name
all the time.
- build and use our version of libxml
- the cli tools had a linker error due
to it trying to link a shared version
of libxml, disabled both and zlib 1.2.3
with a patch since we do not want/need
them for blender.
- postfix the libraries with _d for debug
automatically so we don't have to fix that
during the harvest.
due to this only being windows changes no
rebuild needed for the other platforms.
After the last library update cve_check still reported some false positives.
One GMP issues was mitigated with a patch in the library update. The others
are ignored, with a description explaining why they do not affect Blender.
Ref D16269, T101403
THis is bumping dependencies to fix known CVEs, with the exception of
OpenImageIO which also includes bugfixes for performance and correctness
with some image types.
zlib 1.2.12 -> 1.2.13
freetype 2.11.1 -> 2.12.1
openimageio 2.3.13.0 -> 2.3.20.0
python 3.10.2 -> 3.10.8
openjpeg 2.4.0 -> 2.5.0
ffmpeg 5.0 -> 5.1.2
sndfile 1.0.28 -> 1.1.0
xml2 2.9.10 -> 2.10.3
expat 2.4.4 -> 2.4.9
openssl 1.1.1g/i -> 1.1.1q
sqlite 3.31.1 -> 3.37.2
Notable changes:
* AOM: the hack we had in place to make it not detect pthreads on windows no
longer worked with a more recent cmake version. Disabled pthreads with a
diff on Windows.
* Python: embedded copy of zlib 2.1.12 swapped out for our 2.1.13 copy with
some folder manipulation on Windows.
* Freetype: was harbouring a copy of zlib 2.1.12 as well, so that had to end.
* FFmpeg: patch used to fix D11796 is no longer needed. Add new patch to deal
with simple_idct.asm generating an object file with no sections in it,
backport from upstream commit.
* TinyXML: still being downloaded but no longer used by OpenColorIO, removed.
* GMP applied upstream patch to fix CVE-2021-43618, as there is no release yet.
* SQLite and Libsndfile patches no longer needed.
Includes contributes by Ray Molenkamp, Campbell Barton and Brecht Van Lommel.
Ref T101403
Differential Revision: https://developer.blender.org/D16269
- Include libxkbcommon headers: needed for building Blender with Wayland.
- Remove Wayland libraries (now wayland is built locally).
- Split up packages into two lists, one for building libs,
another for building Blender - since it's useful to be able to build
Blender, testing the libs work as expected.
This removes the dependency on wayland-devel, with some potential
for errors caused from using two different versions.
Note that the generated files are currently unchanged.
- Prefer bash over sh to so it's unambiguous which features can be used.
- Use array syntax to specify main package list, allowing to comment
why packages are required and making git-blame easier to track down
commits that add/remove packages.
- Add license header.
harvesting openpgl_Exports-debug.cmake was missing for
windows in the build script, and we manually added it
to svn when openpgl landed when we realized it was
missing.
This fixes the issue at the root of the problem, so
the file will not be missing for future updates.
This change adds support for intels cve-bin-tool [1]
in the deps builder. This adds 2 new targets to the
builder that do not build automatically but can be
build on demand when required.
`make cve_check` will output to the console.
`make cve_check_html` will output a html file that
can be shared with other people.
Requirements:
- A working installation of cve-bin-tool on the system
Not required but higly recommended:
- Obtaining a key from the nvd [2] to speed up the
database download. you can pass the key to cmake
using `-DCVE_CHECK_NVD_KEY=your_api_key`
[1] https://github.com/intel/cve-bin-tool
[2] https://nvd.nist.gov/developers/request-an-api-key
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D16160
The headers on centOS7 (Wayland 1.15) aren't compatible with the headers
generated by wayland-scanner 1.20.
Include the headers to support building Wayland on systems with
older Wayland installed.
Using these headers will be part of a separate commit.
Part of D16091.
Add a script to set up a Linux CentOS 7 machine so that it can be used
to build the precompiled libraries for Linux.
Reviewed By: sergey, brecht
Differential Revision: https://developer.blender.org/D16171
During the 3.3 release some packages were missing
in SVN during the release and it ended up building
the release tarball without issues when re-running
the `make source_archive_complete` command after it
failed initially. The tarball however had 0 byte files
for the missing packages.... not good.
This diff hardens the download process by :
1) Validating all required variables are set. This
catches the erroneously attempt at downloading the
nanovdb package even though we have removed it
from versions.cmake but neglected to remove it
from download.cmake
2) When a download fails (due to either a missing
package, or bad download URL) FILE Download will
warn about a hash mismatch but will carry on
happily, you then have to go into the file system
go delete the 0 byte file to retry the download.
We know for a fact the file is bad when it is 0
bytes, just delete it.
3) When we are using the blender repository
(and likely building a source archive) explicitly
validate the hash of all packages. Normally the
build process does this, however when building
a source archive the build does not actually
run for a dep. So preform this check during the
configuration stage.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D16124
- fix the source package download URL
- patch the `meson.build` file to allow the CentOS 7 meson version to be
used
- only build what we need (`wayland-scanner`), and not the libraries,
documentation, and tests.
Ref: D16074
This also reverts commit f6664217b3
"Build: temporarily disable wayland dependency".
This is needed to ensure and up to date "wayland-scanner" is used,
as versions before 1.20.0 generate headers incompatible with
dynamic linking (WITH_GHOST_WAYLAND_DYNLOAD).
As the centos7 version of wayland is 1.15 so make this part of Blender's
dependencies on Linux.
We intend to enable Wayland for Blender 3.4 release, this is needed for
the build-bot.
Reviewed By: brecht
Ref D16074
This has been a pain, newer OpenVDB forcefully trying to use more recent
system TBB (oneTBB) instead of the one built by this script.
Also include a few minor unrelated fixes.
Ref T99618.
With libepoxy we can choose between EGL and GLX at runtime, as well as
dynamically open EGL and GLX libraries without linking to them.
This will make it possible to build with Wayland, EGL, GLVND support while
still running on systems that only have X11, GLX and libGL. It also paves
the way for headless rendering through EGL.
libepoxy is a new library dependency, and is included in the precompiled
libraries. GLEW is no longer a dependency, and WITH_SYSTEM_GLEW was removed.
Includes contributions by Brecht Van Lommel, Ray Molenkamp, Campbell Barton
and Sergey Sharybin.
Ref T76428
Differential Revision: https://developer.blender.org/D15291
* BLENDER_VERSION_CYCLE set to beta
* Update pipeline_config.yaml to point to 3.2 branches and svn tags
* Update and uncomment BLENDER_VERSION in download.cmake
* OpenEXR 3.1.4 -> 3.1.5, this fixes several issues OSS fuzz found.
* libtiff 4.3.0 -> 4.4.0, this fixes several CVE's.
This also converts the harvest of libtiff on windows to a post install handler,
there's a few left but Windows is getting close to being harvest free.
Differential Revision: https://developer.blender.org/D15478
Ray Molenkamp
Xavier Hallade
bvpav
Xavier Hallade
Campbell Barton
Bastien Montagne
Brecht Van Lommel
Brecht Van Lommel
Jesse Yurkovich
Julian Eisel
YimingWu
Dalai Felinto
Dalai Felinto
Thomas Dinges
Thomas Dinges
Sergey Sharybin
Sebastian Herholz
Sybren A. Stüvel
Christian Rauch