Commit Graph

2858 Commits

Author SHA1 Message Date
Bastien Montagne 092424dae3 install_deps: Fix OIIO depending on (system...) openVDB. 2021-10-08 14:52:06 +02:00
Brecht Van Lommel 0fd0b0643a Build: search for hipcc in HIP_ROOT_DIR/bin 2021-10-06 17:44:25 +02:00
Brecht Van Lommel 55b8fc718a Cycles: improve detection of HIP compiler for buildbot
And fix various broken things in the HIP kernel compilation.
2021-10-05 13:47:50 +02:00
Campbell Barton 6e48a51af7 check_cppcheck: use quiet output
Without this, each cppcheck invocation included all defines/includes
flooding the console with unhelpful information.

Also remove nonexistent directory to exclude.
2021-10-04 13:12:37 +11:00
Campbell Barton 606271966e check_cppcheck: use '--cppcheck-build-dir'
Use a temporary directory for faster performance.
2021-10-04 13:12:36 +11:00
Campbell Barton e0e7a5522f project_source_info: queue_processes() now waits for jobs to finish
queue_processes() - used for some of the "make check_*" utilities,
wasn't waiting for all jobs to finish before returning.

This conflicted with running cleanup operations.
2021-10-04 13:12:35 +11:00
Ray Molenkamp a78d3c5261 Fix: T91602 ffmpeg crash
Issue caused by our patch in rB1af722b81912
we replaced an array with a memory allocation
but we forgot to update the assert which now
used an invalid method to calculate the array
size.

SVN libs will have to be updated before
T91602 will be fixed for end users.
2021-09-22 17:29:23 -06:00
Brecht Van Lommel 0803119725 Cycles: merge of cycles-x branch, a major update to the renderer
This includes much improved GPU rendering performance, viewport interactivity,
new shadow catcher, revamped sampling settings, subsurface scattering anisotropy,
new GPU volume sampling, improved PMJ sampling pattern, and more.

Some features have also been removed or changed, breaking backwards compatibility.
Including the removal of the OpenCL backend, for which alternatives are under
development.

Release notes and code docs:
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles
https://wiki.blender.org/wiki/Source/Render/Cycles

Credits:
* Sergey Sharybin
* Brecht Van Lommel
* Patrick Mours (OptiX backend)
* Christophe Hery (subsurface scattering anisotropy)
* William Leeson (PMJ sampling pattern)
* Alaska (various fixes and tweaks)
* Thomas Dinges (various fixes)

For the full commit history, see the cycles-x branch. This squashes together
all the changes since intermediate changes would often fail building or tests.

Ref T87839, T87837, T87836
Fixes T90734, T89353, T80267, T80267, T77185, T69800
2021-09-21 14:55:54 +02:00
Brecht Van Lommel da4796ebf7 Build: change make update to not print errors regarding submodule branches
Instead of trying to checkout non-existent branches and getting confusing fatal
error prints, check if the branch exists first.

Ref D12560
2021-09-20 14:05:56 +02:00
Bastien Montagne 11e11c41f2 make_update: Fix case where a sub-module would not have required branch.
Issue revealed by rB546314fc9669 change, also error itself exited before
that commit.

Now we do accept git command to fail when trying to checkout the
specified branch from sub-modules, and only actually error in case the
fall-back branch (aka master) cannot be properly checked out.

Thanks fot Ray molenkamp (@LazyDodo) for report and initial patch
(D12560).
2021-09-20 12:44:01 +02:00
Brecht Van Lommel 603ae580ce Build: show better "make update" error message when in detached HEAD state 2021-09-13 13:14:17 +02:00
Bastien Montagne 9d6e960e2c Fix `make_utils.py` recent API breackage.
rB546314fc9669 broke `svn_libraries_base_url` utils API compatibility
for no good reasons, making new `branch` argument explicitely optional.

Reported on chat by  Michael Kowalski (@makowalski), thanks!
2021-09-13 11:41:43 +02:00
Ray Molenkamp ee3b4e8420 Windows: Fix VS2022 detection
VS2019 had a compiler update moving it into the
range that was used to detect VS2022. This patch
updates the detection to the current VS2022
preview compiler version.

Reported by Jesse Y on chat.
2021-09-09 17:19:58 -06:00
Antonio Vazquez bd79d6067c make.bat: Fix missing quotes in python detection 2021-09-06 17:58:38 +02:00
Bastien Montagne 546314fc96 Build utils: `make_update`: Add option to choose SVN branch.
Needed for studio sprite-fright frozen branch.

Also do not overwrite branch for git sub-modules when it is defined, and
fallback to `master` branch in case specified branch is not found in a
specific sub-repository.
2021-09-02 14:56:46 +02:00
Ray Molenkamp 518b97e674 Windows/Ninja: Optimize linker performance
The /Zc:inline flag is by default off in the MSVC
compiler however when you build with msbuild it adds
it to the build flags on its own.

Ninja however does not decide on its own to add
flags you didn't ask for and was building without
this flag.

This change explicitly adds the compiler flag so
msbuild and ninja builds are once more building
with the same build flags leading to smaller .obj
files when building with ninja and lightening the
workload for the linker.

This flag is available starting MSVC 2013 update 2
so does not need to be guarded with version checks.
2021-08-25 10:55:45 -06:00
Jeroen Bakker 8165333de9 Pipeline: Use more explicit cuda versions. 2021-08-23 13:47:40 +02:00
Lukas Stockner 2ea66af742 Add support for Zstandard compression for .blend files
Compressing blendfiles can help save a lot of disk space, but the slowdown
while loading and saving is a major annoyance.
Currently Blender uses Zlib (aka gzip aka Deflate) for compression, but there
are now several more modern algorithms that outperform it in every way.

In this patch, I decided for Zstandard aka Zstd for several reasons:
- It is widely supported, both in other programs and libraries as well as in
  general-purpose compression utilities on Unix
- It is extremely flexible - spanning several orders of magnitude of
  compression speeds depending on the level setting.
- It is pretty much on the Pareto frontier for all of its configurations
  (meaning that no other algorithm is both faster and more efficient).

One downside of course is that older versions of Blender will not be able to
read these files, but one can always just re-save them without compression or
decompress the file manually with an external tool.

The implementation here saves additional metadata into the compressed file in
order to allow for efficient seeking when loading. This is standard-compliant
and will be ignored by other tools that support Zstd.
If the metadata is not present (e.g. because you manually compressed a .blend
file with another tool), Blender will fall back to sequential reading.

Saving is multithreaded to improve performance. Loading is currently not
multithreaded since it's not easy to predict the access patterns of the
loading code when seeking is supported.
In the future, we might want to look into making this more predictable or
disabling seeking for the main .blend file, which would then allow for
multiple background threads that decompress data ahead of time.

The compression level was chosen to get sizes comparable to previous versions
at much higher speeds. In the future, this could be exposed as an option.

Reviewed By: campbellbarton, brecht, mont29

Differential Revision: https://developer.blender.org/D5799
2021-08-21 21:39:06 +02:00
Bastien Montagne 96d0cd57dc Fix T90719: Boost sources dowload address needed to be updated. 2021-08-17 18:01:42 +02:00
Ankit Meel b83ee724a4 Fix T90418: macOS codesign fails with dylib next to executable
Change the dylib folder relative to `Blender` executable to be
the same as before rB652fbc200500497a67bd11d18b786587ba34e3d9 and same
as bpy.so : `@loader_path/../Resources/${BLENDER_VERSION}/lib`
2021-08-10 10:30:55 +05:30
Brecht Van Lommel ff594715b8 Build: macOS library upgrade fixes
* Revert back to OpenMP 9.0.1 due to bug causing cloth physics test to fail.
* Skip flex build on macOS to avoid link error, only reason we build this is
  due to old flex version on Linux CentOS 7.
* Fix PNG cmake argument that expects lowercase on instead of ON.

 Ref T90507, T88438
2021-08-09 16:50:31 +02:00
Aaron Carlisle d245782b80 Windows: Add support to compile python api docs from make file
This adds support to compile the html python api docs from the command line by running `make doc_py` matching support between windows and unix.

This patch also makes it so the compiler is not needed if you set the `blender_bin` variable, this affects icon generation as well.

In the future, I want to move away from generating the build output in the build directory but that can come in a later change.

Reviewed By: LazyDodo

Differential Revision: https://developer.blender.org/D12144
2021-08-06 13:55:14 -04:00
Germano Cavalcante 89014b51f1 Xcode: support cmake options for grouping in folders
The Xcode IDE can also benefit from the options:
- WINDOWS_USE_VISUAL_STUDIO_SOURCE_FOLDERS
- WINDOWS_USE_VISUAL_STUDIO_PROJECT_FOLDERS

So add suport to these options and also renames them as they are no
longer limited to just Windows and Visual Studio.

Reviewed By: brecht, ankitm

Differential Revision: https://developer.blender.org/D12132
2021-08-05 15:31:41 -03:00
Campbell Barton 604ae5f7b6 Cleanup: tab indentation for CMake / GNUmakefile 2021-08-05 12:03:41 +10:00
Ray Molenkamp 1def985d78 Windows: Add icons and icons_geom to make.bat
This adds support for building the icons from make.bat
unlike bash there is no passing environment variables
on the command line.

The scripts go out of their way to locate both blender
and inkscape however if they are not found, the user is
given a helpful error message telling them how to set
the variables.

Although some extra help can be given there, if your
normal build is a 2019 full build running

`make 2019 full icons`

will help it find the blender executable as well.

finally if you know the name of your build folder
running

`make builddir build_windows_Lite_x64_vc16_Release icons`

will also work, if all fails you can point directly to
the blender executable by running

`set BLENDER_BIN=c:\where\blender\lives\blender.exe`

before running `make icons` or `make icons_geom`

The python scripts needed some small modifications since
without the PATHEXT, SystemRoot and SystemDrive
environment variables python will not initialize properly
on windows. (Not blender related, even mainline python
won't start without those)
2021-08-04 19:32:24 -06:00
Ankit Meel 18d900caca macOS: Fix OpenMP dynamic loader error. 2021-08-04 01:22:27 +05:30
Bastien Montagne 57281b73c4 Install_deps: Always re-create shortcuts to installed lib paths.
For some reasons looks like those shortcuts could get out of sync, which
created weird hard to understand building errors.

So for sake of simplicity and security, just re-create them all the
time, just like we update ld paths.
2021-08-03 17:22:48 +02:00
Ankit Meel 652fbc2005 macOS: Portable builds with dynamic libraries.
For Blender.app: dropping libomp.dylib next to Blender executable is
enough for it getting picked up since `@executable_path` is an rpath.

For non-distributed binaries datatoc, makesdna, tests etc, code for
copying libomp.dylib to build folder is removed and replaced by
CMake's rpath option for *build* tree.

For bpy.so, the post build rpath change has also been replaced by CMake
rpath option for *install* tree.

Since -id has been changed in D11748, remove the
`install_name_tool -change ...` command.

Any dylib can just be dropped at `MAC_BLENDER_TARGET_DYLIBS_DIR`
hereafter. Appending dylib path to `CMAKE_BUILD_RPATH` will be needed
for datatoc etc if linked against one (instead of copying the
dylibs around).

Reviewed By: #platform_macos, brecht
Differential Revision: https://developer.blender.org/D11997
2021-08-03 20:49:40 +05:30
Brecht Van Lommel eccd8af828 Revert "GHOST/X11: enable EGL"
This is causing issues for some users launching Blender, because EGL indirectly
requires GLVND, which is not installed by default on e.g. Ubuntu.

This reverts commit 0b18a618b8.

Fixes T90374

Ref D12034
2021-08-02 17:19:17 +02:00
James Monteath 0e4f7b4a4b Delete pipeline_config.json file. The yaml file is now used.
Update README.md.
2021-08-02 16:57:29 +02:00
James Monteath 8796a2827f Update REAME.md file 2021-07-30 13:40:48 +02:00
Christian Rauch 0b18a618b8 GHOST/X11: enable EGL
This will replace GLX with EGL for X11. GLEW does not support GLX and EGL
at the same time. Most distributions build GLEW with GLX support, so we
have to use the externally provided GLEW and build with EGL support.
This effectively sets WITH_SYSTEM_GLEW to OFF for all Linux configurations.

Differential Revision: https://developer.blender.org/D12034
2021-07-29 17:39:55 +01:00
Ray Molenkamp cae18abef4 deps/win: Remove media foundation dep for ffmpeg
This caused a blender load error on windows N,
given we do not use these codecs they can safely
be disabled.

This will fix T90200 once the new libraries are
in SVN
2021-07-29 10:21:43 -06:00
Bastien Montagne 33322086f6 install_deps: update OIDN to 1.4.1, and ISPC to 1.16.0.
Ref. T88438.
2021-07-29 17:00:49 +02:00
Bastien Montagne ceec7cabf5 install_deps: Update OSL to 1.11.14.1.
This has been a huge pain to get working, for several reasons (new flags
needed, patching is now mandatory, etc.).

Further more, discovered that debian OIIO package is now silently
relying on OpenCV, without even proper handling of this dependency (at
least in the `-dev` package), so had to revert to force-build own OIIO
again on that distro for the time being.

Ref. T88438.
2021-07-29 17:00:48 +02:00
Bastien Montagne b60e72100b Cleanup: OSL buildlib patch: Remove `.rej` part.
This patch contained changes for an `.rej` rejection file generated by
failed patch apply... Definitly nothing to do here.
2021-07-29 17:00:48 +02:00
Bastien Montagne 21b4799793 install_deps: Update OIIO to 1.1.15.1.
Ref. T88438.
2021-07-29 17:00:48 +02:00
Bastien Montagne bd2bfa469f install_deps: Update llvm to 12.0 (with minimal now being 11.0).
Ref T88438.
2021-07-29 17:00:48 +02:00
Bastien Montagne 12d93e44d0 Install_deps: add flex dependency.
Was already installed on Debian-like and Fedore/Suse actually, now also
explicitely required on Arch and listed in docs.

Ref. T88438.
2021-07-29 17:00:48 +02:00
Bastien Montagne 4d28703277 Install_deps: Add zstd dependency.
Ref T88438.
2021-07-29 17:00:48 +02:00
Ray Molenkamp 0f1c84f171 deps: Reduce llvm/clang footprint for windows
We shipped the whole bin folder for llvm/clang
while we only needed clang-format, by shipping
just the bits we need we save about 700 megabytes
off our svn lib download.
2021-07-29 07:45:38 -06:00
Brecht Van Lommel 073bf8bf52 Cycles: remove WITH_CYCLES_DEBUG, add WITH_CYCLES_DEBUG_NAN
WITH_CYCLES_DEBUG was used for rendering BVH debugging passes. But since we
mainly use Embree an OptiX now, this information is no longer important.

WITH_CYCLES_DEBUG_NAN will enable additional checks for NaNs and invalid values
in the kernel, for Cycles developers. Previously these asserts where enabled in
all debug builds, but this is too likely to crash Blender in scenes that render
fine regardless of the NaNs. So this is behind a CMake option now.

Fixes T90240
2021-07-28 19:27:57 +02:00
Sybren A. Stüvel ae034d27d2 Deps: ensure osl/bin/oslc is using static libpng
Pass `-DLINKSTATIC=ON` to the OSL CMake, to ensure it statically links to
our libpng. Previously this was only applied on Windows, it's now on all
platforms.
2021-07-27 19:49:29 +02:00
Jeroen Bakker 07688ca2d2 Added YAML builtbot config file.
Builtbot is switching over from json to yaml. Both
configuration files should be kept in sync for now.

The json file will be removed when everything works as expected.
2021-07-27 15:06:43 +02:00
Sybren A. Stüvel c5e5ac4a17 Deps builder: OIIO/OSL/ISPC/OIDN/LLVM/Flex updates
This diff somewhat snowballed out of updating OIDN to 1.4.1 it had some
changes that allowed us to remove the arm hacks we had in place and
revert to using identical versions for a whole bunch of deps. But that
required an update to ISPC which needed a newer LLVM and if we're
updating LLVM we may as well update OSL, and when we update OSL, OIIO
may as well be dragged in soo......anyhow...

This diff updates:

LLVM 9.0.0 (11.0.1 for mac/arm) -> 12.0.0
OIIO 2.1.15.0 -> 2.2.15.1
OSL 1.11.10.0 -> 1.11.14.1
winflex_bison 2.5.5-> 2.5.24 (ispc needed newer bison, windows only dep)
OIDN 1.4.0 -> 1.4.1
ISPC v1.14.1(random hash for mac/arm) -> v1.16.0
Flex 2.6.4 (ISPC needed newer Flex than available on CentOS 7)

and removes most of the "special arm/mac" versions. I think just ssl and
embree are left with special versions.

notable changes:
@LazyDodo included some clang headers in the linux/mac harvest which are
needed to start writing custom clang based tooling like D9465 these were
already shipping on windows, but not the other platforms.

[macOS] Change the `LC_ID_DYLIB` of OpenMP  for {D11997}. This changes
where the executables look for dylibs.

Reviewed By: sebbas, LazyDodo

Differential Revision: https://developer.blender.org/D11748
2021-07-27 10:15:31 +02:00
Brecht Van Lommel cf74cd9367 Cycles: upgrade CUDA to 11.4
This fixes a performance regression on Ampere cards, on specific scenes like
classroom. For cycles-x there is little difference, but this is still helpful
for LTS releases, and we need to upgrade at some point anyway.
2021-07-26 19:46:51 +02:00
Ankit Meel a9121640be macOS Cleanup: Remove old version specific code
Reviewed By: #platform_macos, brecht
Differential Revision: https://developer.blender.org/D12021
2021-07-26 18:39:08 +05:30
Campbell Barton 265c3a4724 Cleanup: replace NB with NOTE in comments 2021-07-21 13:10:32 +10:00
Campbell Barton d35b14449e Cleanup: ensure one newline at end of file, strip trailing space 2021-07-15 17:53:26 +10:00
Campbell Barton 41e0a5b5ed CMake: update config for checking utility 2021-07-15 17:52:27 +10:00