Commit Graph

14 Commits

Author SHA1 Message Date
Brecht Van Lommel 335f40ebfa Tests: include device type in benchmark graph labels 2021-10-06 14:21:26 +02:00
Brecht Van Lommel cc653c9b02 Fix potential render tests error with invalid utf-8 characters
In general should not happen, but better to report the actual error instead
of the Python test code failing.
2021-09-28 20:50:38 +02:00
Brecht Van Lommel 0803119725 Cycles: merge of cycles-x branch, a major update to the renderer
This includes much improved GPU rendering performance, viewport interactivity,
new shadow catcher, revamped sampling settings, subsurface scattering anisotropy,
new GPU volume sampling, improved PMJ sampling pattern, and more.

Some features have also been removed or changed, breaking backwards compatibility.
Including the removal of the OpenCL backend, for which alternatives are under
development.

Release notes and code docs:
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles
https://wiki.blender.org/wiki/Source/Render/Cycles

Credits:
* Sergey Sharybin
* Brecht Van Lommel
* Patrick Mours (OptiX backend)
* Christophe Hery (subsurface scattering anisotropy)
* William Leeson (PMJ sampling pattern)
* Alaska (various fixes and tweaks)
* Thomas Dinges (various fixes)

For the full commit history, see the cycles-x branch. This squashes together
all the changes since intermediate changes would often fail building or tests.

Ref T87839, T87837, T87836
Fixes T90734, T89353, T80267, T80267, T77185, T69800
2021-09-21 14:55:54 +02:00
Brecht Van Lommel e43ecca016 Tests: measure time per frame in animation performance benchmark
Instead of a fixed number of frames, so that benchmarking takes about the
same time on any machine.
2021-09-20 19:03:22 +02:00
Brecht Van Lommel 42215d7cb8 Tests: more graceful handling of keyboard interrupting benchmarks
Leave current test result unchanged and stop executing immediately,
so it can be continued.
2021-09-10 16:48:26 +02:00
Brecht Van Lommel 5bfc3a3421 Fix error running benchmark script with environment variables for builds
Ref D12434
2021-09-08 19:58:27 +02:00
Brecht Van Lommel 6fc94d1848 Tests: updates for performance benchmarking
* Make "run" command (re-)run all tests, add "update" command to only
  run queued and outdated tests equivalent to the old "run" command.
* Support specifying environment variables for revisions, to easily
  compare multiple parameter values.
* Better sorting of revisions in graph.
2021-09-08 16:40:58 +02:00
Brecht Van Lommel 214e4aac97 Fix Python error in ./benchmark init after recent changes 2021-08-19 17:45:37 +02:00
Brecht Van Lommel 22b03e1c68 Fix Python error in benchmark executable detection after recent changes 2021-07-26 18:04:40 +02:00
Brecht Van Lommel 320f34af86 Tests: continue running benchmarks if some tests fail to build or run
Convenient when testing many revisions where some might be broken.
2021-07-22 16:35:00 +02:00
Brecht Van Lommel 19b597c55d Tests: improve finding of Blender executables in benchmarking
* Allow specifying a folder and automatically setting the proper executable
  name depending on the operating system
* Use executables from configs for listing devices instead of a blender
  command being available
2021-07-22 16:34:57 +02:00
Brecht Van Lommel 37a5ff4a84 Tests: support graphing peak memory in Cycles performance tests
The general graphing mechanism will create one graph for each output
variable. So it's not limited to time and memory, but that is what the
Cycles tests now output.
2021-07-14 16:12:10 +02:00
Campbell Barton 432bfbf7a3 Cleanup: pep8 2021-07-06 12:05:27 +10:00
Brecht Van Lommel dc3f46d96b Tests: performance testing framework
These are scripts for benchmarking Blender features on real-world .blend
files. They were originally written for benchmarking Cycles performance, and
were made generic so they can be used for more Blender features.

The benchmarks can be run locally by developers. But the plan is to also run
these as part of continuous integration to track performance over time.

Currently there are tests for Cycles rendering and .blend file loading.

Documentation:
https://wiki.blender.org/wiki/Tools/Tests/Performance

Main features:
* User created configurations to quickly run, re-run and analyze a selected
  subset of tests.
* Supports both benchmarking with existing builds, and automatic building of
  specified git commits, tags and branches.
* Generate HTML page with bar and line graphs from test results.
* Controlled using simple command line tool.
* For writing tests, convenient abstraction to run a Python function in Blender
  with arguments and return value.

Ref T74730

Differential Revision: https://developer.blender.org/D11662
2021-07-05 12:32:32 +02:00