Commit Graph

43 Commits

Author SHA1 Message Date
Campbell Barton e955c94ed3 License Headers: Set copyright to "Blender Authors", add AUTHORS
Listing the "Blender Foundation" as copyright holder implied the Blender
Foundation holds copyright to files which may include work from many
developers.

While keeping copyright on headers makes sense for isolated libraries,
Blender's own code may be refactored or moved between files in a way
that makes the per file copyright holders less meaningful.

Copyright references to the "Blender Foundation" have been replaced with
"Blender Authors", with the exception of `./extern/` since these this
contains libraries which are more isolated, any changed to license
headers there can be handled on a case-by-case basis.

Some directories in `./intern/` have also been excluded:

- `./intern/cycles/` it's own `AUTHORS` file is planned.
- `./intern/opensubdiv/`.

An "AUTHORS" file has been added, using the chromium projects authors
file as a template.

Design task: #110784

Ref !110783.
2023-08-16 00:20:26 +10:00
Campbell Barton 89811cb96f Cleanup: unused-imports in tests/ 2023-07-25 21:43:53 +10:00
Campbell Barton 932a85a24f Fix tests failing from errors in the users startup file
Tests should never depend on the users startup.blend which can have
settings that interfere with tests.

Failure to load the user startup.blend for e.g. prevented
bl_alembic_io_test from passing.
2023-07-25 21:20:10 +10:00
Sergey Sharybin 4f9a9abcf1 Merge branch 'blender-v3.6-release' 2023-06-15 13:40:38 +02:00
Sergey Sharybin 2d3c9bcee8 Benchmark: Pack python expression into a single line
There should be no functional changes for the typical usecase,
but it allows to have more tricky setups like pointing to a
BAT script to override some configuration.

The issue is that BAT scripts do not support new lines in the
command line arguments. That's where single-line python expression
helps.

For example, it is possible to point benchmark script to a blender.bat
which contains

    blender.exe --python-expr "import bpy; bpy.context.preferences.addons['cycles'].preferences.use_oneapirt = False" %*

to have side-by-side numbers of oneAPI with and without HW RT.

Without this change the %* is which did not work: the BAT
script did not "see" part of the command line past the new line.

Pull Request: https://projects.blender.org/blender/blender/pulls/109006
2023-06-15 13:40:17 +02:00
Campbell Barton 65f99397ec License headers: use SPDX-FileCopyrightText in all sources 2023-06-15 13:35:34 +10:00
Sergey Sharybin fc440c4fa0 Fix monster performance benchmark run on Windows
The scene contains some interesting names, which requires to be
written as utf-8. And on Windows file descriptor is not guaranteed
to be using utf-8. Or, will error out if the invalid utf-8 sequence
is written.

This change makes it so running benchmarks on windows it fully successful.

Pull Request: https://projects.blender.org/blender/blender/pulls/108982
2023-06-14 18:11:22 +02:00
Sergey Sharybin 5115afaca2 Rename benchmark to benchmark.py
Windows does not really have an idea of shebangs, and it needs to
go via a file extension to see that the script is to be executed
by Python.

This change simplifies execution from `python3 benchmark ...`
to `benchmark.py ...`.

Pull Request: https://projects.blender.org/blender/blender/pulls/108971
2023-06-14 12:41:24 +02:00
Patrick Mours 8a32d56056 Tests: Fix device list of benchmark script only showing a single GPU
Pull Request #104583
2023-02-10 19:38:37 +01:00
Sergey Sharybin bd6b0bac88 Update references to the new projects platform and main branch 2023-02-07 14:18:19 +01:00
Jacques Lucke d49b850399 Tests: run geometry nodes benchmarks multiple times to reduce noise
Ideally, we would also get variance information out of the test, but that
seems a bit more complex to implement. For now just run the test a couple
of times and average the timings.

The test now runs between 5 and 100 times, depending on how long it
to run the test once.
2023-01-06 13:04:49 +01:00
Jacques Lucke 006e905957 Tests: support running benchmark when Blender prints unrelated text
I noticed that sometimes the geometry nodes benchmark would not work
if there were some left-over debug prints in the code. The reason it did
not work was because the prints were mixed with the test output print.
I also tried using explicit flusing and `atexit` for printing the test output,
but that didn't solve it. Just printing additional newlines to better separate
the test output from other printed output helped though.
2023-01-06 13:04:49 +01:00
Hans Goudey 3a41e0f611 Tests: Automated geometry nodes benchmark
Add a script for a very simple object evaluation benchmark.
There could be more advanced ways of measuring the time
per-node or per modifier, but this just loads the file, tags
the active object for a reevaluation, and times how long
that takes.

Differential Revision: https://developer.blender.org/D16604
2022-11-26 17:15:55 -06:00
Brecht Van Lommel cc12f3e0ba Cleanup: fix various typos
Contributed by luzpaz.

Differential Revision: https://developer.blender.org/D15588
2022-08-03 19:28:05 +02:00
Campbell Barton feeb8310c8 Cleanup: format 2022-06-30 12:14:23 +10:00
Jeroen Bakker 66f826ae85 Benchmark: Add eevee viewport playback tests.
This commit adds the ability to test Eevee viewport playback performance tests.

Tests should be placed in `lib/benchmarks/eevee/*/*.blend`. {rBL62962} added
initial test files. See https://wiki.blender.org/wiki/Tools/Tests/Performance how
to set it up.

To record the playback performance the test start the viewport playback, and adds
a post frame change handler.

This handler will go over the next steps:

* Ensures the viewport is set to rendered mode.
* Wait for shaders to be compiled. Utilizes `bpy.app.is_job_running` function when
  available (v3.3) to wait for shader compilation to finish. When not available will wait
  for one minute.
* Draw several warmup frames
* Record for 10 seconds tracking the number of frames drawn and performance counters.
* When ready print the result to the console. The results will be extracted when the
  benchmark has run.

## Example report
```
                                         master               v3.0                 v3.1                 v3.2
T88219                                   0.0860s              0.0744s              0.0744s              0.0851s
blender290-fox                           1.3056s              0.8744s              0.7994s              1.2809s
```

{F13232387}

Reviewed By: brecht, fclem

Maniphest Tasks: T99136

Differential Revision: https://developer.blender.org/D15302
2022-06-29 07:55:51 +02:00
Campbell Barton bba757ef81 Cleanup: various minor changes
- Add missing doxy-section for Apply Parent Inverse Operator
- Use identity for None comparison in Python.
- Remove newline from operator doc-strings.
- Use '*' prefix multi-line C comment blocks.
- Separate filenames from doc-strings.
- Remove break after return.
2022-04-24 13:41:03 +10:00
Campbell Barton 65a1fcdaf7 Cleanup: run autopep8 on tests/performance/benchmark
This file was skipped by source/tools/utils/autopep8_clean.py
since it doesn't have a .py extension, running the autopep8 tool
recursively detects Python scripts without extensions.
2022-04-20 17:07:46 +10:00
Campbell Barton 3035235def Cleanup: run autopep8 on tests/ 2022-04-19 15:25:55 +10:00
Campbell Barton c434782e3a File headers: SPDX License migration
Use a shorter/simpler license convention, stops the header taking so
much space.

Follow the SPDX license specification: https://spdx.org/licenses

- C/C++/objc/objc++
- Python
- Shell Scripts
- CMake, GNUmakefile

While most of the source tree has been included

- `./extern/` was left out.
- `./intern/cycles` & `./intern/atomic` are also excluded because they
  use different header conventions.

doc/license/SPDX-license-identifiers.txt has been added to list SPDX all
used identifiers.

See P2788 for the script that automated these edits.

Reviewed By: brecht, mont29, sergey

Ref D14069
2022-02-11 09:14:36 +11:00
Brecht Van Lommel 8ae6995e98 Tests: show more precise numbers in benchmark charts 2022-01-14 15:17:22 +01:00
Brecht Van Lommel 7d5ef64bfb Cleanup: fix typos in comments and docs
Contributed by luzpaz.

Differential Revision: https://developer.blender.org/D13264
2021-11-19 12:46:49 +01:00
Brecht Van Lommel 1b55b911f2 Merge branch 'blender-v3.0-release' 2021-11-12 20:04:05 +01:00
Brecht Van Lommel 71131b4969 Tests: disable audio system for performance tests 2021-11-12 20:03:46 +01:00
Hans Goudey 4648c4990c Merge branch 'blender-v3.0-release' 2021-11-09 13:08:36 -06:00
Ray Molenkamp 7c25399576 tests/benchmarks: Fix operation on windows
The test script did not work on windows
since it had some trouble importing the
api module on the blender side of things.

turning the file path to the module into
a raw string literal sidesteps the
backslash issue in the path.

Differential Revision: https://developer.blender.org/D13163

Reviewed by: brecht
2021-11-09 09:58:48 -07:00
Campbell Barton 27b37517f8 Cleanup: use static sets 2021-11-08 17:00:36 +11:00
Campbell Barton c516659b5e Cleanup: use static sets 2021-11-08 14:52:08 +11:00
Brecht Van Lommel fc36772b06 Tests: minor updates to benchmark script for running on buildbot
* graph command accepts folder of json files as input
* reset command clears log files
2021-10-28 22:41:40 +02:00
Brecht Van Lommel 335f40ebfa Tests: include device type in benchmark graph labels 2021-10-06 14:21:26 +02:00
Brecht Van Lommel cc653c9b02 Fix potential render tests error with invalid utf-8 characters
In general should not happen, but better to report the actual error instead
of the Python test code failing.
2021-09-28 20:50:38 +02:00
Brecht Van Lommel 0803119725 Cycles: merge of cycles-x branch, a major update to the renderer
This includes much improved GPU rendering performance, viewport interactivity,
new shadow catcher, revamped sampling settings, subsurface scattering anisotropy,
new GPU volume sampling, improved PMJ sampling pattern, and more.

Some features have also been removed or changed, breaking backwards compatibility.
Including the removal of the OpenCL backend, for which alternatives are under
development.

Release notes and code docs:
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles
https://wiki.blender.org/wiki/Source/Render/Cycles

Credits:
* Sergey Sharybin
* Brecht Van Lommel
* Patrick Mours (OptiX backend)
* Christophe Hery (subsurface scattering anisotropy)
* William Leeson (PMJ sampling pattern)
* Alaska (various fixes and tweaks)
* Thomas Dinges (various fixes)

For the full commit history, see the cycles-x branch. This squashes together
all the changes since intermediate changes would often fail building or tests.

Ref T87839, T87837, T87836
Fixes T90734, T89353, T80267, T80267, T77185, T69800
2021-09-21 14:55:54 +02:00
Brecht Van Lommel e43ecca016 Tests: measure time per frame in animation performance benchmark
Instead of a fixed number of frames, so that benchmarking takes about the
same time on any machine.
2021-09-20 19:03:22 +02:00
Brecht Van Lommel 42215d7cb8 Tests: more graceful handling of keyboard interrupting benchmarks
Leave current test result unchanged and stop executing immediately,
so it can be continued.
2021-09-10 16:48:26 +02:00
Brecht Van Lommel 5bfc3a3421 Fix error running benchmark script with environment variables for builds
Ref D12434
2021-09-08 19:58:27 +02:00
Brecht Van Lommel 6fc94d1848 Tests: updates for performance benchmarking
* Make "run" command (re-)run all tests, add "update" command to only
  run queued and outdated tests equivalent to the old "run" command.
* Support specifying environment variables for revisions, to easily
  compare multiple parameter values.
* Better sorting of revisions in graph.
2021-09-08 16:40:58 +02:00
Brecht Van Lommel 214e4aac97 Fix Python error in ./benchmark init after recent changes 2021-08-19 17:45:37 +02:00
Brecht Van Lommel 22b03e1c68 Fix Python error in benchmark executable detection after recent changes 2021-07-26 18:04:40 +02:00
Brecht Van Lommel 320f34af86 Tests: continue running benchmarks if some tests fail to build or run
Convenient when testing many revisions where some might be broken.
2021-07-22 16:35:00 +02:00
Brecht Van Lommel 19b597c55d Tests: improve finding of Blender executables in benchmarking
* Allow specifying a folder and automatically setting the proper executable
  name depending on the operating system
* Use executables from configs for listing devices instead of a blender
  command being available
2021-07-22 16:34:57 +02:00
Brecht Van Lommel 37a5ff4a84 Tests: support graphing peak memory in Cycles performance tests
The general graphing mechanism will create one graph for each output
variable. So it's not limited to time and memory, but that is what the
Cycles tests now output.
2021-07-14 16:12:10 +02:00
Campbell Barton 432bfbf7a3 Cleanup: pep8 2021-07-06 12:05:27 +10:00
Brecht Van Lommel dc3f46d96b Tests: performance testing framework
These are scripts for benchmarking Blender features on real-world .blend
files. They were originally written for benchmarking Cycles performance, and
were made generic so they can be used for more Blender features.

The benchmarks can be run locally by developers. But the plan is to also run
these as part of continuous integration to track performance over time.

Currently there are tests for Cycles rendering and .blend file loading.

Documentation:
https://wiki.blender.org/wiki/Tools/Tests/Performance

Main features:
* User created configurations to quickly run, re-run and analyze a selected
  subset of tests.
* Supports both benchmarking with existing builds, and automatic building of
  specified git commits, tags and branches.
* Generate HTML page with bar and line graphs from test results.
* Controlled using simple command line tool.
* For writing tests, convenient abstraction to run a Python function in Blender
  with arguments and return value.

Ref T74730

Differential Revision: https://developer.blender.org/D11662
2021-07-05 12:32:32 +02:00