Solves problem with different order of codesign server startup and
mount of network shares: avoids exception happening when server is
started prior to the mounts are ready.
Is achieved by replacing hard-coded signed/unsigned file names with
"<uuid>" which acts as a "request ID". This way multiple workers can
put their requests into a single directory without collisions. The
code sign server will handle the requests sequentially in an unknown
order.
The directory layout on worker goes as following:
<Worker>
<Builder Name>
blender.git/
build/
install/
lib/
Adding an extra <Builder Name> after build is redundant.
Differential Revision: https://developer.blender.org/D8045
The file subversion is no longer used in the Python API or user interface,
and is now internal to Blender.
User interface, Python API and file I/O metadata now use more consistent
formatting for version numbers. Official releases use "2.83.0", "2.83.1",
and releases under development use "2.90.0 Alpha", "2.90.0 Beta".
Some Python add-ons may need to lower the Blender version in bl_info to
(2, 83, 0) or (2, 90, 0) if they used a subversion number higher than 0.
https://wiki.blender.org/wiki/Reference/Release_Notes/2.83/Python_API#Compatibility
This change is in preparation of LTS releases, and also brings us more
in line with semantic versioning.
Fixes T76058.
Differential Revision: https://developer.blender.org/D7748
Seems like sometimes files are being only partially ready, which makes it so there
are unsigned files, failing to deliver fully signed bundle.
Now expected archive file size is stored into stamp file and is checked against
size of the archive file on another side.
There are some bare prints used for debugging, would need to switch it to a proper
logger (or to be removed).
From looking into builder's logs it seems that stamp file is picked
up prior to actual archive: sometimes worker reports missing archive
file, from a code path which is only possible if there is a stamp file.
Could be something with IO scheduling where bigger file is sent to
Samba server after smaller file.
Hopefully with this change this will not happen anymore.
It changes name to be blender-<version>-linux64.
Since CentOS is used as a base host for builds there is no real need
in specifying libc version. Is unlikely anything older could be used
anyway.
Also make bitness to be the same as windows. It is something what
users will read easier.
xz compresses 25% better than bz2, reducing download times and server load.
The numbers:
blender-2.80-linux-glibc217-x86_64.tar.bz2 (release): 134 886 174 bytes
with xz: 96 181 604 bytes (-28.7%)
with xz -9: 93 871 548 bytes (-30.4%)
blender-2.81-7c1fbe24ca33-linux-glibc217-x86_64.tar.bz2 (beta): 173 600 363 bytes
with xz: 133 100 664 bytes (-23.3%)
with xz -9: 129 534 124 bytes (-25.4%)
xz also decompresses more than twice as fast as bz2, however compression needs
four times as long (on my 7-year-old laptop 3-4 minutes instead of <1).
Also xz has become more common than bz2, e.g. Debian/Ubuntu deb packages have
been xz-compressed for years, so the dpkg package manager as well as systemd
and grub all depend on liblzma being present, whereas bz2 is becoming more and
more optional.
Current Linux archives also include the UID/GID of whatever user account
happens to be used for building by the blender.org infrastructure. If someone
then installs these archives as root e.g. to /usr/local/... and doesn't pay
full attention the files remain owned by a regular user, which is a serious
security issue. This patch fixes that by setting the UID/GID to 0.
Differential Revision: https://developer.blender.org/D6138
The script requires Python 3.7 as a very minimum, and CentOS is
only 3.6.
On macOC there was an access to a None object, due to missing
implementation of code signer on this platform.
This changes integrates code signing steps into a buildbot worker
process.
The configuration requires having a separate machine running with
a shared folder access between the signing machine and worker machine.
Actual signing is happening as a "POST-INSTALL" script run by CMake,
which allows to sign any binary which ends up in the final bundle.
Additionally, such way allows to avoid signing binaries in the build
folder (if we were signing as a built process, which iwas another
alternative).
Such complexity is needed on platforms which are using CPack to
generate final bundle: CPack runs INSTALL target into its own location,
so it is useless to run signing on a folder which is considered INSTALL
by the buildbot worker.
There is a signing script which can be used as a standalone tool,
making it possible to hook up signing for macOS's bundler.
There is a dummy Linux signer implementation, which can be activated
by returning True from mock_codesign in linux_code_signer.py.
Main purpose of this signer is to give an ability to develop the
scripts on Linux environment, without going to Windows VM.
The code is based on D6036 from Nathan Letwory.
Differential Revision: https://developer.blender.org/D6216
This wasn't an issue in the real buildbot environment since the
precompiled libraries are compiled with same ABI as the compiler
used for Blender build. But it was causing issues when building
Blender using buildbot scripts (for troubleshooting purposes)
on a machine with different default compiler ABI.
Usually ABI detection is happening in platform_unix.cmake when
detecting whether there are any precompiled libraries folder
available. This detection is not happening when library folder
is provided explicitly, expecting ABI to be setup explicitly
as well.
Removes custom logic from buildbot's packing step.
This also removes icons/ folder, but CMake was already copying the
icons to the root of the install folder.
The CMAKE_SOURCES variable is not yet initialized when the buildbot
configuration is read. This is similar to the include of full release
configuration happening earlier in the file.
Blender can only be run correctly from the install path since it requires Python
scripts, dynamic libraries and other files to be present. By default the install
path is the same as the build path, so it works anyway. But on the buildbot it
isn't. There was a workaround but it failed on Windows and macOS.
Now tests run from the install path. Detecting that path for ctest is more
complicated than I would like, but I couldn't find a better solution.
Ref T69541.
* Auto detect rc and release version cycle in BKE_blender_version.h.
* On Windows, generate zip and installer if a release is detected.
* On macOS, always generate a dmg instead of zip.
* Use standard package names without hash if a release is detected.
* Buildbot package names now match platform names in releases.
Ref T67056
Differential Revision: https://developer.blender.org/D5643
Brecht Van Lommel
Sergey Sharybin
Brecht Van Lommel
Campbell Barton
Jens
Brecht Van Lommel
Patrick Mours