For Blender this release is interesting because of:
- Covariance estimation (not used in Blender yet, but now we
might use it for keyframe selection instead of havingown
implementation).
- Significant performance improvements to loss function and
dense linear solvers and automatic differentiation.
Unfortunately, didn't notice speedup of tracking itself,
but camera reconstruction now happens around 2 times faster
on my laptop,
- Better inner iteration step acceptance and stopping.
Particle system code used global variable to sort hair by orig index,
which is not safe for threading at all.
Replaced this with usage of reentrant version of qsort, which is
now implemented in BLI. It was moved from recast navigation code
to BLI, so more areas could use it (if needed).
--
svn merge -r59086:59087 ^/branches/soc-2013-depsgraph_mt
This commit includes all the changes made for plane tracker
in tomato branch.
Movie clip editor changes:
- Artist might create a plane track out of multiple point
tracks which belongs to the same track (minimum amount of
point tracks is 4, maximum is not actually limited).
When new plane track is added, it's getting "tracked"
across all point tracks, which makes it stick to the same
plane point tracks belong to.
- After plane track was added, it need to be manually adjusted
in a way it covers feature one might to mask/replace.
General transform tools (G, R, S) or sliding corners with
a mouse could be sued for this. Plane corner which
corresponds to left bottom image corner has got X/Y axis
on it (red is for X axis, green for Y).
- Re-adjusting plane corners makes plane to be "re-tracked"
for the frames sequence between current frame and next
and previous keyframes.
- Kayframes might be removed from the plane, using Shit-X
(Marker Delete) operator. However, currently manual
re-adjustment or "re-track" trigger is needed.
Compositor changes:
- Added new node called Plane Track Deform.
- User selects which plane track to use (for this he need
to select movie clip datablock, object and track names).
- Node gets an image input, which need to be warped into
the plane.
- Node outputs:
* Input image warped into the plane.
* Plane, rasterized to a mask.
Masking changes:
- Mask points might be parented to a plane track, which
makes this point deforming in a way as if it belongs
to the tracked plane.
Some video tutorials are available:
- Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4
- Artist video: https://vimeo.com/71727578
This is mine and Keir's holiday code project :)
This adds a new Euclidean resection method, used to create the
initial reconstruction in the motion tracker, to libmv. The method
is based on the Procrustes PNP algorithm (aka "PPnP"). Currently
the algorithm is not connected with the motion tracker, but it
will be eventually since it supports initialization.
Having an initial guess when doing resection is important for
ambiguous cases where potentially the user could offer extra
guidance to the solver, in the form of "this point is in front of
that point".
--
svn merge -r58821:58822 ^/branches/soc-2011-tomato
Clean up inconsistencies in the libmv C API:
- All type identifiers are libmv_TypeName
- All function identifiers libmv_functionName
- Prefer libmv_nounVerb function names (e.g. libmv_featuresDestroy)
- Match Blender code formatting rather than Google
- Spelling corrections
Code review: https://codereview.appspot.com/11494044/
Implements an automatic keyframe selection algorithm which uses
couple of approaches to find out best keyframes candidates:
- First, slightly modifier Pollefeys's criteria is used, which
limits correspondence ration from 80% to 100%. This allows to
reject keyframe candidate early without doing heavy math in
cases there're not much common features with first keyframe.
- Second step is based on Geometric Robust Information Criteria
(aka GRIC), which checks whether features motion between
candidate keyframes is better defined by homography or
fundamental matrices.
To be a good keyframe candidate, fundamental matrix need to
define motion better than homography (in this case F-GRIC will
be smaller than H-GRIC).
This two criteria are well described in this paper:
http://www.cs.ait.ac.th/~mdailey/papers/Tahir-KeyFrame.pdf
- Final step is based on estimating reconstruction error of
a full-scene solution using candidate keyframes. This part
is based on the following paper:
ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/ECCV2004-TTHBAW.pdf
This step requires reconstruction using candidate keyframes
and obtaining covariance matrix of 3D points positions.
Reconstruction was done pretty much straightforward using
other simple pipeline routines, and for covariance estimation
pseudo-inverse of Hessian is used, which is in this case
(J^T * J)+, where + denotes pseudo-inverse.
Jacobian matrix is estimating using Ceres evaluate API.
This is also crucial to get rid of possible gauge ambiguity,
which is in our case made by zero-ing 7 (by gauge freedoms
number) eigen values in pseudo-inverse.
There're still room for improving and optimizing the code,
but we need some point to start with anyway :)
Thanks to Keir Mierle and Sameer Agarwal who assisted a lot
to make this feature working.
Will be reverted as soon as the x64 compiler is fixed.
For now it shouldn't have an impact on tracking performance. My test have shown no significant speed difference to official VC2008 build of 2.67.
- Added const modifiers where it makes sense and
helps keep code safe.
- Reshuffled argument to match <inputs>,<outputs>
convention on parameters.
- Pass values to ApplyRadialDistortionCameraIntrinsics
by a constant reference.
This will save lots of CPU ticks passing relatively
heavy jet objects to this function when running
bundle adjustment.
Makes code in tracking.cc much easier to understand and modify,
without worring to breck compulation with Libmv disabled.
It is still possible compilation will break due to libmv-capi
changes, but that's not happening so much often.
- Ensures fix for msvc2012 is applying correct.
- Some code cleanup to match libmv's code style.
- Do not include points which were intersect
behind the camera to a reconstruction.
- Includes changes needed for keyframe selection.
Additional changes:
- Cleaned up sources to reduce mess in some
big functions.
- Removed unused function from libmv c-api.
- Made functions naming more consistent.
- Use bool for internal stuff in tracking.c.
Shall be no functional changes :)
This check is actually redundant, because empty intrinsics
will have focal length of 1.0, which means original comment
about BundleIntrinsics was not truth.
It is possible that external user will send focal length of
zero to be refined, but blender prevents this from happening.
In cases keyframes are no so good, algebraic two frames construction
could produce result, for which more aggressive Ceres-based BA code
will fall to a solution for which points goes behind the camera,
which is not so nice.
Seems in newer Ceres returning false from cost functor wouldn't
abort solution, but will restrict solver from moving points behind
the camera.
Works fine in own tests, but requires more tests.
Made it so reconstructed scene always scaled in a way
that variance of camera centers is unity.
This solves "issues" when different keyframes will
give the same reprojection error but will give scenes
with different.scale, which could easily have been
considered as a bad keyframe combination.
This change is essential for automatic keyframe
selection algorithm to work reliable for user.
Remove stray BT_USE_SSE_IN_API definitions.
Was causing problems especially for 32 bit windows.
It's not quite clear why they were added in the first place since
this should be defined in btScalar.h, needs further investigation.
Thanks to Francisco De La Cruz (xercesblue) for looking into this.
Should fix [#35071] Bullet Convex Hull Crashes on Win32 with SSE
In some cases (was noticed on not good enough keyframe
pair) bundle adjuster could have moved bundles behind
the camera.
This could indeed lead to lower rewprojection error but
this is just pointless thing to do.
Now added check to residuals functor which will return
false to Ceres in cases point moved behind the camera
to prevent such issues.
This brings a fixes for threading issue in BLAS
making BA step more robust (there were some in-detemrinacy
caused by this threading issue).
Also brings some optimizations, which does not directly
affect on blender.
This commit bundles new libmv version from own branch
which brings fix for wrong parameter block used for
modal solver parameterization.
Fixes#34985: Crash with Motion tracker (Tripod Motion)
- Get rid of rotation matrix parameterization,
use angle-axis instead.
Also Joined rotation and translation into a
single parameter block.
This made minimization go significantly faster,
like 1.3x times in average.
- Fix first camera when bundling. This is to
address orientation ambiguity.
Reconstruction result could still vary in
size, but that's another issue to be addressed
later.
Additional change:
Split EuclideanBundleCommonIntrinsics into
smaller functions, so it's now a bit easier
to follow.
This made preview working but that broke internals
of tracking.
Namely, BlurredImageAndDerivativesChannels is giving
much more blurred image because it was assuming pixel
center is an integer position.
Guess other parts of libmv used to suffer because of
this issue.
Now pixel centering happens in blender side, and
libmv assumes integer position is a pixel center.
Thins brings up some speed improvements:
SPARSE_SCHUR is approx 1.3-1.5x times faster
ITERATIVE_SCHUR is approx 1.2x times faster
For blender this means camera solution go a bit
faster now. Would not have affect on tracking
speed.
- pass string size to BLI_timestr() to avoid possible buffer overrun.
- quiet warning for mingw.
- include guards for windows utf conversion funcs.
- fix for mistage in edge-angle-selection check.
- some style cleanup.
This commit implements multi-threaded calculation of frames
when building proxies. Both scaling and undistortion steps
are now threaded.
Frames and proxy resolution are still handled one-by-one,
saving files after every single step. So if HDD is not so
fast, this commit could have not so much benefit.
Internal changes:
- Added IMB_scaleImBuf_threaded which scales given image
buffer in multiple threads and uses bilinear filtering.
- libmv's camera intrinsics now have SetThreads() method
which is used to specify how many OpenMP threads to use
for buffer distortion/undistortion.
And yeah, this code is using OpenMP for threading.
- Reshuffled a bit libmv-capi calls and added function
BKE_tracking_distortion_set_threads to specify number
of threads used by intrinscis.
Jens Verwiebe
Joseph Mansfield
Sergey Sharybin
Thomas Dinges
Peter Schlaile
Campbell Barton
Jürgen Herrmann
Brecht Van Lommel
Sergej Reich