This will allow much finer controll over how we copy data-blocks, from
full copy in Main database, to "lighter" ones (out of Main, inside an
already allocated datablock, etc.).
This commit also transfers a llot of what was previously handled by
per-ID-type custom code to generic ID handling code in BKE_library.
Hopefully will avoid in future inconsistencies and missing bits we had
all over the codebase in the past.
It also adds missing copying handling for a few types, most notably
Scene (which where using a fully customized handling previously).
Note that the type of allocation used during copying (regular in Main,
allocated but outside of Main, or not allocated by ID handling code at
all) is stored in ID's, which allows to handle them correctly when
freeing. This needs to be taken care of with caution when doing 'weird'
unusual things with ID copying and/or allocation!
As a final note, while rather noisy, this commit will hopefully not
break too much existing branches, old 'API' has been kept for the main
part, as a wrapper around new code. Cleaning it up will happen later.
Design task : T51804
Phab Diff: D2714
Noisy change, but safe, and better do it sooner than later if we are to
rework copying code. Also, previous commit shows this *is* useful to
catch some mistakes.
See this page for motivation and description of concepts:
https://github.com/Ichthyostega/blender/wiki
See this video for UI explanation and demonstration of usage
http://vimeo.com/blenderHack/stabilizerdemo
This proposal attempts to improve usability of Blender's image stabilization
feature for real-world footage esp. with moving and panning camera. It builds
upon the feature tracking to get a measurement of 2D image movement.
- Use a weighted average of movement contributions (instead of a median).
- Allow for rotation compensation and zoom (image scale) compensation.
- Allow to pick a different set of tracks for translation and for
rotation/zoom.
- Treat translation / rotation / zoom contributions systematically in a
similar way.
- Improve handling of partial tracking data with gaps and varying
start / end points.
- Have a user definable anchor frame and interpolate / extrapolate data to
avoid jumping back to "neutral" position when no tracking data is available.
- Support for travelling and panning shots by including an //intended//
position/rotation/zoom ("target position"). The idea is for these parameters
to be //animated// by the user, in order to supply an smooth, intended
camera movement. This way, we can keep the image content roughly in frame
even when moving completely away from the initial view.
A known shortcoming is that the pivot point for rotation compensation is set to
the translation compensated image center. This can produce spurious rotation on
travelling shots, which needs to be compensated manually (by animating the
target rotation parameter). There are several possible ways to address that
problem, yet all of them are considered beyond the scope of this improvement
proposal for now.
Own modifications:
- Restrict line length, it's really handy for split-view editing
- In motion tracking we prefer fully human-readable comments, meaning we
don't use doxygen with it's weird markup and comments are supposed to
start with capital and end with a full stop,
- Add explicit comparison of pointer to NULL.
Reviewers: sergey
Subscribers: kusi, kdawg, forest-house, mardy, Samoth, plasmasolutions, willolis, sebastian_k, hype, enetheru, sunboy, jta, leon_cheung
Maniphest Tasks: T49036
Differential Revision: https://developer.blender.org/D583
As title says... Copying tracking data from movieclip was not the simplest thing...
Reviewers: sergey
Differential Revision: https://developer.blender.org/D2126
The title actually tells it all, this commit switches Blender to use the new
autotrack API from Libmv.
From the user point of view it means that prediction model is now used when
tracking which gives really nice results.
All the other changes are not really visible for users, those are just frame
accessors, caches and so for the new API.
This commit makes it so CameraIntrinsics is no longer hardcoded
to use the traditional polynomial radial distortion model. Currently
the distortion code has generic logic which is shared between
different distortion models, but had no other models until now.
This moves everything specific to the polynomial radial distortion
to a subclass PolynomialDistortionCameraIntrinsics(), and adds a
new division distortion model suitable for cameras such as the
GoPro which have much stronger distortion due to their fisheye lens.
This also cleans up the internal API of CameraIntrinsics to make
it easier to understand and reduces old C-style code.
New distortion model is available in the Lens panel of MCE.
- Polynomial is the old well-known model
- Division is the new one which s intended to deal better with huge
distortion.
Coefficients of this model works independent from each other
and for division model one probably want to have positive values
to have a barrel distortion.
Switch the detector API to a single function which accepts
a float image and detector options. This makes usage of
feature detection more unified across different algorithms.
Options structure is pretty much straightforward and contains
detector to be used and all the detector-specific settings.
Also implemented Harris feature detection algorithm which
is not as fast as FAST one but is expected to detect more
robust feature points. It is also likely that less features
are detected, but better quality than quantity.
Blender will now use Harris detector by default, later we'll
remove FAST detector.
File tracking.c became rather huge and annoying to
maintain and it really contains several independent
areas of motrack pipeline.
Now we've got:
* tracking.c: general-purpose functions which are used
by blender, clip editor, RNA and so.
* tracking_detect.c: feature detection functions
(blender-side, logic is still in libmv).
* tracking_plane_tracker.c: blender-side 2D tracking logic.
* tracking_plane_tracker.c: plane track tracker.
* tracking_solver.c: functions for camera solving.
* tracking_stabilize.c: 2D stabilization functions.
* tracking_util.c: utility functions for all those files
and which shouldn't be public.
Added a weight slider to track which defines
how much particular track affects in a final
reconstruction. This weight is for sure
animateable.
Currently it affects on BA step only which in
most cases will work just fine.
The usecase of this slider is to have it set
to 1.0 most of the time where the track is
good, but blend it's weight down to 0 when
tracker looses the track. This will prevent
camera from jump.
Tutorial is to be done by Sebastian.
From the math point of view there're two cases:
- Clearing the keyframe between two other ones.
In this case tracker will first track plane from
left keyframe to right one without doing any kind
of blending. This will make plane stick to the
actual plane motion, but lead to possible jump
at the right keyframe.
Second step is to track from the right keyframe
to the left one with blending. This gives nice
transition at the point of second keyframe and
this mimics situation when you've been setting
keyframes from left to right.
- Clearing left-most/right-most keyframe.
In this case it's enough to only re-track the
plane without blending from the neighbor keyframe
without blending.
- Do plane re-evaluation only when transform is actually done.
Before this re-evaluation happened on every mouse move.
- Added a flag "Auto Keyframe" for the plane track, which does:
* If Auto Keyframe is enabled, then every manual edit of the
plane will create a new keyframe at current frame and update
plane motion between current frame and previous/next keyframe.
This now also implies blending detected motion with neighbor
keyframes, so there's no jump happening.
No automatic update on manual point tracks edit will happen.
* If auto Keyframe is disabled, then no keyframes are adding
to the plane and every plane tweak will re-evaluate in on
the whole frame range.
In this case manual tweaks to point tracks and re-tracking
them implies plane re-evaluation.
This commit includes all the changes made for plane tracker
in tomato branch.
Movie clip editor changes:
- Artist might create a plane track out of multiple point
tracks which belongs to the same track (minimum amount of
point tracks is 4, maximum is not actually limited).
When new plane track is added, it's getting "tracked"
across all point tracks, which makes it stick to the same
plane point tracks belong to.
- After plane track was added, it need to be manually adjusted
in a way it covers feature one might to mask/replace.
General transform tools (G, R, S) or sliding corners with
a mouse could be sued for this. Plane corner which
corresponds to left bottom image corner has got X/Y axis
on it (red is for X axis, green for Y).
- Re-adjusting plane corners makes plane to be "re-tracked"
for the frames sequence between current frame and next
and previous keyframes.
- Kayframes might be removed from the plane, using Shit-X
(Marker Delete) operator. However, currently manual
re-adjustment or "re-track" trigger is needed.
Compositor changes:
- Added new node called Plane Track Deform.
- User selects which plane track to use (for this he need
to select movie clip datablock, object and track names).
- Node gets an image input, which need to be warped into
the plane.
- Node outputs:
* Input image warped into the plane.
* Plane, rasterized to a mask.
Masking changes:
- Mask points might be parented to a plane track, which
makes this point deforming in a way as if it belongs
to the tracked plane.
Some video tutorials are available:
- Coder video: http://www.youtube.com/watch?v=vISEwqNHqe4
- Artist video: https://vimeo.com/71727578
This is mine and Keir's holiday code project :)
Hopefully it's more readable now. Took me a while to remmeber
all the stuff going on here while was looking into possibility
of implementing some feature here.
This means when you've got reconstructed scene assigned to a
3d camera (via camera solver constraint) and applies scale on
this camera from Ctrl-A menu, scale will be applied on the
reconstructed scene and reset camera size to identity.
This is very useful feature for scene orientation, when you'll
just scale camera by S in the viewport to match bundles
some points in the space, and then you'll easiy make camera
have identity scale (which is needed for nice working moblur
and other things mentioning by Sebastian :) without loosing
scale of bundles themselves.
Behavior of apply scale for cameras without clip assigned
to them does not change at all.
This operator will run a tracker from previous
keyframe to current frame for all selected markers.
Current markers positions are considering initial
position guess which could be updated by a tracker
for better match.
Useful in cases when feature disappears from the
frame and then appears again. Usage in this case
is the following:
- When feature point re-appeared on frame, manully
place marker on it.
- Use Refine Markers operation (which is in Track
panel) to allow tracker to find a better match.
Depending on direction of tracking use either
Forwards or Backwards refining. It's easy: if
trackign happens forwards, use Refine Frowards,
otherwise use Refine Backwards :)
This commit implements multi-threaded calculation of frames
when building proxies. Both scaling and undistortion steps
are now threaded.
Frames and proxy resolution are still handled one-by-one,
saving files after every single step. So if HDD is not so
fast, this commit could have not so much benefit.
Internal changes:
- Added IMB_scaleImBuf_threaded which scales given image
buffer in multiple threads and uses bilinear filtering.
- libmv's camera intrinsics now have SetThreads() method
which is used to specify how many OpenMP threads to use
for buffer distortion/undistortion.
And yeah, this code is using OpenMP for threading.
- Reshuffled a bit libmv-capi calls and added function
BKE_tracking_distortion_set_threads to specify number
of threads used by intrinscis.
Issue was caused by the way how pattern sampling happens in case of
anchored display: track offset is applying on search buffer which
means offset is rounding to an integer. Fractional pat of offset was
completely ignoring which lead to jumps in pattern buffer.
This was only a visualization issue in track preview widget.
Calculate dependent area based on distortion model rather than
using 15% of image resolution for this.
Some assumptions here:
- We're assuming real-life camera calibration is used here
- Maximal undistortion delta would be achieved on frame boundary
- Distortion fully goes inside frame
This makes it possible to approximate margin for distortion by
checking undistortion delta across frame boundary and use it
for dependent area of interest.
We do not use any formula-based equation here because we're likely
support other distortion models and in that case it'll be stupid
to try detecting formula here.
- Re-arrange functions in headers and implementation file to make them
more grouped by entity they're operating with. Also order of functions
in implementation file should match order of functions in header for
easier navigation.
- Rename some functions to match conventions of naming public functions.
- Some code de-duplication, still some room for improvements tho.
- Split main 2D tracking functions into smaller steps to make it more clear.
Accidentally OpenMP was disabled in some of previous commits, re-enable it.
It was a bit confusing to synchronize settings used in pre-calculated
dopesheet channels which was storing in tracking data with settings
used for display which is in space data.
This was initially done by converting one flags to other and checking
if space's settings matches pre-calculated one, but that had several
issues if two different dopesheet are using different settings:
- Channels would be re-calculated on every redraw for each of spaces
- Dopesheet operators could fail due to the could be using channels
calculated for other space.
That was also quite nasty code checking if requested settings matches
pre-calculated one.
Added option to use Grease Pencil datablock as a mask for pattern
when doing motion tracking. Option could be found in Tracking Settings
panel.
All strokes would be rasterized separately from each other and every
stroke is treating as a closed spline.
Also added option to apply a mask on track preview which is situated
just after B/B/W channel button under track preview.
Helps keeping features tracked when there's large scale happens
without need to manually re-adjust search area.
Currently using factor of pattern's boundbox scale, but probably
could be done in more accurate way?
===========================================
Major list of changes done in tomato branch:
- Add a planar tracking implementation to libmv
This adds a new planar tracking implementation to libmv. The
tracker is based on Ceres[1], the new nonlinear minimizer that
myself and Sameer released from Google as open source. Since
the motion model is more involved, the interface is
different than the RegionTracker interface used previously
in Blender.
The start of a C API in libmv-capi.{cpp,h} is also included.
- Migrate from pat_{min,max} for markers to 4 corners representation
Convert markers in the movie clip editor / 2D tracker from using
pat_min and pat_max notation to using the a more general, 4-corner
representation.
There is still considerable porting work to do; in particular
sliding from preview widget does not work correct for rotated
markers.
All other areas should be ported to new representation:
* Added support of sliding individual corners. LMB slide + Ctrl
would scale the whole pattern
* S would scale the whole marker, S-S would scale pattern only
* Added support of marker's rotation which is currently rotates
only patterns around their centers or all markers around median,
Rotation or other non-translation/scaling transformation of search
area doesn't make sense.
* Track Preview widget would display transformed pattern which
libmv actually operates with.
- "Efficient Second-order Minimization" for the planar tracker
This implements the "Efficient Second-order Minimization"
scheme, as supported by the existing translation tracker.
This increases the amount of per-iteration work, but
decreases the number of iterations required to converge and
also increases the size of the basin of attraction for the
optimization.
- Remove the use of the legacy RegionTracker API from Blender,
and replaces it with the new TrackRegion API. This also
adds several features to the planar tracker in libmv:
* Do a brute-force initialization of tracking similar to "Hybrid"
mode in the stable release, but using all floats. This is slower
but more accurate. It is still necessary to evaluate if the
performance loss is worth it. In particular, this change is
necessary to support high bit depth imagery.
* Add support for masks over the search window. This is a step
towards supporting user-defined tracker masks. The tracker masks
will make it easy for users to make a mask for e.g. a ball.
Not exposed into interface yet/
* Add Pearson product moment correlation coefficient checking (aka
"Correlation" in the UI. This causes tracking failure if the
tracked patch is not linearly related to the template.
* Add support for warping a few points in addition to the supplied
points. This is useful because the tracking code deliberately
does not expose the underlying warp representation. Instead,
warps are specified in an aparametric way via the correspondences.
- Replace the old style tracker configuration panel with the
new planar tracking panel. From a users perspective, this means:
* The old "tracking algorithm" picker is gone. There is only 1
algorithm now. We may revisit this later, but I would much
prefer to have only 1 algorithm. So far no optimization work
has been done so the speed is not there yet.
* There is now a dropdown to select the motion model. Choices:
* Translation
* Translation, rotation
* Translation, scale
* Translation, rotation, scale
* Affine
* Perspective
* The old "Hybrid" mode is gone; instead there is a toggle to
enable or disable translation-only tracker initialization. This
is the equivalent of the hyrbid mode before, but rewritten to work
with the new planar tracking modes.
* The pyramid levels setting is gone. At a future date, the planar
tracker will decide to use pyramids or not automatically. The
pyramid setting was ultimately a mistake; with the brute force
initialization it is unnecessary.
- Add light-normalized tracking
Added the ability to normalize patterns by their average value while
tracking, to make them invariant to global illumination changes.
Additional details could be found at wiki page [2]
[1] http://code.google.com/p/ceres-solver
[2] http://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.64/Motion_Tracker
All operators which changes tracking data now just tags dopsheet as outdated,
actual re-calculaiton of happens only when this information is actually needed
(like on dopesheet draw).
This makes things a bit faster when there's no dopesheet visible in current
screen and also makes it much easier to update dopesheet using dependency
graph.
Also renamed dopesheet_sort_order to dopesheet_sort_method in rna and internal
stuff which makes much more sense and also correlated with naming in
file browser.
Supported sorting by name, longest tracked segment and total tracked frames.
Internally tracks are stored in Tracking datablock, but sort order is
a clip editor space property and sorting happens on clip editor draw.
If there's no dopesheet opened with different sort orders it's not
a problem due to re-sorting wouldn't happen.
Also fixed draw issue of tracked segments introduced in previous commit.
- Displays dopesheet information for selected tracks, and currently does not
support any kind of editing.
- Changed regions to use the whole main region for such views as curves and dopesheet.
This allows to have own panels with tools/properties in this area.
- Active clip is getting synchronized between different clip editor editors in the
same screen, so updating of curve/dopesheet views happens automatically when one
changes current clip in one of this editors.
- Panels in toolbox and properties panels are now separated to rely on current view
mode, but some operators and poll functions still need to be updated.
- Added new screen called "Movie Tracking" where layout is configured to
display timeline, main clip window, curves and dopesheet.
- If track was hidded in clip editor it wasn't highlighted in 3d viewport
even if it's selected.
- If two tracks has got the same bundle coordinate and one of them is
selected it might not be highlighted in 3d viewport because of
draw order.
Bastien Montagne
Ichthyostega
Sergey Sharybin
Campbell Barton