| Age | Commit message (Collapse) | Author |
|---|
|
Issue was caused by blender internal accessing data from DNA during rendering.
There's no simple solution to make stuff thread safe, so for now simply restart
rendering on frame update.
|
|
Cases like using subsurfed object as a boolean operand can't be evaluated
on GPU and needs to have all the CCG on CPU.
This commit resolves existing configuration to survive, but new configurations
would need to have some sort of forced object update so all the data is being
moved on CPU if it was previously on GPU.
|
|
This commit integrates the work done so far on the new dependency graph system,
where goal was to replace legacy depsgraph with the new one, supporting loads of
neat features like:
- More granular dependency relation nature, which solves issues with fake cycles
in the dependencies.
- Move towards all-animatable, by better integration of drivers into the system.
- Lay down some basis for upcoming copy-on-write, overrides and so on.
The new system is living side-by-side with the previous one and disabled by
default, so nothing will become suddenly broken. The way to enable new depsgraph
is to pass `--new-depsgraph` command line argument.
It's a bit early to consider the system production-ready, there are some TODOs
and issues were discovered during the merge period, they'll be addressed ASAP.
But it's important to merge, because it's the only way to attract artists to
really start testing this system.
There are number of assorted documents related on the design of the new system:
* http://wiki.blender.org/index.php/User:Aligorith/GSoC2013_Depsgraph#Design_Documents
* http://wiki.blender.org/index.php/User:Nazg-gul/DependencyGraph
There are also some user-related information online:
* http://code.blender.org/2015/02/blender-dependency-graph-branch-for-users/
* http://code.blender.org/2015/03/more-dependency-graph-tricks/
Kudos to everyone who was involved into the project:
- Joshua "Aligorith" Leung -- design specification, initial code
- Lukas "lukas_t" Toenne -- integrating code into blender, with further fixes
- Sergey "Sergey" "Sharybin" -- some mocking around, trying to wrap up the
project and so
- Bassam "slikdigit" Kurdali -- stressing the new system, reporting all the
issues and recording/writing documentation.
- Everyone else who i forgot to mention here :)
|
|
This commit only adds callbacks which then later be used with major dependency
graph commit, keeping the upcoming commit more clean to follow.
Should be no functional changes so far still.
|
|
|
|
|
|
viewport preview.
Fix T39286: Display percentage ignored in Cycles viewport.
The threaded depsgraph update changes included a cleanup of the global
is_rendering flag, which was replaced by a general EvalContext being
passed to dupli functions.
Problem is that the global flag was true for viewport duplis before
(ugly hack), which was used as a check for generating dupli orco/UV from
mesh data layers. The new flag is stricter and only true for actual
renders, which disables these attributes and breaks the Cycles
Texture Coordinates and UVMap nodes.
The solution is to extend the simple for_render boolean to an enum:
* VIEWPORT: OpenGL viewport drawing (dupli tex coords omitted)
* PREVIEW: Viewport preview render (simplified modifiers)
* RENDER: Full render with all details and attributes
There are still some areas that need to be examined, in particular
modifiers seem to totally ignore the EvaluationContext!
Instead they generally execute without render params from the depsgraph
(BKE_object_handle_update_ex) and are built with render settings
explicitly.
Differential Revision: https://developer.blender.org/D613
|
|
|
|
This function used ugly hack with static variable which was
preventing some type checks in DAG nodes. Using this variable
form multiple threads is not considered safe, apparently.
Solved by moving this variable inside the DAGForest structure.
so it's global for the graph now, but different graphs does not
run into conflicts.
This required passing the forest to some functions, which doesn't
look so much nice, but don't want to spend time on making this
code look beautiful because it is really to be replaced by the
new dependency graph.
This is really bad bug actually which is must go to 'a'.
|
|
|
|
Issue was caused by cycles setting scene frame which will update scene for
all the layers (not just visible ones) which confuses depsgraph making
objects which are needed as dependency are not really evaluated.
Made it so setting frame via scene.frame_set() which check whether update
need to be flushed to an invisible objects and do this if so.
Not ideal solution but seems to be safest at this point.
|
|
|
|
It was an issue with early object update check which was
screwing up for second scene because of first one reset
ID recalc flags.
From the comment in the code about this:
We need to check all visible scenes, otherwise resetting
OB_ID changed flag will only work fine for first scene of
multiple visible and all the rest will skip update.
This could also lead to wrong behavior scene update handlers
because of missing ID datablock changed flags.
This is a bit of a bummer to allocate list here, but likely
it wouldn't become too much bad because it only happens when
objects were actually changed.
|
|
Issue was caused by some objects being in bMain and tagged
for update but not being in the DAG. This means objects
wouldn't be updated and their recalc flag remains untouched
triggering threaded for the next frame.
Solved by tweaking POST_UPDATE_HANDLER_WORKAROUND in the way
that it checks objects' recalc flags from the DAG, not from
the bMain. This will work a bit longer since DAG stored more
nodes than objects in the scene, but this code only runs in
cases when there're some objects tagged for update, which
keeps overall CPU usage on such a workaround pretty low.
Now CPU usage on 11a_comp scene from project Pampa went down
from ~15% down to ~5% (2,69 release uses ~%7).
Pointed by Thomas Dinges in IRC.
|
|
Graph traversal which is based on counting parents which are still
to be updated fails in cases there are cycles in the graph.
If there are cyclic dependencies in the scene all the objects from
the cycles will be updated in a single thread now one by one. This
makes blender behave the same way as it was before multi-threaded
DAG landed to master.
This needed to tweak depsgraph a bit so now dag_check_cycle() sets
is_acyclic field of DAG forest if there are cycles in the graph.
TODO: It might be possible to save some time on evaluation when
all the tagged objects were updated in multi-threaded DAG
traversal.
|
|
This solves threading conflict which happens when having
multiple objects using Curve Deform modifier with the same
curve datablock. This conflict was caused by the fact that
curve_deform_verts() used to temporary override curve's
flags to make it path is there.
Actually, it was setting CU_FOLLOW flag temporary which
was only used where_on_path() (only in terms that this
temporary assignment only affected this function) but it
is now commented out for a while, so no reason to set
this flag temporary, If it's ever to be done, we'll need
to pass flags as an additional function argument.
For the path creation i've extended DegNode structure
which now holds extra bits which indicates what additional
data depending on the graph topology is to be evaluated.
Currently this is only used to indicate that curve needs
path to be evaluated regardless to cu->flag state. This
is so Curve Deform modifier is always happy.
In the future this flag might also be used to indicate
whether bmesh verts are to update (see recent commit to
3-vertex parent crash fix) or to indicate that the object
is the motherball etc.
|
|
DAG node tagging was rather an experiment to make derived render working.
However, it ended up in a whole can of worms and need to be re-considered.
It is likely that regular object update tagging and scene update routines
are to be used for this.
Meanwhile no need to keep extra field in dag node. Would save us the whole
byte of the struct which we can use for other purposes meanwhile.
|
|
Summary:
Made objects update happening from multiple threads. It is a task-based
scheduling system which uses current dependency graph for spawning new
tasks. This means threading happens on object level, but the system is
flexible enough for higher granularity.
Technical details:
- Uses task scheduler which was recently committed to trunk
(that one which Brecht ported from Cycles).
- Added two utility functions to dependency graph:
* DAG_threaded_update_begin, which is called to initialize threaded
objects update. It will also schedule root DAG node to the queue,
hence starting evaluation process.
Initialization will calculate how much parents are to be evaluation
before current DAG node can be scheduled. This value is used by task
threads for faster detecting which nodes might be scheduled.
* DAG_threaded_update_handle_node_updated which is called from task
thread function when node was fully handled.
This function decreases num_pending_parents of node children and
schedules children with zero valency.
As it might have become clear, task thread receives DAG nodes and
decides which callback to call for it.
Currently only BKE_object_handle_update is called for object nodes.
In the future it'll call node->callback() from Ali's new DAG.
- This required adding some workarounds to the render pipeline.
Mainly to stop using get_object_dm() from modifiers' apply callback.
Such a call was only a workaround for dependency graph glitch when
rendering scene with, say, boolean modifiers before displaying
this scene.
Such change moves workaround from one place to another, so overall
hackentropy remains the same.
- Added paradigm of EvaluaitonContext. Currently it's more like just a
more reliable replacement for G.is_rendering which fails in some
circumstances.
Future idea of this context is to also store all the local data needed
for objects evaluation such as local time, Copy-on-Write data and so.
There're two types of EvaluationContext:
* Context used for viewport updated and owned by Main. In the future
this context might be easily moved to Window or Screen to allo
per-window/per-screen local time.
* Context used by render engines to evaluate objects for render purposes.
Render engine is an owner of this context.
This context is passed to all object update routines.
Reviewers: brecht, campbellbarton
Reviewed By: brecht
CC: lukastoenne
Differential Revision: https://developer.blender.org/D94
|
|
|
|
and add more documentation about the public functions. Also removed unused graph
traversal code and other minor unused functions.
|
|
valid on load).
|
|
besides performance in some cases.
* DAG_scene_sort is now removed and replaced by DAG_relations_tag_update in
most cases. This will clear the dependency graph, and only rebuild it right
before it's needed again when the scene is re-evaluated.
This is done because DAG_scene_sort is slow when called many times from
python operators. Further the scene argument is not needed because most
operations can potentially affect more than the current scene.
* DAG_scene_relations_update will now rebuild the dependency graph if it's not
there yet, and DAG_scene_relations_rebuild will force a rebuild for the rare
cases that need it.
* Remove various places where ob->recalc was set manually. This should go
through DAG_id_tag_update() in nearly all cases instead since this is now
a fast operation. Also removed DAG_ids_flush_update that goes along with
such manual tagging of ob->recalc.
|
|
|
|
|
|
|
|
added some missing functions too - which are not used yep but should be there for api completeness.
* CDDM_set_mloop
* CDDM_set_mpoly
* BLI_mempool_count
|
|
without the underscores these clogged up the namespace for autocompleation which was annoying.
|
|
worked by accident before, because of flags that weren't cleared properly. Now
moved the call to update render engines into scene_update_* itself.
|
|
the console, useful for debugging and understanding the relations.
|
|
* Adds two new python handlers: scene_update_pre() and scene_update_post()
These run before and after Blender does a scene update on making modifications
to the scene.
* Datablocks now have an is_updated property. This will be set to true in the
above callbacks if the datablock was tagged to be updated. This works for the
most common datablocks used for rendering: object, material, world, lamsp,
texture, mesh, curve.
* Datablock collections also have an is_updated property. If this is set, it
means one datablock of this type was added, removed or modified. It's also
useful as a quick check to avoid looping over all datablocks.
* RenderEngine.view_update() can also check these properties, for interactive
viewport rendering.
http://wiki.blender.org/index.php/Dev:2.6/Source/Render/UpdateAPI
|
|
http://markmail.org/message/fp7ozcywxum3ar7n
|
|
|
|
restore, would not get their dependencies updated when they became visible.
It happend with a shrinkwrap modifier in these reports, but could happen with
other modifiers too.
Now we keep track of which layers have ever been updated since load, and tag
objects on them to be recalculated when they become visible.
|
|
|
|
Surprising this wasnt noticed in a much more obvious case:
- Key Location, Move, Rotate, Undo-Rotate >> Resets to keyed location as well.
This was happening because DAG_on_load_update() was called on read_undosave(), flagging 'ob->adt->recalc |= ADT_RECALC_ANIM;'
Fix by adding an option to DAG_on_load_update(), not to recalculate time flags.
|
|
only tags the ID and does the actual flush/update delayed, before the next
redraw. For objects the update was already delayed, just flushing wasn't
yet.
This should help performance in python and animation editors, by making
calls to RNA property update quicker. Still need to add calls in a few
places where this was previously avoided due to bad performance.
|
|
own files.
No functional changes.
Where necessary extern "C" {} blocks have been added.
|
|
or get it from the context instead.
|
|
|
|
properly on file loading. Some things get preserved on file save/load,
like object matrices and armature poses, but other things need to be remade
like derivedmeshes and displists. The latter were not tagged for recalc on
load causing them to be made on countall or redraw typically, so not in the
right order and dependencies on hidden layer were not done at all.
Now these get tagged for recalc and flags flushed on load. There shouldn't
be much if any slowdown on opening existing files, if there is it should be
fixable.
|
|
Editors Modules
* render/ module added in editors, moved the preview render code there and
also shading related operators.
* physics/ module made more consistent with other modules. renaming files,
making a single physics_ops.c for operators and keymaps. Also move all
particle related operators here now.
* space_buttons/ now should have only operators relevant to the buttons
specificially.
Updates & Notifiers
* Material/Texture/World/Lamp can now be passed to DAG_id_flush_update,
which will go back to a callback in editors. Eventually these should
be in the depsgraph itself, but for now this gives a unified call for
doing updates.
* GLSL materials are now refreshed on changes. There's still various
cases missing,
* Preview icons now hook into this system, solving various update cases
that were missed before.
* Also fixes issue in my last commit, where some preview would not render,
problem is avoided in the new system.
Icon Rendering
* On systems with support for non-power of two textures, an OpenGL texture
is now used instead of glDrawPixels. This avoids problems with icons get
clipped on region borders. On my Linux desktop, this gives an 1.1x speedup,
and on my Mac laptop a 2.3x speedup overall in redrawing the full window,
with the default setup. The glDrawPixels implementation on Mac seems to
have a lot of overhread.
* Preview icons are now drawn using proper premul alpha, and never faded so
you can see them clearly.
* Also tried to fix issue with texture node preview rendering, globals can't
be used with threads reliably.
|
|
* Move function to compute visible screen layers to BKE.
* Use this now in the depsgraph, was still using this all layers
to flush. Still missing a way to get the current scene in
background mode..
* Also two more function to not require a scene pointer anymore:
* DAG_object_update_flags is now DAG_id_update_flags.
* DAG_ids_flush_update is now available next to DAG_scene_flush_update.
|
|
Notifiers
---------
Various fixes for wrong use of notifiers, and some new notifiers
to make things a bit more clear and consistent, with two notable
changes:
* Geometry changes are now done with NC_GEOM, rather than
NC_OBJECT|ND_GEOM_, so an object does need to be available.
* Space data now use NC_SPACE|ND_SPACE_*, instead of data
notifiers or even NC_WINDOW in some cases. Note that NC_SPACE
should only be used for notifying about changes in space data,
we don't want to go back to allqueue(REDRAW..).
Depsgraph
---------
The dependency graph now has a different flush call:
DAG_object_flush_update(scene, ob, flag)
is replaced by:
DAG_id_flush_update(id, flag)
It still works basically the same, one difference is that it now
also accepts object data (e.g. Mesh), again to avoid requiring an
Object to be available. Other ID types will simply do nothing at
the moment.
Docs
----
I made some guidelines for how/when to do which kinds of updates
and notifiers. I can't specify totally exact how to make these
decisions, but these are basically the guidelines I use. So, new
and updated docs are here:
http://wiki.blender.org/index.php/BlenderDev/Blender2.5/NotifiersUpdates
http://wiki.blender.org/index.php/BlenderDev/Blender2.5/DataNotifiers
|
|
* Optimized RNA property lookups and path resolving, still can be
much better, but now the 1000 IPO example on bf-taskforce25
runs at reasonable speed.
* Also an optimization in the depsgraph when dealing with many
objects, this was actually also a bottleneck here.
|
|
blocks that were previously missed; and b) greatly increase my
ohloh stats!
|
|
=======================
Caching and Baking:
- The point cache is now cleared on DAG_object_flush_update(), and not cleared for time dependency graph updates.
- There is now a Bake button instead of Protect. Also cache start and end frames were added to softbody and particles.
- The cloth autoprotect feature was removed.
- The Ctrl+B menu now also bakes cloth and particles next to softbody and fluids. Additionally there are now frree bake and free cache menu entries.
- The point cache api has been changed. There is now a PTCacheID struct for each point cache type that can be filled and then used to call the point cache functions.
- PointCache struct was added to DNA and is automatically allocated for each physics type.
- Soft body now supports Bake Editing just like cloth.
- Tried to make the systems deal consistently with time ipo's and offsets. Still not sure it all works correct, but too complicated to solve completely now.
Library Linking:
- Added some more warnings to prevent editing settings on library linked objects.
- Linked objects now read from the cache located next to the original library file, and never write to it. This restores old behavior for softbodies. For local simulation the mesh and not the object should be linked.
- Dupligroups and proxies can't create local point caches at the moment, how to implement that I'm not sure. We probably need a proxy point cache for that to work (ugh).
Physics UI:
- Renamed deflection panel to collision for consistency and reorganized the buttons. Also removed some softbody collision buttons from the softbody panel that were duplicated in this panel for cloth.
- Tweaked field panel buttons to not jump around when changing options.
- Tabbing e.g. Soft Body Collision into the Soft Body panel, it now only shows Collision to make the panel names readable.
- I tried to make enabled/disabling physics more consistent, since all three system did things different. Now the two modifier buttons to enable the modifier for the viewport and rendering are also duplicated in the physics panels. Toggling the Soft Body and Cloth buttons now both remove their modifiers.
- Fixed modifier error drawing glitch.
Particles:
- Particles are now recalculated more often than before. Previously it did partial updates based on the changes, but that doesn't work well with DAG_object_flush_update() ..
- Fixed memory leak loading keyed particle system. Now keys are not written to file anymore but always created after loading.
- Make particle threads work with autothreads.
Continue Physics:
- The timeline play now has a Continue Physics option in the playback menu, which keeps the simulations going without writing them to the cache.
- This doesn't always work that well, some changes are not immediately updated, but this can be improved later. Still it's fun to get a feel for the physics.
Todo:
- Point cache can get out of sync with and undo and changing a file without saving it.
- Change the point cache file format to store a version (so old point cache files can be either converted or at least ignored), and to do correct endian conversion.
- Menu item and/or buttons for Ctrl+B.
- A system("rm ..") was changed to remove() since the former is very slow for clearing point caches. These system() calls were already giving trouble in a bug in the tracker, but really most use of this system("") should be changed and tested.
- The Soft Body Collision and Clot Collision panel titles don't mention there's point cache settings there too, doing that makes them unreadable with the default panel setup.. but may need to make the names longer anyway.
|
|
-> Any Group Duplicate now can get local timing and local NLA override. This
enables to control the entire animation system of the Group.
Two methods for this have been implemented.
1) The quick way: just give the duplicator a "Startframe" offset.
2) Advanced: in the NLA Editor you can add ActionStrips to the duplicator
to override NLA/action of any Grouped Object.
For "Group NLA" to work, an ActionStrip needs to know which Object in a
group it controls. On adding a strip, the code checks if an Action was
already used by an Object in the Group, and assigns it automatic to that
Object.
You can also set this in the Nkey "Properties" panel for the strip.
Change in NLA: the SHIFT+A "Add strip" command now always adds strips to
the active Object. (It used to check where mouse was). This allows to add
NLA strips to Objects that didn't have actions/nla yet.
Important note: In Blender, duplicates are fully procedural and generated
on the fly for each redraw. This means that redraw speed equals to stepping
through frames, when using animated Duplicated Groups.
-> Recoded entire duplicator system
The old method was antique and clumsy, using globals and full temporal
copies of Object. The new system is nicer in control, faster, and since it
doesn't use temporal object copies anymore, it works better with Derived
Mesh and DisplayList and rendering.
By centralizing the code for duplicating, more options can be easier added.
Features to note:
- Duplicates now draw selected/unselected based on its Duplicator setting.
- Same goes for the drawtype (wire, solid, selection outline, etc)
- Duplicated Groups can be normally selected too
Bonus goodie: SHIFT+A (Toolbox) now has entry "Add group" too, with a
listing of all groups, allowing to add Group instances immediate.
-> Library System
- SHIFT+F4 data browse now shows the entire path for linked data
- Outliner draws Library Icons to denote linked data
- Outliner operation added: "Make Local" for library data.
- Outliner now also draws Groups in regular view, allowing to unlink too.
-> Fixes
- depsgraph missed signal update for bone-parented Objects
- on reading file, the entire database was tagged to "recalc" fully,
causing unnecessary slowdown on reading.
Might have missed stuff... :)
|
|
In PoseMode, press Wkey or use the Pose pulldown menu. It calculates the
positions of all selected Bone end points, over the time as indicated with
the Scene start/end frame. This then is drawn as a path, with little black
dots on every frame, and a white dot on every 10 frames.
Paths are not saved in files, and not calculated automatic yet on changes.
To make this relative fast, but also reliable, I had to add a new method
in the Dependency graph system, to find exactly (and only) these parents
of an Object that influence its position. This is needed because the path
should show the actual global coordinates of the entire animation system.
|
|
When using 3d window "unlocked" (own layer and own camera possible), the
dependency updates didn't work, these were only using Scene layers.
Changed the dependency graph call DAG_scene_flush_update()
to accept an argument for all visible layers in a Screen.
|
|
Aim was to get a total refresh of the animation system. This
is needed because;
- we need to upgrade it with 21st century features
- current code is spaghetti/hack combo, and hides good design
- it should become lag-free with using dependency graphs
A full log, with complete code API/structure/design explanation
will follow, that's a load of work... so here below the list with
hot changes;
- The entire object update system (matrices, geometry) is now
centralized. Calls to where_is_object and makeDispList are
forbidden, instead we tag objects 'changed' and let the
depgraph code sort it out
- Removed all old "Ika" code
- Depgraph is aware of all relationships, including meta balls,
constraints, bevelcurve, and so on.
- Made depgraph aware of relation types and layers, to do smart
flushing of 'changed' events. Nothing gets calculated too often!
- Transform uses depgraph to detect changes
- On frame-advance, depgraph flushes animated changes
Armatures;
Almost all armature related code has been fully built from scratch.
It now reveils the original design much better, with a very clean
implementation, lag free without even calculating each Bone more than
once. Result is quite a speedup yes!
Important to note is;
1) Armature is data containing the 'rest position'
2) Pose is the changes of rest position, and always on object level.
That way more Objects can use same Pose. Also constraints are in Pose
3) Actions only contain the Ipos to change values in Poses.
- Bones draw unrotated now
- Drawing bones speedup enormously (10-20 times)
- Bone selecting in EditMode, selection state is saved for PoseMode,
and vice-versa
- Undo in editmode
- Bone renaming does vertexgroups, constraints, posechannels, actions,
for all users of Armature in entire file
- Added Bone renaming in NKey panel
- Nkey PoseMode shows eulers now
- EditMode and PoseMode now have 'active' bone too (last clicked)
- Parenting in EditMode' CTRL+P, ALT+P, with nice options!
- Pose is added in Outliner now, with showing that constraints are in
the Pose, not Armature
- Disconnected IK solving from constraints. It's a separate phase now,
on top of the full Pose calculations
- Pose itself has a dependency graph too, so evaluation order is lag free.
TODO NOW;
- Rotating in Posemode has incorrect inverse transform (Martin will fix)
- Python Bone/Armature/Pose API disabled... needs full recode too
(wait for my doc!)
- Game engine will need upgrade too
- Depgraph code needs revision, cleanup, can be much faster!
(But, compliments for Jean-Luc, it works like a charm!)
- IK changed, it now doesnt use previous position to advance to next
position anymore. That system looks nice (no flips) but is not well
suited for NLA and background render.
TODO LATER;
We now can do loadsa new nifty features as well; like:
- Kill PoseMode (can be option for armatures itself)
- Make B-Bones (Bezier, Bspline, like for spines)
- Move all silly button level edit to 3d window (like CTRL+I = add
IK)
- Much better & informative drawing
- Fix action/nla editors
- Put all ipos in Actions (object, mesh key, lamp color)
- Add hooks
- Null bones
- Much more advanced constraints...
Bugfixes;
- OGL render (view3d header) had wrong first frame on anim render
- Ipo 'recording' mode had wrong playback speed
- Vertex-key mode now sticks to show 'active key', until frame change
-Ton-
|
