Fix T41258: Crash when entering edit mode while viewport render is enabled

The issue was caused by the render engine loading edit mesh, which re-allocates
mesh array which might be referenced by other object's derived meshed.

Worst thing about this is that updating render engine happens from the end of
scene update function, after all the objects are updated and so. This is needed
so render engine gets the update objects which is correct.

The only proper way to solve the issue is to make it so viewport engine does not
leave objects in inconsistent state, meaning nobody will reference to freed data.

In order to reach this we do edit mesh loading before running objects update so
all the objects which uses that mesh will have proper references in the derived
mesh.

This also solves old creepyness which happened before when having single object
in edit mode. tweaking it will calculate derived mesh as a part of scene update,
then this derived mesh will be freed by edit mesh loading and viewport will be
creating derived mesh again.

Now render engine is expected to do nothing with meshes which are in edit mode,
but they still need to load edit data for non0meshes. It's not really easy to
do from the BKE level because needed functions are implemented in the editor.

Thanks Campbell for the review!

Differential Revision: https://developer.blender.org/D697

1 files changed, 5 insertions, 0 deletions

diff --git a/source/blender/editors/object/object_edit.c b/source/blender/editors/object/object_edit.c
index 20e31e3909e..3b60bbbb37e 100644
--- a/source/blender/editors/object/object_edit.c
+++ b/source/blender/editors/object/object_edit.c
@@ -363,6 +363,11 @@ static bool ED_object_editmode_load_ex(Object *obedit, const bool freedata)
 		if (freedata) free_editMball(obedit);
 	}
 
+	/* Tag update so no access to freed data referenced from
+	 * derived cache will happen.
+	 */
+	DAG_id_tag_update((ID *)obedit->data, 0);
+
 	return true;
 }