Collections and groups unification

OVERVIEW

* In 2.7 terminology, all layers and groups are now collection datablocks.
* These collections are nestable, linkable, instanceable, overrideable, ..
  which opens up new ways to set up scenes and link + override data.
* Viewport/render visibility and selectability are now a part of the collection
  and shared across all view layers and linkable.
* View layers define which subset of the scene collection hierarchy is excluded
  for each. For many workflows one view layer can be used, these are more of an
  advanced feature now.

OUTLINER

* The outliner now has a "View Layer" display mode instead of "Collections",
  which can display the collections and/or objects in the view layer.
* In this display mode, collections can be excluded with the right click menu.
  These will then be greyed out and their objects will be excluded.
* To view collections not linked to any scene, the "Blender File" display mode
  can be used, with the new filtering option to just see Colleciton datablocks.
* The outliner right click menus for collections and objects were reorganized.
* Drag and drop still needs to be improved. Like before, dragging the icon or
  text gives different results, we'll unify this later.

LINKING AND OVERRIDES

* Collections can now be linked into the scene without creating an instance,
  with the link/append operator or from the collections view in the outliner.
* Collections can get static overrides with the right click menu in the outliner,
  but this is rather unreliable and not clearly communicated at the moment.
* We still need to improve the make override operator to turn collection instances
  into collections with overrides directly in the scene.

PERFORMANCE

* We tried to make performance not worse than before and improve it in some
  cases. The main thing that's still a bit slower is multiple scenes, we have to
  change the layer syncing to only updated affected scenes.
* Collections keep a list of their parent collections for faster incremental
  updates in syncing and caching.
* View layer bases are now in a object -> base hash to avoid quadratic time
  lookups internally and in API functions like visible_get().

VERSIONING

* Compatibility with 2.7 files should be improved due to the new visibility
  controls. Of course users may not want to set up their scenes differently
  now to avoid having separate layers and groups.
* Compatibility with 2.8 is mostly there, and was tested on Eevee demo and Hero
  files. There's a few things which are know to be not quite compatible, like
  nested layer collections inside groups.
* The versioning code for 2.8 files is quite complicated, and isolated behind
  #ifdef so it can be removed at the end of the release cycle.

KNOWN ISSUES

* The G-key group operators in the 3D viewport were left mostly as is, they
  need to be modified still to fit better.
* Same for the groups panel in the object properties. This needs to be updated
  still, or perhaps replaced by something better.
* Collections must all have a unique name. Less restrictive namespacing is to
  be done later, we'll have to see how important this is as all objects within
  the collections must also have a unique name anyway.
* Full scene copy and delete scene are exactly doing the right thing yet.

Differential Revision: https://developer.blender.org/D3383

https://code.blender.org/2018/05/collections-and-groups/

1 files changed, 169 insertions, 39 deletions

diff --git a/source/blender/makesrna/intern/rna_group.c b/source/blender/makesrna/intern/rna_group.c
index bf64bb1181f..0f62d8d20d8 100644
--- a/source/blender/makesrna/intern/rna_group.c
+++ b/source/blender/makesrna/intern/rna_group.c
@@ -41,11 +41,21 @@
 #include "DNA_scene_types.h"
 #include "DNA_object_types.h"
 
-#include "BKE_group.h"
+#include "DEG_depsgraph.h"
+
+#include "BKE_collection.h"
+#include "BKE_layer.h"
 
 #include "WM_api.h"
 
-static PointerRNA rna_Group_objects_get(CollectionPropertyIterator *iter)
+static void rna_Collection_all_objects_begin(CollectionPropertyIterator *iter, PointerRNA *ptr)
+{
+	Collection *collection = (Collection *)ptr->data;
+	ListBase collection_objects = BKE_collection_object_cache_get(collection);
+	rna_iterator_listbase_begin(iter, &collection_objects, NULL);
+}
+
+static PointerRNA rna_Collection_all_objects_get(CollectionPropertyIterator *iter)
 {
 	ListBaseIterator *internal = &iter->internal.listbase;
 
@@ -54,89 +64,209 @@ static PointerRNA rna_Group_objects_get(CollectionPropertyIterator *iter)
 	return rna_pointer_inherit_refine(&iter->parent, &RNA_Object, base->object);
 }
 
-static void rna_Group_objects_link(Group *group, ReportList *reports, Object *object)
+static void rna_Collection_objects_begin(CollectionPropertyIterator *iter, PointerRNA *ptr)
+{
+	Collection *collection = (Collection *)ptr->data;
+	rna_iterator_listbase_begin(iter, &collection->gobject, NULL);
+}
+
+static PointerRNA rna_Collection_objects_get(CollectionPropertyIterator *iter)
 {
-	if (!BKE_group_object_add(group, object)) {
-		BKE_reportf(reports, RPT_ERROR, "Object '%s' already in group '%s'", object->id.name + 2, group->id.name + 2);
+	ListBaseIterator *internal = &iter->internal.listbase;
+
+	/* we are actually iterating a ObjectBase list, so override get */
+	CollectionObject *cob = (CollectionObject *)internal->link;
+	return rna_pointer_inherit_refine(&iter->parent, &RNA_Object, cob->ob);
+}
+
+static void rna_Collection_objects_link(Collection *collection, Main *bmain, ReportList *reports, Object *object)
+{
+	if (!BKE_collection_object_add(bmain, collection, object)) {
+		BKE_reportf(reports, RPT_ERROR, "Object '%s' already in collection '%s'", object->id.name + 2, collection->id.name + 2);
 		return;
 	}
 
 	WM_main_add_notifier(NC_OBJECT | ND_DRAW, &object->id);
 }
 
-static void rna_Group_objects_unlink(Group *group, ReportList *reports, Object *object)
+static void rna_Collection_objects_unlink(Collection *collection, Main *bmain, ReportList *reports, Object *object)
 {
-	if (!BKE_group_object_unlink(group, object)) {
-		BKE_reportf(reports, RPT_ERROR, "Object '%s' not in group '%s'", object->id.name + 2, group->id.name + 2);
+	if (!BKE_collection_object_remove(bmain, collection, object, false)) {
+		BKE_reportf(reports, RPT_ERROR, "Object '%s' not in collection '%s'", object->id.name + 2, collection->id.name + 2);
 		return;
 	}
 
 	WM_main_add_notifier(NC_OBJECT | ND_DRAW, &object->id);
 }
 
+static void rna_Collection_children_begin(CollectionPropertyIterator *iter, PointerRNA *ptr)
+{
+	Collection *collection = (Collection *)ptr->data;
+	rna_iterator_listbase_begin(iter, &collection->children, NULL);
+}
+
+static PointerRNA rna_Collection_children_get(CollectionPropertyIterator *iter)
+{
+	ListBaseIterator *internal = &iter->internal.listbase;
+
+	/* we are actually iterating a CollectionBase list, so override get */
+	CollectionChild *child = (CollectionChild *)internal->link;
+	return rna_pointer_inherit_refine(&iter->parent, &RNA_Collection, child->collection);
+}
+
+static void rna_Collection_children_link(Collection *collection, Main *bmain, ReportList *reports, Collection *child)
+{
+	if (!BKE_collection_child_add(bmain, collection, child)) {
+		BKE_reportf(reports, RPT_ERROR, "Collection '%s' already in collection '%s'", child->id.name + 2, collection->id.name + 2);
+		return;
+	}
+
+	WM_main_add_notifier(NC_OBJECT | ND_DRAW, &child->id);
+}
+
+static void rna_Collection_children_unlink(Collection *collection, Main *bmain, ReportList *reports, Collection *child)
+{
+	if (!BKE_collection_child_remove(bmain, collection, child)) {
+		BKE_reportf(reports, RPT_ERROR, "Collection '%s' not in collection '%s'", child->id.name + 2, collection->id.name + 2);
+		return;
+	}
+
+	WM_main_add_notifier(NC_OBJECT | ND_DRAW, &child->id);
+}
+
+static void rna_Collection_flag_update(Main *bmain, Scene *scene, PointerRNA *ptr)
+{
+	Collection *collection = (Collection *)ptr->data;
+	BKE_collection_object_cache_free(collection);
+	BKE_main_collection_sync(bmain);
+	WM_main_add_notifier(NC_SCENE | ND_OB_SELECT, scene);
+}
+
 #else
 
-/* group.objects */
-static void rna_def_group_objects(BlenderRNA *brna, PropertyRNA *cprop)
+/* collection.objects */
+static void rna_def_collection_objects(BlenderRNA *brna, PropertyRNA *cprop)
 {
 	StructRNA *srna;
-/*	PropertyRNA *prop; */
-
 	FunctionRNA *func;
 	PropertyRNA *parm;
 	
-	RNA_def_property_srna(cprop, "GroupObjects");
-	srna = RNA_def_struct(brna, "GroupObjects", NULL);
-	RNA_def_struct_sdna(srna, "Group");
-	RNA_def_struct_ui_text(srna, "Group Objects", "Collection of group objects");
+	RNA_def_property_srna(cprop, "CollectionObjects");
+	srna = RNA_def_struct(brna, "CollectionObjects", NULL);
+	RNA_def_struct_sdna(srna, "Collection");
+	RNA_def_struct_ui_text(srna, "Collection Objects", "Collection of collection objects");
 
 	/* add object */
-	func = RNA_def_function(srna, "link", "rna_Group_objects_link");
-	RNA_def_function_flag(func, FUNC_USE_REPORTS);
-	RNA_def_function_ui_description(func, "Add this object to a group");
-	/* object to add */
+	func = RNA_def_function(srna, "link", "rna_Collection_objects_link");
+	RNA_def_function_flag(func, FUNC_USE_REPORTS | FUNC_USE_MAIN);
+	RNA_def_function_ui_description(func, "Add this object to a collection");
 	parm = RNA_def_pointer(func, "object", "Object", "", "Object to add");
 	RNA_def_parameter_flags(parm, PROP_NEVER_NULL, PARM_REQUIRED);
 
 	/* remove object */
-	func = RNA_def_function(srna, "unlink", "rna_Group_objects_unlink");
-	RNA_def_function_ui_description(func, "Remove this object to a group");
-	RNA_def_function_flag(func, FUNC_USE_REPORTS);
-	/* object to remove */
+	func = RNA_def_function(srna, "unlink", "rna_Collection_objects_unlink");
+	RNA_def_function_ui_description(func, "Remove this object from a collection");
+	RNA_def_function_flag(func, FUNC_USE_REPORTS | FUNC_USE_MAIN);
 	parm = RNA_def_pointer(func, "object", "Object", "", "Object to remove");
 	RNA_def_parameter_flags(parm, 0, PARM_REQUIRED);
 }
 
+/* collection.children */
+static void rna_def_collection_children(BlenderRNA *brna, PropertyRNA *cprop)
+{
+	StructRNA *srna;
+	FunctionRNA *func;
+	PropertyRNA *parm;
 
-void RNA_def_group(BlenderRNA *brna)
+	RNA_def_property_srna(cprop, "CollectionChildren");
+	srna = RNA_def_struct(brna, "CollectionChildren", NULL);
+	RNA_def_struct_sdna(srna, "Collection");
+	RNA_def_struct_ui_text(srna, "Collection Children", "Collection of child collections");
+
+	/* add child */
+	func = RNA_def_function(srna, "link", "rna_Collection_children_link");
+	RNA_def_function_flag(func, FUNC_USE_REPORTS | FUNC_USE_MAIN);
+	RNA_def_function_ui_description(func, "Add this collection as child of this collection");
+	parm = RNA_def_pointer(func, "child", "Collection", "", "Collection to add");
+	RNA_def_parameter_flags(parm, PROP_NEVER_NULL, PARM_REQUIRED);
+
+	/* remove child */
+	func = RNA_def_function(srna, "unlink", "rna_Collection_children_unlink");
+	RNA_def_function_ui_description(func, "Remove this child collection from a collection");
+	RNA_def_function_flag(func, FUNC_USE_REPORTS | FUNC_USE_MAIN);
+	parm = RNA_def_pointer(func, "child", "Collection", "", "Collection to remove");
+	RNA_def_parameter_flags(parm, 0, PARM_REQUIRED);
+}
+
+
+void RNA_def_collections(BlenderRNA *brna)
 {
 	StructRNA *srna;
 	PropertyRNA *prop;
 
-	srna = RNA_def_struct(brna, "Group", "ID");
-	RNA_def_struct_ui_text(srna, "Group", "Group of Object data-blocks");
+	srna = RNA_def_struct(brna, "Collection", "ID");
+	RNA_def_struct_sdna(srna, "Group"); /* it is actually Collection but for 2.8 the dna is patched! */
+	RNA_def_struct_ui_text(srna, "Collection", "Collection of Object data-blocks");
 	RNA_def_struct_ui_icon(srna, ICON_GROUP);
-	/* this is done on save/load in readfile.c, removed if no objects are in the group */
+	/* this is done on save/load in readfile.c, removed if no objects are in the collection and not in a scene */
 	RNA_def_struct_clear_flag(srna, STRUCT_ID_REFCOUNT);
 
 	prop = RNA_def_property(srna, "dupli_offset", PROP_FLOAT, PROP_TRANSLATION);
 	RNA_def_property_float_sdna(prop, NULL, "dupli_ofs");
-	RNA_def_property_ui_text(prop, "Dupli Offset", "Offset from the origin to use when instancing as DupliGroup");
+	RNA_def_property_ui_text(prop, "Dupli Offset", "Offset from the origin to use when instancing");
 	RNA_def_property_ui_range(prop, -10000.0, 10000.0, 10, RNA_TRANSLATION_PREC_DEFAULT);
 	RNA_def_property_update(prop, NC_OBJECT | ND_DRAW, NULL);
 
 	prop = RNA_def_property(srna, "objects", PROP_COLLECTION, PROP_NONE);
-	RNA_def_property_collection_sdna(prop, NULL, "view_layer->object_bases", NULL);
 	RNA_def_property_struct_type(prop, "Object");
-	RNA_def_property_ui_text(prop, "Objects", "A collection of this groups objects");
-	RNA_def_property_collection_funcs(prop, NULL, NULL, NULL, "rna_Group_objects_get", NULL, NULL, NULL, NULL);
-	rna_def_group_objects(brna, prop);
-
-	prop = RNA_def_property(srna, "view_layer", PROP_POINTER, PROP_NONE);
-	RNA_def_property_struct_type(prop, "ViewLayer");
-	RNA_def_property_flag(prop, PROP_NEVER_NULL);
-	RNA_def_property_clear_flag(prop, PROP_EDITABLE);
-	RNA_def_property_ui_text(prop, "View Layer", "Group internal view layer");
+	RNA_def_property_ui_text(prop, "Objects", "Objects that are directly in this collection");
+	RNA_def_property_collection_funcs(prop, "rna_Collection_objects_begin",
+	                                        "rna_iterator_listbase_next",
+	                                        "rna_iterator_listbase_end",
+	                                        "rna_Collection_objects_get",
+	                                        NULL, NULL, NULL, NULL);
+	rna_def_collection_objects(brna, prop);
+
+	prop = RNA_def_property(srna, "all_objects", PROP_COLLECTION, PROP_NONE);
+	RNA_def_property_struct_type(prop, "Object");
+	RNA_def_property_ui_text(prop, "All Objects", "Objects that are in this collection and its child collections");
+	RNA_def_property_collection_funcs(prop, "rna_Collection_all_objects_begin",
+	                                        "rna_iterator_listbase_next",
+	                                        "rna_iterator_listbase_end",
+	                                        "rna_Collection_all_objects_get",
+	                                        NULL, NULL, NULL, NULL);
+
+	prop = RNA_def_property(srna, "children", PROP_COLLECTION, PROP_NONE);
+	RNA_def_property_struct_type(prop, "Collection");
+	RNA_def_property_ui_text(prop, "Children", "Collections that are immediate children of this collection");
+	RNA_def_property_collection_funcs(prop, "rna_Collection_children_begin",
+	                                        "rna_iterator_listbase_next",
+	                                        "rna_iterator_listbase_end",
+	                                        "rna_Collection_children_get",
+	                                        NULL, NULL, NULL, NULL);
+	rna_def_collection_children(brna, prop);
+
+	/* Flags */
+	prop = RNA_def_property(srna, "hide_select", PROP_BOOLEAN, PROP_NONE);
+	RNA_def_property_boolean_sdna(prop, NULL, "flag", COLLECTION_RESTRICT_SELECT);
+	RNA_def_property_flag(prop, PROP_OVERRIDABLE_STATIC);
+	RNA_def_property_ui_icon(prop, ICON_RESTRICT_SELECT_OFF, 1);
+	RNA_def_property_ui_text(prop, "Restrict Select", "Disable collection object selection in the 3D viewport");
+	RNA_def_property_update(prop, NC_SCENE | ND_LAYER_CONTENT, "rna_Collection_flag_update");
+
+	prop = RNA_def_property(srna, "hide_viewport", PROP_BOOLEAN, PROP_NONE);
+	RNA_def_property_boolean_sdna(prop, NULL, "flag", COLLECTION_RESTRICT_VIEW);
+	RNA_def_property_flag(prop, PROP_OVERRIDABLE_STATIC);
+	RNA_def_property_ui_icon(prop, ICON_RESTRICT_VIEW_OFF, 1);
+	RNA_def_property_ui_text(prop, "Restrict Viewport", "Hide collection objects in the 3D viewport");
+	RNA_def_property_update(prop, NC_SCENE | ND_LAYER_CONTENT, "rna_Collection_flag_update");
+
+	prop = RNA_def_property(srna, "hide_render", PROP_BOOLEAN, PROP_NONE);
+	RNA_def_property_boolean_sdna(prop, NULL, "flag", COLLECTION_RESTRICT_RENDER);
+	RNA_def_property_flag(prop, PROP_OVERRIDABLE_STATIC);
+	RNA_def_property_ui_icon(prop, ICON_RESTRICT_RENDER_OFF, 1);
+	RNA_def_property_ui_text(prop, "Restrict Render", "Hide collection objects in renders");
+	RNA_def_property_update(prop, NC_SCENE | ND_LAYER_CONTENT, "rna_Collection_flag_update");
 }
 
 #endif