Fix #36226, Select Linked works not in touch with Prefs.

When setting keymap properties to values equalling the RNA default, they will
get "unset" and automatic operator behavior is used. There is no way to
explicitly set the default value as a user.

1) To allow distinguishing uninitialized (not set) properties in the keymap
items, a few changes to the RNA struct comparison function are needed: Instead
of allowing only strict/non-strict comparison of 2 properties A and B in a
struct, this now has 3 modes:
* STRICT: compare only the actual property values (same as 'strict' before)
* UNSET_MATCH_ANY: if either A or B is unset, consider them a match (same as
non-strict before)
* UNSET_MATCH_NONE: if one property is set and the other not, consider them a
mismatch.

The new UNSET_MATCH_NONE mode is useful for keymaps, because it allows keeping
user-defined property values in the keymap even if they match the default
property value (see wm_keymap_diff function in wm_keymap.c)

2) A new operator is added for unsetting ID properties in the RMB context menu
and in user preferences next to keymap properties. This only works on ID
properties and deletes the ID property storage, so that the default value is used.

In the user preferences for keymaps the properties are shown in an inactive
layout to indicate that the default value is used (which some operators such as
the "select linked" op from the report use to trigger automatic behavior). When
the user sets a property it gets set and stays that way until explicitly "unset"
using the new operator.

1 files changed, 13 insertions, 7 deletions

diff --git a/source/blender/makesrna/intern/rna_access.c b/source/blender/makesrna/intern/rna_access.c
index 918c67b4513..b164a3c3ed9 100644
--- a/source/blender/makesrna/intern/rna_access.c
+++ b/source/blender/makesrna/intern/rna_access.c
@@ -6316,14 +6316,20 @@ void _RNA_warning(const char *format, ...)
 #endif
 }
 
-bool RNA_property_equals(PointerRNA *a, PointerRNA *b, PropertyRNA *prop, bool is_strict)
+bool RNA_property_equals(PointerRNA *a, PointerRNA *b, PropertyRNA *prop, eRNAEqualsMode mode)
 {
 	int len, fromlen;
 
-	/* if not strict, uninitialized properties are assumed to match */
-	if (!is_strict)
-		if (!(RNA_property_is_set(a, prop) && RNA_property_is_set(b, prop)))
+	if (mode == RNA_EQ_UNSET_MATCH_ANY) {
+		/* uninitialized properties are assumed to match anything */
+		if (!RNA_property_is_set(a, prop) || !RNA_property_is_set(b, prop))
 			return true;
+	}
+	else if (mode == RNA_EQ_UNSET_MATCH_NONE) {
+		/* unset properties never match set properties */
+		if (RNA_property_is_set(a, prop) != RNA_property_is_set(b, prop))
+			return false;
+	}
 
 	/* get the length of the array to work with */
 	len = RNA_property_array_length(a, prop);
@@ -6437,7 +6443,7 @@ bool RNA_property_equals(PointerRNA *a, PointerRNA *b, PropertyRNA *prop, bool i
 			if (!STREQ(RNA_property_identifier(prop), "rna_type")) {
 				PointerRNA propptr_a = RNA_property_pointer_get(a, prop);
 				PointerRNA propptr_b = RNA_property_pointer_get(b, prop);
-				return RNA_struct_equals(&propptr_a, &propptr_b, is_strict);
+				return RNA_struct_equals(&propptr_a, &propptr_b, mode);
 			}
 			break;
 		}
@@ -6449,7 +6455,7 @@ bool RNA_property_equals(PointerRNA *a, PointerRNA *b, PropertyRNA *prop, bool i
 	return true;
 }
 
-bool RNA_struct_equals(PointerRNA *a, PointerRNA *b, bool is_strict)
+bool RNA_struct_equals(PointerRNA *a, PointerRNA *b, eRNAEqualsMode mode)
 {
 	CollectionPropertyIterator iter;
 //	CollectionPropertyRNA *citerprop;  /* UNUSED */
@@ -6470,7 +6476,7 @@ bool RNA_struct_equals(PointerRNA *a, PointerRNA *b, bool is_strict)
 	for (; iter.valid; RNA_property_collection_next(&iter)) {
 		PropertyRNA *prop = iter.ptr.data;
 
-		if (!RNA_property_equals(a, b, prop, is_strict)) {
+		if (!RNA_property_equals(a, b, prop, mode)) {
 			equals = false;
 			break;
 		}