Armature: add Inherit Scale options to remove shear or average the scale.

As an inherent property of matrix-based transformation math, non-
uniform scaling of a parent bone induces shear into the transform
matrix of any rotated child. Such matrices cannot be cleanly
decomposed into a combination of location/rotation/scale, which
causes issues for rigging and animation tools.

Blender bones have options to exclude rotation and/or scale from the
inherited transformation, but don't have any support for removing the
often undesired shear component. That goal requires replacing simple
parenting with a combination of multiple bones and constraints. The
same is true about the goal of inheriting some scale, but completely
avoiding shear.

This patch replaces the old Inherit Scale checkbox with a enum that
supports multiple options:

* Full: inherit all effects of scale, like with enabled Inherit Scale.

* Fix Shear: removes shear from the final inherited transformation.

  The cleanup math is specifically designed to preserve the main
  axis of the bone, its length and total volume, and minimally
  affect roll on average. It however will not prevent reappearance
  of shear due to local rotation of the child or its children.

* Average: inherit uniform scale that represents the parent volume.

  This is the simplest foolproof solution that will inherit some
  scale without ever causing shear.

* None: completely remove scale and shear.

* None (Legacy): old disabled Inherit Scale checkbox.

  This mode does not handle parent shear in any way, so the child
  is likely to end up having both scale and shear. It is retained
  for backward compatibility.

Since many rigging-related addons access the use_inherit_scale
property from Python, it is retained as a backward compatibility
stub that provides the old functionality.

As a side effect of reworking the code, this also fixes a matrix
multiplication order bug in the Inherit Rotation code, which caused
the parent local scale to be applied in world space. In rigger
opinion this option is useless in production rigs, so this fix
should not be a problem.

Reviewers: brecht

Differential Revision: https://developer.blender.org/D5588

1 files changed, 74 insertions, 2 deletions

diff --git a/source/blender/makesrna/intern/rna_armature.c b/source/blender/makesrna/intern/rna_armature.c
index 7949db9055f..cfcc37adf6c 100644
--- a/source/blender/makesrna/intern/rna_armature.c
+++ b/source/blender/makesrna/intern/rna_armature.c
@@ -303,6 +303,40 @@ static void rna_Bone_layer_set(PointerRNA *ptr, const bool *values)
   BKE_armature_refresh_layer_used(arm);
 }
 
+/* TODO: remove the deprecation stubs. */
+static bool rna_use_inherit_scale_get(char inherit_scale_mode)
+{
+  return inherit_scale_mode <= BONE_INHERIT_SCALE_FIX_SHEAR;
+}
+
+static void rna_use_inherit_scale_set(char *inherit_scale_mode, bool value)
+{
+  bool cur_value = (*inherit_scale_mode <= BONE_INHERIT_SCALE_FIX_SHEAR);
+  if (value != cur_value) {
+    *inherit_scale_mode = (value ? BONE_INHERIT_SCALE_FULL : BONE_INHERIT_SCALE_NONE);
+  }
+}
+
+static bool rna_EditBone_use_inherit_scale_get(PointerRNA *ptr)
+{
+  return rna_use_inherit_scale_get(((EditBone *)ptr->data)->inherit_scale_mode);
+}
+
+static void rna_EditBone_use_inherit_scale_set(PointerRNA *ptr, bool value)
+{
+  rna_use_inherit_scale_set(&((EditBone *)ptr->data)->inherit_scale_mode, value);
+}
+
+static bool rna_Bone_use_inherit_scale_get(PointerRNA *ptr)
+{
+  return rna_use_inherit_scale_get(((Bone *)ptr->data)->inherit_scale_mode);
+}
+
+static void rna_Bone_use_inherit_scale_set(PointerRNA *ptr, bool value)
+{
+  rna_use_inherit_scale_set(&((Bone *)ptr->data)->inherit_scale_mode, value);
+}
+
 static void rna_Armature_layer_set(PointerRNA *ptr, const bool *values)
 {
   bArmature *arm = (bArmature *)ptr->data;
@@ -773,6 +807,28 @@ static void rna_def_bone_common(StructRNA *srna, int editbone)
       {0, NULL, 0, NULL, NULL},
   };
 
+  static const EnumPropertyItem prop_inherit_scale_mode[] = {
+      {BONE_INHERIT_SCALE_FULL, "FULL", 0, "Full", "Inherit all effects of parent scaling"},
+      {BONE_INHERIT_SCALE_FIX_SHEAR,
+       "FIX_SHEAR",
+       0,
+       "Fix Shear",
+       "Inherit scaling, but remove shearing of the child in the rest orientation"},
+      {BONE_INHERIT_SCALE_AVERAGE,
+       "AVERAGE",
+       0,
+       "Average",
+       "Inherit uniform scaling representing the overall change in the volume of the parent"},
+      {BONE_INHERIT_SCALE_NONE, "NONE", 0, "None", "Completely ignore parent scaling"},
+      {BONE_INHERIT_SCALE_NONE_LEGACY,
+       "NONE_LEGACY",
+       0,
+       "None (Legacy)",
+       "Ignore parent scaling without compensating for parent shear. "
+       "Replicates the effect of disabling the original Inherit Scale checkbox"},
+      {0, NULL, 0, NULL, NULL},
+  };
+
   PropertyRNA *prop;
 
   /* strings */
@@ -832,9 +888,25 @@ static void rna_def_bone_common(StructRNA *srna, int editbone)
   RNA_def_property_ui_text(prop, "Deform", "Enable Bone to deform geometry");
   RNA_def_property_update(prop, 0, "rna_Armature_update_data");
 
+  prop = RNA_def_property(srna, "inherit_scale", PROP_ENUM, PROP_NONE);
+  RNA_def_property_ui_text(
+      prop, "Inherit Scale", "Specifies how the bone inherits scaling from the parent bone");
+  RNA_def_property_enum_sdna(prop, NULL, "inherit_scale_mode");
+  RNA_def_property_enum_items(prop, prop_inherit_scale_mode);
+  RNA_def_property_update(prop, 0, "rna_Armature_update_data");
+
+  /* TODO: remove the compatibility stub. */
   prop = RNA_def_property(srna, "use_inherit_scale", PROP_BOOLEAN, PROP_NONE);
-  RNA_def_property_ui_text(prop, "Inherit Scale", "Bone inherits scaling from parent bone");
-  RNA_def_property_boolean_negative_sdna(prop, NULL, "flag", BONE_NO_SCALE);
+  RNA_def_property_ui_text(
+      prop, "Inherit Scale", "DEPRECATED: Bone inherits scaling from parent bone");
+  if (editbone) {
+    RNA_def_property_boolean_funcs(
+        prop, "rna_EditBone_use_inherit_scale_get", "rna_EditBone_use_inherit_scale_set");
+  }
+  else {
+    RNA_def_property_boolean_funcs(
+        prop, "rna_Bone_use_inherit_scale_get", "rna_Bone_use_inherit_scale_set");
+  }
   RNA_def_property_update(prop, 0, "rna_Armature_update_data");
 
   prop = RNA_def_property(srna, "use_local_location", PROP_BOOLEAN, PROP_NONE);