Armature: add Inherit Scale options to remove shear or average the scale.

As an inherent property of matrix-based transformation math, non-
uniform scaling of a parent bone induces shear into the transform
matrix of any rotated child. Such matrices cannot be cleanly
decomposed into a combination of location/rotation/scale, which
causes issues for rigging and animation tools.

Blender bones have options to exclude rotation and/or scale from the
inherited transformation, but don't have any support for removing the
often undesired shear component. That goal requires replacing simple
parenting with a combination of multiple bones and constraints. The
same is true about the goal of inheriting some scale, but completely
avoiding shear.

This patch replaces the old Inherit Scale checkbox with a enum that
supports multiple options:

* Full: inherit all effects of scale, like with enabled Inherit Scale.

* Fix Shear: removes shear from the final inherited transformation.

  The cleanup math is specifically designed to preserve the main
  axis of the bone, its length and total volume, and minimally
  affect roll on average. It however will not prevent reappearance
  of shear due to local rotation of the child or its children.

* Average: inherit uniform scale that represents the parent volume.

  This is the simplest foolproof solution that will inherit some
  scale without ever causing shear.

* None: completely remove scale and shear.

* None (Legacy): old disabled Inherit Scale checkbox.

  This mode does not handle parent shear in any way, so the child
  is likely to end up having both scale and shear. It is retained
  for backward compatibility.

Since many rigging-related addons access the use_inherit_scale
property from Python, it is retained as a backward compatibility
stub that provides the old functionality.

As a side effect of reworking the code, this also fixes a matrix
multiplication order bug in the Inherit Rotation code, which caused
the parent local scale to be applied in world space. In rigger
opinion this option is useless in production rigs, so this fix
should not be a problem.

Reviewers: brecht

Differential Revision: https://developer.blender.org/D5588

1 files changed, 24 insertions, 8 deletions

diff --git a/source/blender/editors/armature/pose_transform.c b/source/blender/editors/armature/pose_transform.c
index 854fb237929..52a3f7711c4 100644
--- a/source/blender/editors/armature/pose_transform.c
+++ b/source/blender/editors/armature/pose_transform.c
@@ -241,13 +241,21 @@ static void applyarmature_process_selected_rec(bArmature *arm,
 
       /* Parent effects on the bone transform that have to be removed. */
       BKE_bone_offset_matrix_get(bone, offs_bone);
-      BKE_bone_parent_transform_calc_from_matrices(
-          bone->flag, offs_bone, bone->parent->arm_mat, pchan_eval->parent->pose_mat, &old_bpt);
+      BKE_bone_parent_transform_calc_from_matrices(bone->flag,
+                                                   bone->inherit_scale_mode,
+                                                   offs_bone,
+                                                   bone->parent->arm_mat,
+                                                   pchan_eval->parent->pose_mat,
+                                                   &old_bpt);
 
       /* Applied parent effects that have to be kept, if any. */
       float(*new_parent_pose)[4] = pstate ? pstate->new_rest_mat : bone->parent->arm_mat;
-      BKE_bone_parent_transform_calc_from_matrices(
-          bone->flag, offs_bone, bone->parent->arm_mat, new_parent_pose, &new_bpt);
+      BKE_bone_parent_transform_calc_from_matrices(bone->flag,
+                                                   bone->inherit_scale_mode,
+                                                   offs_bone,
+                                                   bone->parent->arm_mat,
+                                                   new_parent_pose,
+                                                   &new_bpt);
 
       BKE_bone_parent_transform_invert(&old_bpt);
       BKE_bone_parent_transform_combine(&new_bpt, &old_bpt, &invparent);
@@ -283,8 +291,12 @@ static void applyarmature_process_selected_rec(bArmature *arm,
 
     /* Include applied parent effects. */
     BKE_bone_offset_matrix_get(bone, offs_bone);
-    BKE_bone_parent_transform_calc_from_matrices(
-        bone->flag, offs_bone, pstate->bone->arm_mat, pstate->new_rest_mat, &bpt);
+    BKE_bone_parent_transform_calc_from_matrices(bone->flag,
+                                                 bone->inherit_scale_mode,
+                                                 offs_bone,
+                                                 pstate->bone->arm_mat,
+                                                 pstate->new_rest_mat,
+                                                 &bpt);
 
     unit_m4(new_pstate.new_rest_mat);
     BKE_bone_parent_transform_apply(&bpt, new_pstate.new_rest_mat, new_pstate.new_rest_mat);
@@ -306,8 +318,12 @@ static void applyarmature_process_selected_rec(bArmature *arm,
       /* Compute the channel coordinate space matrices for the new rest state. */
       invert_m4_m4(inv_parent_arm, pstate->new_arm_mat);
       mul_m4_m4m4(offs_bone, inv_parent_arm, new_pstate.new_arm_mat);
-      BKE_bone_parent_transform_calc_from_matrices(
-          bone->flag, offs_bone, pstate->new_arm_mat, pstate->new_arm_mat, &bpt);
+      BKE_bone_parent_transform_calc_from_matrices(bone->flag,
+                                                   bone->inherit_scale_mode,
+                                                   offs_bone,
+                                                   pstate->new_arm_mat,
+                                                   pstate->new_arm_mat,
+                                                   &bpt);
 
       /* Re-apply the location to keep the final effect. */
       invert_m4(bpt.loc_mat);