Added an assert() check for normalized quats which exposed a number of bugs where normalized quat was incorrectly assumed.

This would have made bug #25003 very simple to find.

- Objects had their quats normalized when calculating their matrix, this is inconstant with pose bones and isn't useful for animation.
  Also it wasn't normalizing the delta rotation so these would give bad rotations.

- Converting between rotation modes BKE_rotMode_change_values() assumed normal length quat. changing quat to euler rotation for eg could change the bone.

- Clear rotation and transform were not normalizing the quat when 4d loc was disabled on quat rotation, corrected and also made it so the quat scale is restored after conversion so animations curves dont jump.

There is 1 case in mat3_to_quat_is_ok() where quat_to_mat3 on an unnormalized quat is needed, for this I had to add an ugly static function quat_to_mat3_no_assert(), but overall its worthwhile IMHO to be able to find incorrect use of rotation conversion.

1 files changed, 15 insertions, 10 deletions

diff --git a/source/blender/editors/transform/transform.c b/source/blender/editors/transform/transform.c
index 36438712d09..4b6079001ff 100644
--- a/source/blender/editors/transform/transform.c
+++ b/source/blender/editors/transform/transform.c
@@ -1917,23 +1917,29 @@ static void protectedQuaternionBits(short protectflag, float *quat, float *oldqu
 	}
 	else {
 		/* quaternions get limited with euler... (compatability mode) */
-		float eul[3], oldeul[3], quat1[4];
-		
-		QUATCOPY(quat1, quat);
-		quat_to_eul( eul,quat);
-		quat_to_eul( oldeul,oldquat);
-		
+		float eul[3], oldeul[3], nquat[4], noldquat[4];
+		float qlen;
+
+		qlen= normalize_qt_qt(nquat, quat);
+		normalize_qt_qt(noldquat, oldquat);
+
+		quat_to_eul(eul, nquat);
+		quat_to_eul(oldeul, noldquat);
+
 		if (protectflag & OB_LOCK_ROTX)
 			eul[0]= oldeul[0];
 		if (protectflag & OB_LOCK_ROTY)
 			eul[1]= oldeul[1];
 		if (protectflag & OB_LOCK_ROTZ)
 			eul[2]= oldeul[2];
-		
+
 		eul_to_quat( quat,eul);
+
+		/* restore original quat size */
+		mul_qt_fl(quat, qlen);
 		
 		/* quaternions flip w sign to accumulate rotations correctly */
-		if ( (quat1[0]<0.0f && quat[0]>0.0f) || (quat1[0]>0.0f && quat[0]<0.0f) ) {
+		if ( (nquat[0]<0.0f && quat[0]>0.0f) || (nquat[0]>0.0f && quat[0]<0.0f) ) {
 			mul_qt_fl(quat, -1.0f);
 		}
 	}
@@ -2013,8 +2019,7 @@ static void constraintob_from_transdata(bConstraintOb *cob, TransData *td)
 			   we don't necessarily end up with a rotation matrix, and
 			   then conversion back to quat gives a different result */
 			float quat[4];
-			copy_qt_qt(quat, td->ext->quat);
-			normalize_qt(quat);
+			normalize_qt_qt(quat, td->ext->quat);
 			quat_to_mat4(cob->matrix, quat);
 		}
 		else if (td->ext->rotOrder == ROT_MODE_AXISANGLE) {