aacenc_is: add a flag to use pure coefficients instead

This commit adds a flag to use the pure coefficients instead
of the processed ones (sce->coeffs). This is needed because
IS will apply the changes to the coefficients immediately
before the adjust_common_prediction function and it doesn't
make sense to measure stereo channel coefficient difference
when one of the channels coefficients are all zero.

Therefore add a flag to use pure coefficients in that case.
TNS is the only thing touching the coefficients before IS
so common window prediction will not take that into account
but the effect of the TNS filter per coefficient can be small
(a few percent) so to some approximation it's fine to just
ignore that.

Also fixed a small error which doesn't alter the results
that much. pow(sqrt(number), 3.0/4.0) == pow(number, 3.0/8.0) !=
pow(number, 3.0/4.0).

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>

1 files changed, 4 insertions, 4 deletions

diff --git a/libavcodec/aacenc_pred.c b/libavcodec/aacenc_pred.c
index fafe0029d4..eb8e7f7d27 100644
--- a/libavcodec/aacenc_pred.c
+++ b/libavcodec/aacenc_pred.c
@@ -179,15 +179,15 @@ void ff_aac_adjust_common_prediction(AACEncContext *s, ChannelElement *cpe)
                 for (i = 0; i < sce0->ics.swb_sizes[g]; i++) {
                     float coef0 = sce0->pcoeffs[start+(w+w2)*128+i];
                     float coef1 = sce1->pcoeffs[start+(w+w2)*128+i];
-                    ener0 += coef0*coef0;
-                    ener1 += coef1*coef1;
+                    ener0  += coef0*coef0;
+                    ener1  += coef1*coef1;
                     ener01 += (coef0 + coef1)*(coef0 + coef1);
                 }
             }
             ph_err1 = ff_aac_is_encoding_err(s, cpe, start, w, g,
-                                             ener0, ener1, ener01, -1);
+                                             ener0, ener1, ener01, 1, -1);
             ph_err2 = ff_aac_is_encoding_err(s, cpe, start, w, g,
-                                             ener0, ener1, ener01, +1);
+                                             ener0, ener1, ener01, 1, +1);
             erf = ph_err1.error < ph_err2.error ? &ph_err1 : &ph_err2;
             if (erf->pass) {
                 sce0->ics.prediction_used[sfb] = 1;