Safe guard zero median filter outcome case
authorJingning Han <jingning@google.com>
Wed, 13 Mar 2019 19:06:39 +0000 (12:06 -0700)
committerJingning Han <jingning@google.com>
Wed, 13 Mar 2019 19:11:04 +0000 (12:11 -0700)
In case median filter returns 0 value, bypass the Wiener filter
stage. This avoids potential divided by 0 case.

In the same place use a temp variable to take the Wiener filter
output instead of returning to the coeff array.

Change-Id: I45f57c515b4062a0aa1f312eda852462cb655d8e

vp9/encoder/vp9_encoder.c

index f3f05d8..fb462c0 100644 (file)
@@ -4793,9 +4793,12 @@ static void set_mb_wiener_variance(VP9_COMP *cpi) {
       // Wiener filter
       for (idx = 1; idx < coeff_count; ++idx) {
         int64_t sqr_coeff = (int64_t)coeff[idx] * coeff[idx];
-        coeff[idx] = (int16_t)((sqr_coeff * coeff[idx]) /
-                               (sqr_coeff + (int64_t)median_val * median_val));
-        wiener_variance += (int64_t)coeff[idx] * coeff[idx];
+        int64_t tmp_coeff = (int64_t)coeff[idx];
+        if (median_val) {
+          tmp_coeff = (sqr_coeff * coeff[idx]) /
+                      (sqr_coeff + (int64_t)median_val * median_val);
+        }
+        wiener_variance += tmp_coeff * tmp_coeff;
       }
       cpi->mb_wiener_variance[mb_row * cm->mb_cols + mb_col] =
           wiener_variance / coeff_count;