swscale: fix signed overflow in yuv2mono_X_c_template
authorMans Rullgard <mans@mansr.com>
Sat, 26 Nov 2011 21:53:00 +0000 (21:53 +0000)
committerMans Rullgard <mans@mansr.com>
Sat, 26 Nov 2011 22:53:47 +0000 (22:53 +0000)
As old bits are shifted out of the accumulator, they cause signed
overflows when they reach the end.  Making the variable unsigned fixes
this.

Signed-off-by: Mans Rullgard <mans@mansr.com>
libswscale/swscale.c

index 98daf22..975a0bd 100644 (file)
@@ -510,7 +510,7 @@ yuv2mono_X_c_template(SwsContext *c, const int16_t *lumFilter,
     const uint8_t * const d128=dither_8x8_220[y&7];
     uint8_t *g = c->table_gU[128] + c->table_gV[128];
     int i;
-    int acc = 0;
+    unsigned acc = 0;
 
     for (i = 0; i < dstW - 1; i += 2) {
         int j;