Fix overflow issue in 16x16 quantization SSSE3
authorJingning Han <jingning@google.com>
Fri, 6 Sep 2013 23:06:21 +0000 (16:06 -0700)
committerJingning Han <jingning@google.com>
Sat, 7 Sep 2013 04:06:10 +0000 (21:06 -0700)
commit09bc942b47ac996047c7176e92e739313a23b3b8
tree4664d239f8790d4693fa20533b4b09726316d80e
parentd1268c592183db41d6586029a333eee155449d85
Fix overflow issue in 16x16 quantization SSSE3

The 16x16 transform unit test suggested that the peak coefficient
value can reach 32639. This could cause potential overflow issue
in the SSSE3 implmentation of 16x16 block quantization. This commit
fixes this issue by replacing addition with saturated addition.

Change-Id: I6d5bb7c5faad4a927be53292324bd2728690717e
vp9/encoder/vp9_encodemb.c
vp9/encoder/vp9_quantize.c
vp9/encoder/x86/vp9_quantize_ssse3.asm