swscale: fix overflow in gray16 vertical scaling.
authorRonald S. Bultje <rsbultje@gmail.com>
Sat, 17 Dec 2011 19:56:40 +0000 (19:56 +0000)
committerMans Rullgard <mans@mansr.com>
Sat, 17 Dec 2011 22:41:53 +0000 (22:41 +0000)
commit72dafea0fc0eb7230d7ebb0a7bc803e13b72aaad
treecee4a2212c3d21555a2c899c848191662dd18d0c
parent86a10281c7f6204b3ac1253718971b1f8c106699
swscale: fix overflow in gray16 vertical scaling.

This fixes the same overflow as in the RGB48/16-bit YUV scaling;
some filters can overflow both negatively and positively (e.g.
spline/lanczos), so we bias a signed integer so it's "half signed"
and "half unsigned", and can cover overflows in both directions
while maintaining full 31-bit depth.

Signed-off-by: Mans Rullgard <mans@mansr.com>
libswscale/swscale.c