mpegvideo: fix invalid memory access for small video dimensions
authorJohn Brooks <john.brooks@bluecherry.net>
Tue, 13 Dec 2011 00:04:14 +0000 (17:04 -0700)
committerAnton Khirnov <anton@khirnov.net>
Sat, 21 Jan 2012 07:28:08 +0000 (08:28 +0100)
commit9b6aafba6c06ef62783dd5e9c5ed668f3a095128
tree3a02ccf7a244c2326bdfe49bf1b5a853b2281cfb
parent5d95112d926760046bc0b07aeda5cfc766ba54a6
mpegvideo: fix invalid memory access for small video dimensions

When either video dimension is only one macroblock, subtractions
based on v_edge_pos and the macroblock size may be negative. In
that situation, an unsigned comparison isn't sufficent to test for
MV overruns, because a limit of (unsigned)-1 will let any other
value pass.

Signed-off-by: Anton Khirnov <anton@khirnov.net>
libavcodec/mpegvideo.c
libavcodec/mpegvideo_common.h