codecparsers: vc1: fix bitplanes decoding (DIFF6 or NORM6 residual bytes).

Fix parsing of residual bytes. This is a two-step process. First,
remaining colums of full vertical resolution (<height>) need to be
processed. Next, remaining bytes in the first row can be processed,
while taking into account the fact that we may have filled in the
first columns already. So, this is not full horizontal resolution.

The following figure helps in understanding the expected order of
operations, for a 8x5 MBs bitplane.

    5 5 6 6 6 6 6 6
    5 5 1 1 1 2 2 2
    5 5 1 1 1 2 2 2
    5 5 3 3 3 4 4 4
    5 5 3 3 3 4 4 4

So, after tiles 1 to 4 are decoded, vertical tile 5 needs to be
processed (2x5 MBs) and then the horizontal tile 6 (6x1 MBs).

https://bugzilla.gnome.org/show_bug.cgi?id=692461

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
(cherry picked from commit 10639eb88972bf89656af694b8bf8c4b41255c69)

diff --git a/gst-libs/gst/codecparsers/gstvc1parser.c b/gst-libs/gst/codecparsers/gstvc1parser.c
index a4113efe87d8c1e87b78e9f108a16fd87a18a7ae..ea55a8e5503eb2c2d84ea088381916de9869335f 100644 (file)
--- a/gst-libs/gst/codecparsers/gstvc1parser.c
+++ b/gst-libs/gst/codecparsers/gstvc1parser.c
@@ -556,7 +556,7 @@ bitplane_decoding (GstBitReader * br, guint8 * data,
 
       if (x) {
         if (data)
-          pdata = data + y * stride;
+          pdata = data;
         if (!decode_colskip (br, pdata, x, height, stride, invert_mask))
           goto failed;
       }
@@ -564,7 +564,7 @@ bitplane_decoding (GstBitReader * br, guint8 * data,
       if (y) {
         if (data)
           pdata = data + x;
-        if (!decode_rowskip (br, pdata, width, y, stride, invert_mask))
+        if (!decode_rowskip (br, pdata, width - x, y, stride, invert_mask))
           goto failed;
       }
       break;