mpeg2: fix size calculation from sequence_extension().
authorSreerenj Balachandran <sreerenj.balachandran@intel.com>
Sun, 12 Feb 2012 09:21:52 +0000 (11:21 +0200)
committerGwenole Beauchesne <gwenole.beauchesne@intel.com>
Fri, 24 Feb 2012 10:09:05 +0000 (11:09 +0100)
Original values from sequence_header() are 12-bit and the remaining
2 most significant bits are coming from sequence_extension().

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
gst-libs/gst/vaapi/gstvaapidecoder_mpeg2.c

index b2579b3..e98a3f4 100644 (file)
@@ -396,8 +396,8 @@ decode_sequence_ext(GstVaapiDecoderMpeg2 *decoder, guchar *buf, guint buf_size)
     priv->has_seq_ext = TRUE;
     priv->progressive_sequence = seq_ext->progressive;
 
-    width  = (priv->width  & 0xffff) | ((guint32)seq_ext->horiz_size_ext << 16);
-    height = (priv->height & 0xffff) | ((guint32)seq_ext->vert_size_ext  << 16);
+    width  = (priv->width  & 0x0fff) | ((guint32)seq_ext->horiz_size_ext << 12);
+    height = (priv->height & 0x0fff) | ((guint32)seq_ext->vert_size_ext  << 12);
     GST_DEBUG("video resolution %ux%u", width, height);
 
     if (seq_ext->fps_n_ext && seq_ext->fps_d_ext) {