avviddec: fix invalid alignment calculations.

avcodec_align_dimensions2 uses context->pix_fmt to make its
calculations, we thus need to make sure it is adequately set
when calling it.

Fixes:

gst-launch-1.0 videotestsrc ! video/x-raw, width=1920, height=1080 \
! avenc_mpeg4 ! avdec_mpeg4 ! xvimagesink

This showed invalid writes under valgrind, then segfault.

https://bugzilla.gnome.org/show_bug.cgi?id=792900

diff --git a/ext/libav/gstavviddec.c b/ext/libav/gstavviddec.c
index 4fc6dc2..e3ae04f 100644 (file)
--- a/ext/libav/gstavviddec.c
+++ b/ext/libav/gstavviddec.c
@@ -678,6 +678,12 @@ gst_ffmpegviddec_ensure_internal_pool (GstFFMpegVidDec * ffmpegdec,
   format = gst_ffmpeg_pixfmt_to_videoformat (picture->format);
   gst_video_info_set_format (&info, format, picture->width, picture->height);
 
+  /* If we have not yet been negotiated, a NONE format here would
+   * result in invalid initial dimension alignments, and potential
+   * out of bounds writes.
+   */
+  ffmpegdec->context->pix_fmt = picture->format;
+
   for (i = 0; i < G_N_ELEMENTS (ffmpegdec->stride); i++)
     ffmpegdec->stride[i] = -1;