From: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Date: Mon, 13 Jan 2014 16:18:42 +0000 (+0100)
Subject: vaapiencode: don't crash on NULL encoder on _finish().
X-Git-Tag: 0.5.8~76
X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=2c4fde0eae02383a9780e8e87ce1e3d442bf80ce;p=platform%2Fupstream%2Fgstreamer-vaapi.git

vaapiencode: don't crash on NULL encoder on _finish().

Don't try to destroy an encoder, in GstVideoEncoder::finish() handler,
if it was not created in the first place. Return "not-negotiated" error
since this means we did not even reach GstVideoEncoder::set_format(),
where the encoder could have been created.

This fixes a crash when the vaapiencode_* plug-in elements get deallocated
and that we failed to negotiate either pad.

https://bugzilla.gnome.org/show_bug.cgi?id=719704
---

diff --git a/gst/vaapi/gstvaapiencode.c b/gst/vaapi/gstvaapiencode.c
index b482aef..e4b52f4 100644
--- a/gst/vaapi/gstvaapiencode.c
+++ b/gst/vaapi/gstvaapiencode.c
@@ -587,6 +587,13 @@ gst_vaapiencode_finish (GstVideoEncoder * venc)
   GstVaapiEncoderStatus status;
   GstFlowReturn ret = GST_FLOW_OK;
 
+  /* Don't try to destroy encoder if none was created in the first place.
+     Return "not-negotiated" error since this means we did not even reach
+     GstVideoEncoder::set_format() state, where the encoder could have
+     been created */
+  if (!encode->encoder)
+    return GST_FLOW_NOT_NEGOTIATED;
+
   status = gst_vaapi_encoder_flush (encode->encoder);
 
   GST_VIDEO_ENCODER_STREAM_UNLOCK (encode);