From: Seungha Yang <seungha@centricular.com>
Date: Mon, 8 Jun 2020 14:46:43 +0000 (+0900)
Subject: mftransform: Fix deadlock when MFT requests processing output twice
X-Git-Tag: 1.19.3~507^2~1777
X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=52a82f5b84c74d3e1b94c44d7ee8391be9512551;p=platform%2Fupstream%2Fgstreamer.git

mftransform: Fix deadlock when MFT requests processing output twice

This sequence of event/data flow might happen

1) Initially we have one pending output event
  1-1) Then, process the pending output data
2) No pending input event, then we should wait new pending input event
  2-1) Wakeup by new pending event (but it's pending output event)

In above case, MFT will not report new pending input event
if pending output is not processed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1325>
---

diff --git a/sys/mediafoundation/gstmftransform.cpp b/sys/mediafoundation/gstmftransform.cpp
index 17c38b9..ee30c2e 100644
--- a/sys/mediafoundation/gstmftransform.cpp
+++ b/sys/mediafoundation/gstmftransform.cpp
@@ -539,6 +539,8 @@ gst_mf_transform_process_input (GstMFTransform * object,
   gst_mf_transform_drain_all_events (object);
 
   if (object->hardware) {
+  process_output:
+    /* Process pending output first */
     while (object->pending_have_output > 0) {
       GST_TRACE_OBJECT (object,
           "Pending have output %d", object->pending_have_output);
@@ -581,6 +583,14 @@ gst_mf_transform_process_input (GstMFTransform * object,
           GST_DEBUG_OBJECT (object, "Unhandled event %d", type);
           break;
       }
+
+      /* If MFT doesn't want to handle input yet but we have pending output,
+       * process output again */
+      if (object->pending_have_output > 0 && object->pending_need_input == 0) {
+        GST_TRACE_OBJECT (object,
+            "Only have pending output, process output again");
+        goto process_output;
+      }
     }
   }