va: h264dec: Try to use ConstrainedBaseline or Main to decode BaseLine.

In the h264, the Baseline profile is widely misused. A lot of streams declare
that they are the Baseline, but in fact they just conform to ConstrainedBaseline.
The features such as FMO and ASO are not used at all.
If the decoder does not strictly conforms to the SPEC, we can just use Baseline
or Main profile to decode it to avoid lots of streams failure.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2428>

diff --git a/sys/va/gstvah264dec.c b/sys/va/gstvah264dec.c
index 4d342a880218984ffe97cd57a6f691a7ac824c89..aba8bab70d58a46b55b2010a529672156b775acf 100644 (file)
--- a/sys/va/gstvah264dec.c
+++ b/sys/va/gstvah264dec.c
@@ -602,13 +602,21 @@ _get_profile (GstVaH264Dec * self, const GstH264SPS * sps, gint max_dpb_size)
 
   switch (sps->profile_idc) {
     case GST_H264_PROFILE_BASELINE:
-      /* A.2 compliant */
+    {
+      GstH264DecoderCompliance compliance = GST_H264_DECODER_COMPLIANCE_STRICT;
+
+      g_object_get (G_OBJECT (self), "compliance", &compliance, NULL);
+
+      /* A.2 compliant or not strict */
       if (sps->constraint_set0_flag || sps->constraint_set1_flag
-          || sps->constraint_set2_flag) {
+          || sps->constraint_set2_flag
+          || compliance != GST_H264_DECODER_COMPLIANCE_STRICT) {
         profiles[i++] = VAProfileH264ConstrainedBaseline;
         profiles[i++] = VAProfileH264Main;
       }
+
       break;
+    }
     case GST_H264_PROFILE_EXTENDED:
       if (sps->constraint_set1_flag) {  /* A.2.2 (main profile) */
         profiles[i++] = VAProfileH264Main;