codecparsers: vp9statefulparser: Fix the gst_vp9_get_qindex clamp issue.
authorHe Junyan <junyan.he@intel.com>
Wed, 30 Jun 2021 07:23:15 +0000 (15:23 +0800)
committerGStreamer Marge Bot <gitlab-merge-bot@gstreamer-foundation.org>
Fri, 2 Jul 2021 08:44:15 +0000 (08:44 +0000)
The alternate quantizer is a delta value and should be int type.
We mark it wrongly as uint, that will make CLAMP (data, 0, 255)
always choose 255 rather than 0 if the data < 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2369>

gst-libs/gst/codecs/gstvp9statefulparser.c

index 2c25419..134e104 100644 (file)
@@ -1137,7 +1137,7 @@ gst_vp9_get_qindex (const GstVp9SegmentationParams * segmentation_params,
 
   if (gst_vp9_seg_feature_active (segmentation_params, segment_id,
           GST_VP9_SEG_LVL_ALT_Q)) {
-    guint data =
+    gint data =
         segmentation_params->feature_data[segment_id][GST_VP9_SEG_LVL_ALT_Q];
 
     if (!segmentation_params->segmentation_abs_or_delta_update)