v4l2codecs: Fix tiled formats stride conversion
authorNicolas Dufresne <nicolas.dufresne@collabora.com>
Tue, 10 Oct 2023 19:43:07 +0000 (15:43 -0400)
committerTim-Philipp Müller <tim@centricular.com>
Wed, 11 Oct 2023 15:48:50 +0000 (16:48 +0100)
While adding arbitrary tile support, a round up operation was badly
converter. This caused the Y component of the stride to be 0. This
eventually lead to a crash in glupoad preceded by the following
assertion.

  gst_gl_buffer_allocation_params_new: assertion 'alloc_size > 0' failed

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5463>

subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2format.c

index 64c47e1..1110742 100644 (file)
@@ -86,10 +86,9 @@ set_stride (GstVideoInfo * info, gint plane, gint stride)
 
     padded_height = GST_VIDEO_FORMAT_INFO_SCALE_HEIGHT (finfo, plane,
         info->height);
-    padded_height = (padded_height + tile_height - 1) / tile_height;
 
     x_tiles = stride / GST_VIDEO_FORMAT_INFO_TILE_STRIDE (finfo, plane);
-    y_tiles = padded_height / tile_height;
+    y_tiles = (padded_height + tile_height - 1) / tile_height;
     info->stride[plane] = GST_VIDEO_TILE_MAKE_STRIDE (x_tiles, y_tiles);
   } else {
     info->stride[plane] = stride;