avcodecmap: Don't try converting channel layouts with more than 64 channels

We only support up to 64 channels in GStreamer with a specific layout so
it's safe to assume a NONE layout in this case.

Also the arrays of channel positions are allocated everywhere with 64
elements so don't try setting more than 64 to NONE as that will cause
stack corruptions and similar memory safety issues.

Thanks to Natalie Silvanovich for reporting this issue.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-libav/-/issues/92

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-libav/-/merge_requests/120>

diff --git a/ext/libav/gstavcodecmap.c b/ext/libav/gstavcodecmap.c
index b5be4bb..be22f22 100644 (file)
--- a/ext/libav/gstavcodecmap.c
+++ b/ext/libav/gstavcodecmap.c
@@ -102,7 +102,7 @@ gst_ffmpeg_channel_layout_to_gst (guint64 channel_layout, gint channels,
   guint nchannels = 0;
   gboolean none_layout = FALSE;
 
-  if (channel_layout == 0) {
+  if (channel_layout == 0 || channels > 64) {
     nchannels = channels;
     none_layout = TRUE;
   } else {
@@ -163,7 +163,7 @@ gst_ffmpeg_channel_layout_to_gst (guint64 channel_layout, gint channels,
     } else {
       guint i;
 
-      for (i = 0; i < nchannels; i++)
+      for (i = 0; i < nchannels && i < 64; i++)
         pos[i] = GST_AUDIO_CHANNEL_POSITION_NONE;
     }
   }