avcodecmap: Don't try converting channel layouts with more than 64 channels

We only support up to 64 channels in GStreamer with a specific layout so
it's safe to assume a NONE layout in this case.

Also the arrays of channel positions are allocated everywhere with 64
elements so don't try setting more than 64 to NONE as that will cause
stack corruptions and similar memory safety issues.

Thanks to Natalie Silvanovich for reporting this issue.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-libav/-/issues/92

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-libav/-/merge_requests/120>

diff --git a/ext/libav/gstavcodecmap.c b/ext/libav/gstavcodecmap.c
index b5be4bb7a5f2712f78383da9319754a8849e3307..be22f22cf5c7c7b22b13e44b10999adaacbcca2b 100644 (file)
--- a/ext/libav/gstavcodecmap.c
+++ b/ext/libav/gstavcodecmap.c
@@ -102,7 +102,7 @@ gst_ffmpeg_channel_layout_to_gst (guint64 channel_layout, gint channels,
   guint nchannels = 0;
   gboolean none_layout = FALSE;
 
-  if (channel_layout == 0) {
+  if (channel_layout == 0 || channels > 64) {
     nchannels = channels;
     none_layout = TRUE;
   } else {
@@ -163,7 +163,7 @@ gst_ffmpeg_channel_layout_to_gst (guint64 channel_layout, gint channels,
     } else {
       guint i;
 
-      for (i = 0; i < nchannels; i++)
+      for (i = 0; i < nchannels && i < 64; i++)
         pos[i] = GST_AUDIO_CHANNEL_POSITION_NONE;
     }
   }