flacenc: Correctly handle up to 255 cue entries

The counter was using a signed 8 bit integer, which was overflowing
after 127 entries. That was then passed as an unsigned 32 bit integer to
libflac, which caused it to be converted to a huge unsigned number.
That then caused an invalid memory access inside libflac.

As a bonus, signed integer overflow is undefined behaviour.

Instead, use an unsigned 8 bit integer. Once this overflows the existing
code already catches it and stops adding the cue. While FLAC__metadata_object_cuesheet_insert_track()
takes an unsigned 32 bit integer for the track number, FLAC__StreamMetadata_CueSheet_Track is
limiting it to an unsigned 8 bit integer.

Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2921

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5436>

diff --git a/subprojects/gst-plugins-good/ext/flac/gstflacenc.c b/subprojects/gst-plugins-good/ext/flac/gstflacenc.c
index ce91231..266c69f 100644 (file)
--- a/subprojects/gst-plugins-good/ext/flac/gstflacenc.c
+++ b/subprojects/gst-plugins-good/ext/flac/gstflacenc.c
@@ -484,7 +484,7 @@ static gboolean
 add_cuesheet (const GstToc * toc, guint sample_rate,
     FLAC__StreamMetadata * cuesheet)
 {
-  gint8 track_num = 0;
+  guint8 track_num = 0;
   gint64 start, stop;
   gchar *isrc = NULL;
   const gchar *is_legal;