+Raw audio samples are usually passed around in interleaved form in
+GStreamer, which means that if there are multiple audio channels the
+samples for each channel are interleaved in memory, e.g.
+|LEFT|RIGHT|LEFT|RIGHT|LEFT|RIGHT| for stereo audio. A non-interleaved
+or planar arrangement in memory would look like
+|LEFT|LEFT|LEFT|RIGHT|RIGHT|RIGHT| instead, possibly with
+|LEFT|LEFT|LEFT| and |RIGHT|RIGHT|RIGHT| residing in separate memory
+chunks or separated by some padding.
+
+GStreamer has always had signalling for non-interleaved audio, but it
+was never actually properly implemented in any elements. audioconvert
+would advertise support for it, but wasn’t actually able to handle it.
+
+With this release we now have full support for non-interleaved audio as
+well, which means more efficient integration with external APIs that
+handle audio this way, but also more efficient processing of certain
+operations like interleaving multiple 1-channel streams into a
+multi-channel stream which can be done without memory copies now.
+
+New API to support this has been added to the GStreamer Audio support
+library: There is now a new GstAudioMeta which describes how data is
+laid out inside the buffer, and buffers with non-interleaved audio must
+always carry this meta. To access the non-interleaved audio samples you
+must map such buffers with gst_audio_buffer_map() which works much like
+gst_buffer_map() or gst_video_frame_map() in that it will populate a
+little GstAudioBuffer helper structure passed to it with the number of
+samples, the number of planes and pointers to the start of each plane in
+memory. This function can also be used to map interleaved audio buffers
+in which case there will be only one plane of interleaved samples.
+
+Of course support for this has also been implemented in the various
+audio helper and conversion APIs, base classes, and in elements such as
+audioconvert, audioresample, audiotestsrc, audiorate.
+
+Support for Closed Captions and Other Ancillary Data in Video
+
+The video support library has gained support for detecting and
+extracting Ancillary Data from videos as per the SMPTE S291M
+specification, including:
+
+- a VBI (Video Blanking Interval) parser that can detect and extract
+ Ancillary Data from Vertical Blanking Interval lines of component
+ signals. This is currently supported for videos in v210 and UYVY
+ format.
+
+- a new GstMeta for closed captions: GstVideoCaptionMeta. This
+ supports the two types of closed captions, CEA-608 and CEA-708,
+ along with the four different ways they can be transported (other
+ systems are a superset of those).
+
+- a VBI (Video Blanking Interval) encoder for writing ancillary data
+ to the Vertical Blanking Interval lines of component signals.
+
+The new closedcaption plugin in gst-plugins-bad then makes use of all
+this new infrastructure and provides the following elements:
+
+- cccombiner: a closed caption combiner that takes a closed captions
+ stream and another stream and adds the closed captions as
+ GstVideoCaptionMeta to the buffers of the other stream.
+
+- ccextractor: a closed caption extractor which will take
+ GstVideoCaptionMeta from input buffers and output them as a separate
+ closed captions stream.
+
+- ccconverter: a closed caption converter that can convert between
+ different formats
+
+- line21decoder: extract line21 closed captions from SD video streams
+
+- cc708overlay: decodes CEA 608/708 captions and overlays them on
+ video
+
+Additionally, the following elements have also gained Closed Caption
+support:
+
+- qtdemux and qtmux support CEA 608/708 Closed Caption tracks
+
+- mpegvideoparse extracts Closed Captions from MPEG-2 video streams
+
+- decklinkvideosink can output closed captions and decklinkvideosrc
+ can extract closed captions
+
+- playbin and playbin3 learned how to autoplug CEA 608/708 CC overlay
+ elements
+
+The rsclosedcaption plugin in the Rust plugins collection includes a
+MacCaption (MCC) file parser and encoder.
+
+New Elements
+
+- overlaycomposition: New element that allows applications to draw
+ GstVideoOverlayCompositions on a stream. The element will emit the
+ "draw" signal for each video buffer, and the application then
+ generates an overlay for that frame (or not). This is much more
+ performant than e.g. cairooverlay for many use cases, e.g. because
+ pixel format conversions can be avoided or the blitting of the
+ overlay can be delegated to downstream elements (such as
+ gloverlaycompositor). It’s particularly useful for cases where only
+ a small section of the video frame should be drawn on.
+
+- gloverlaycompositor: New OpenGL-based compositor element that
+ flattens any overlays from GstVideoOverlayCompositionMetas into the
+ video stream.
+
+- glalpha: New element that adds an alpha channel to a video stream.
+ The values of the alpha channel can either be set to a constant or
+ can be dynamically calculated via chroma keying. It is similar to
+ the existing alpha element but based on OpenGL. Calculations are
+ done in floating point so results may not be identical to the output
+ of the existing alpha element.
+
+- rtpfunnel funnels together rtp-streams into a single session. Use
+ cases include multiplexing and bundle. webrtcbin uses it to
+ implement BUNDLE support.
+
+- testsrcbin is a source element that provides an audio and/or video
+ stream and also announces them using the recently-introduced
+ GstStream API. This is useful for testing elements such as playbin3
+ or uridecodebin3 etc.
+
+- New closed caption elements: cccombiner, ccextractor, ccconverter,
+ line21decoder and cc708overlay (see above)
+
+- wpesrc: new source element acting as a Web Browser based on WebKit
+ WPE
+
+- Two new OpenCV-based elements: cameracalibrate and cameraundistort
+ who can communicate to figure out distortion correction parameters
+ for a camera and correct for the distortion.
+
+- new sctp plugin based on usrsctp with sctpenc and sctpdec elements
+
+New element features and additions
+
+- playbin3, playbin and playsink have gained a new "text-offset"
+ property to adjust the positioning of the selected subtitle stream
+ vis-a-vis the audio and video streams. This uses subtitleoverlay’s
+ new "subtitle-ts-offset" property. GstPlayer has gained matching API
+ for this, namely gst_player_get_text_video_offset().
+
+- playbin3 buffering improvements: in network playback scenarios there
+ may be multiple inputs to decodebin3, and buffering will be done
+ before decodebin3 using queue2 or downloadbuffer elements inside
+ urisourcebin. Since this is before any parsers or demuxers there may
+ not be any bitrate information available for the various streams, so
+ it was difficult to configure the buffering there smartly within
+ global constraints. This was improved now: The queue2 elements
+ inside urisourcebin will now use the new bitrate query to figure out
+ a bitrate estimate for the stream if no bitrate was provided by
+ upstream, and urisourcebin will use the bitrates of the individual
+ queues to distribute the globally-set "buffer-size" budget in bytes
+ to the various queues. urisourcebin also gained "low-watermark" and
+ "high-watermark" properties which will be proxied to the internal
+ queues, as well as a read-only "statistics" property which allows
+ querying of the minimum/maximum/average byte and time levels of the
+ queues inside the urisourcebin in question.
+
+- splitmuxsink has gained a couple of new features:
+
+ - new "async-finalize" mode: This mode is useful for muxers or
+ outputs that can take a long time to finalize a file. Instead of
+ blocking the whole upstream pipeline while the muxer is doing
+ its stuff, we can unlink it and spawn a new muxer + sink
+ combination to continue running normally. This requires us to
+ receive the muxer and sink (if needed) as factories via the new
+ "muxer-factory" and "sink-factory" properties, optionally
+ accompanied by their respective properties structures (set via
+ the new "muxer-properties" and "sink-properties" properties).
+ There are also new "muxer-added" and "sink-added" signals in
+ case custom code has to be called for them to configure them.
+
+ - "split-at-running-time" action signal: When called by the user,
+ this action signal ends the current file (and starts a new one)
+ as soon as the given running time is reached. If called multiple
+ times, running times are queued up and processed in the order
+ they were given.
+
+ - "split-after" action signal to finish outputting the current GOP
+ to the current file and then start a new file as soon as the GOP
+ is finished and a new GOP is opened (unlike the existing
+ "split-now" which immediately finishes the current file and
+ writes the current GOP into the next newly-started file).
+
+ - "reset-muxer" property: when unset, the muxer is reset using
+ flush events instead of setting its state to NULL and back. This
+ means the muxer can keep state across resets, e.g. mpegtsmux
+ will keep the continuity counter continuous across segments as
+ required by hlssink2.
+
+- qtdemux gained PIFF track encryption box support in addition to the
+ already-existing PIFF sample encryption support, and also allows
+ applications to select which encryption system to use via a
+ "drm-preferred-decryption-system-id" context in case there are
+ multiple options.
+
+- qtmux: the "start-gap-threshold" property determines now whether an
+ edit list will be created to account for small gaps or offsets at
+ the beginning of a stream in case the start timestamps of tracks
+ don’t line up perfectly. Previously the threshold was hard-coded to
+ 1% of the (video) frame duration, now it is 0 by default (so edit
+ list will be created even for small differences), but fully
+ configurable.
+
+- rtpjitterbuffer has improved end-of-stream handling
+
+- rtpmp4vpay will be prefered over rtpmp4gpay for MPEG-4 video in
+ autoplugging scenarios now
+
+- rtspsrc now allows applications to send RTSP SET_PARAMETER and
+ GET_PARAMETER requests using action signals.
+
+- rtspsrc also has a small (100ms) configurable teardown delay by
+ default to try and make sure an RTSP TEARDOWN request gets sent out
+ when the source element shuts down. This will block the downward
+ PAUSED to READY state change for a short time, but can be unset
+ where it’s a problem. Some servers only allow a limited number of
+ concurren clients, so if no proper TEARDOWN is sent clients may have
+ problems connecting to the server for a while.
+
+- souphttpsrc behaves better with low bitrate streams now. Before it
+ would increase the read block size too quickly which could lead to
+ it not reading any data from the socket for a very long time with
+ low bitrate streams that are output live downstream. This could lead
+ to servers kicking off the client.
+
+- filesink: do internal buffering to avoid performance regression with
+ small writes since we bypass libc buffering by using writev()
+
+- identity: add "eos-after" property and fix "error-after" property
+ when the element is reused
+
+- input-selector: lets context queries pass through, so that
+ e.g. upstream OpenGL elements can use contexts and displays
+ advertised by downstream elements
+
+- queue2: avoid ping-pong between 0% and 100% buffering messages if
+ upstream is pushing buffers larger than one of its limits, plus
+ performance optimisations
+
+- opusdec: new "phase-inversion" property to control phase inversion.
+ When enabled, this will slightly increase stereo quality, but
+ produces a stream that when downmixed to mono will suffer audio
+ distortions.
+
+- The x265enc HEVC encoder also exposes a "key-int-max" property to
+ configure the maximum allowed GOP size now.
+
+- decklinkvideosink has seen stability improvements for long-running
+ pipelines (potential crash due to overflow of leaked clock refcount)
+ and clock-slaving improvements when performing flushing seeks
+ (causing stalls in the output timeline), pausing and/or buffering.
+
+- srtpdec, srtpenc: add support for MKIs which allow multiple keys to
+ be used with a single SRTP stream
+
+- The srt Secure Reliable Transport plugin has integrated server and
+ client elements srt{client,server}{src,sink} into one (srtsrc and
+ srtsink), since SRT connection mode can be changed by uri
+ parameters.
+
+- h264parse and h265parse will handle SEI recovery point messages and
+ mark recovery points as keyframes as well (in addition to IDR
+ frames)
+
+- webrtcbin: "add-turn-server" action signal to pass multiple ICE
+ relays (TURN servers).
+
+- The removesilence element has received various new features and
+ properties, such as a
+ "threshold"1 property, detecting silence only after minimum silence time/buffers, a“silent”property to control bus message notifications as well as a“squash”`
+ property.
+
+- AOMedia AV1 decoder gained support for 10/12bit decoding whilst the
+ AV1 encoder supports more image formats and subsamplings now and
+ acquired support for rate control and profile related configuration.
+
+- The Fraunhofer fdkaac plugin can now be built against the 2.0.0
+ version API and has improved multichannel support
+
+- kmssink now supports unpadded 24-bit RGB and can configure mode
+ setting from video info, which enables display of multi-planar
+ formats such as I420 or NV12 with modesetting. It has also gained a
+ number of new properties: The "restore-crtc" property does what it
+ says on the tin and is enabled by default. "plane-properties" and
+ "connector-properties" can be used to pass custom properties to the
+ DRM.
+
+- waylandsink has a "fullscreen" property now.
+
+Plugin and library moves
+
+- The stereo element was moved from -bad into the existing audiofx
+ plugin in -good. If you get duplicate type registration warnings
+ when upgrading, check that you don’t have a stale gststereo plugin
+ lying about somewhere.
+
+GstVideoAggregator, compositor, and OpenGL mixer elements moved from -bad to -base
+
+GstVideoAggregator is a new base class for raw video mixers and muxers
+and is based on [GstAggregator][aggregator]. It provides defined-latency
+mixing of raw video inputs and ensures that the pipeline won’t stall
+even if one of the input streams stops producing data.
+
+As part of the move to stabilise the API there were some last-minute API
+changes and clean-ups, but those should mostly affect internal elements.
+Most notably, the "ignore-eos" pad property was renamed to
+"repeat-after-eos" and the conversion code was moved to a
+GstVideoAggregatorConvertPad subclass to avoid code duplication, make
+things less awkward for subclasses like the OpenGL-based video mixer,
+and make the API more consistent with the audio aggregator API.
+
+It is used by the compositor element, which is a replacement for
+‘videomixer’ which did not handle live inputs very well. compositor
+should behave much better in that respect and generally behave as one
+would expected in most scenarios.
+
+The compositor element has gained support for per-pad blending mode
+operators (SOURCE, OVER, ADD) which determines what operator to use for
+blending this pad over the previous ones. This can be used to implement
+crossfading.
+
+A number of OpenGL-based video mixer elements (glvideomixer, glmixerbin,
+glvideomixerelement, glstereomix, glmosaic) which are built on top of
+GstVideoAggregator have also been moved from -bad to -base now. These
+elements have been merged into the existing OpenGL plugin, so if you get
+duplicate type registration warnings when upgrading, check that you
+don’t have a stale gstopenglmixers plugin lying about somewhere.
+
+Plugin removals
+
+The following plugins have been removed from gst-plugins-bad:
+
+- The experimental daala plugin has been removed, since it’s not so
+ useful now that all effort is focused on AV1 instead, and it had to
+ be enabled explicitly with --enable-experimental anyway.
+
+- The spc plugin has been removed. It has been replaced by the gme
+ plugin.
+
+- The acmmp3dec and acmenc plugins for Windows have been removed. ACM
+ is an ancient legacy API and there was no point in keeping them
+ around for a licensed mp3 decoder now that mp3 patents have expired
+ and we have a decoder in -good. We also didn’t ship these in our
+ cerbero-built Windows packages, so it’s unlikely that they’ll be
+ missed.
+
+
+Miscellaneous API additions
+
+- GstBitwriter: new generic bit writer API to complement the existing
+ bit reader
+
+- gst_buffer_new_wrapped_bytes() creates a wrap buffer from a GBytes
+
+- gst_caps_set_features_simple() sets a caps feature on all the
+ structures of a GstCaps
+
+- New GST_QUERY_BITRATE query: This allows determining from downstream
+ what the expected bitrate of a stream may be which is useful in
+ queue2 for setting time based limits when upstream does not provide
+ timing information. tsdemux, qtdemux and matroskademux have basic
+ support for this query on their sink pads.
+
+- elements: there is a new “Hardware” class specifier. Elements
+ interacting with hardware devices should specify this classifier in
+ their element factory class metadata. This is useful to advertise as
+ one might need to put such elements into READY state to test if the
+ hardware is present in the system for example.
+
+- protection: Add a new definition for unspecified system protection
+
+- take functions for various mini objects that didn’t have them yet:
+ gst_query_take(), gst_message_take(), gst_tag_list_take(),
+ gst_buffer_list_take(). Unlike the various _replace() functions
+ _take() does not increase the reference count but takes ownership of
+ the mini object passed.
+
+- clear functions for various mini object types and GstObject which
+ unrefs the object or mini object (if non-NULL) and sets the variable
+ pointed to to NULL: gst_clear_structure(), gst_clear_tag_list(),
+ gst_clear_query(), gst_clear_message(), gst_clear_event(),
+ gst_clear_caps(), gst_clear_buffer_list(), gst_clear_buffer(),
+ gst_clear_mini_object(), gst_clear_object()
+
+- miniobject: new API gst_mini_object_add_parent() and
+ gst_mini_object_remove_parent()to set parent pointers on mini objects to ensure correct writability: Every container of miniobjects now needs to store itself as parent in the child object, and remove itself again later. A mini object is then only writable if there is at most one parent, that parent is writable itself, and the reference count of the mini object is 1.GstBuffer(for memories),GstBufferList(for buffers),GstSample(for caps, buffer, bufferlist), andGstVideoOverlayComposition`
+ were updated accordingly. Without this it was possible to have
+ e.g. a buffer list with a refcount of 2 used in two places at once
+ that both modify the same buffer with refcount 1 at the same time
+ wrongly thinking it is writable even though it’s really not.
+
+- poll: add API to watch for POLLPRI and stop treating POLLPRI as a
+ read. This is useful to wait for video4linux events which are
+ signalled via POLLPRI.