Trickmodes
-==========
+----------
GStreamer provides API for performing various trickmode playback. This includes:
General seeking overview
-------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~
Consider a typical playback pipeline:
When performing a seek, the following steps have to be taken by the application:
- 1) Create a seek event:
+Create a seek event
+^^^^^^^^^^^^^^^^^^^
- The seek event contains:
+The seek event contains:
- - various flags describing:
- - where to seek to (KEY_UNIT)
- - how accurate the seek should be (ACCURATE)
- - how to perform the seek (FLUSH)
- - what to do when the stop position is reached (SEGMENT).
- - extra playback options (SKIP)
- - a format to seek in, this can be time, bytes, units (frames, samples), ...
- - a playback rate, 1.0 is normal playback speed, positive values bigger than 1.0
- mean fast playback. negative values mean reverse playback. A playback speed of
- 0.0 is not allowed (but is equivalent to PAUSING the pipeline).
- - a start position, this value has to be between 0 and the total duration of the
- file. It can also be relative to the previously configured start value.
- - a stop position, this value has to be between 0 and the total duration. It can
- also be relative to the previously configured stop value.
+- various flags describing:
+ - where to seek to (KEY_UNIT)
+ - how accurate the seek should be (ACCURATE)
+ - how to perform the seek (FLUSH)
+ - what to do when the stop position is reached (SEGMENT).
+ - extra playback options (SKIP)
+- a format to seek in, this can be time, bytes, units (frames, samples), ...
+- a playback rate, 1.0 is normal playback speed, positive values bigger than 1.0
+ mean fast playback. negative values mean reverse playback. A playback speed of
+ 0.0 is not allowed (but is equivalent to PAUSING the pipeline).
+- a start position, this value has to be between 0 and the total duration of the
+ file. It can also be relative to the previously configured start value.
+- a stop position, this value has to be between 0 and the total duration. It can
+ also be relative to the previously configured stop value.
- See also gst_event_new_seek().
+See also gst_event_new_seek().
- 2) Send the seek event to the pipeline with gst_element_send_event()
+Send the seek event
+^^^^^^^^^^^^^^^^^^^
- By default the pipeline will send the event to all sink elements.
- By default an element will forward the event upstream on all sinkpads.
- Elements can modify the format of the seek event. The most common format is
- GST_FORMAT_TIME.
+Send the new seek event to the pipeline with gst_element_send_event().
- One element will actually perform the seek, this is usually the demuxer or
- source element. For more information on how to perform the different seek
- types see part-seeking.txt.
+By default the pipeline will send the event to all sink elements.
+By default an element will forward the event upstream on all sinkpads.
+Elements can modify the format of the seek event. The most common format is
+GST_FORMAT_TIME.
- For client side trickmode a NEW_SEGMENT event will be sent downstream with
- the new rate and start/stop positions. All elements prepare themselves to
- handle the rate (see below). The applied rate of the NEW_SEGMENT event will
- be set to 1.0 to indicate that no rate adjustment has been done.
+One element will actually perform the seek, this is usually the demuxer or
+source element. For more information on how to perform the different seek
+types see part-seeking.txt.
- for server side trick mode a NEW_SEGMENT event is sent downstream with a
- rate of 1.0 and the start/stop positions. The elements will configure themselves
- for normal playback speed since the server will perform the rate conversions.
- The applied rate will be set to the rate that will be applied by the server. This
- is done to insure that the position reporting performed in the sink is aware
- of the trick mode.
+For client side trickmode a SEGMENT event will be sent downstream with
+the new rate and start/stop positions. All elements prepare themselves to
+handle the rate (see below). The applied rate of the SEGMENT event will
+be set to 1.0 to indicate that no rate adjustment has been done.
- When the seek succeeds, the _send_event() function will return TRUE.
+for server side trick mode a SEGMENT event is sent downstream with a
+rate of 1.0 and the start/stop positions. The elements will configure themselves
+for normal playback speed since the server will perform the rate conversions.
+The applied rate will be set to the rate that will be applied by the server. This
+is done to insure that the position reporting performed in the sink is aware
+of the trick mode.
+
+When the seek succeeds, the _send_event() function will return TRUE.
Server side trickmode
----------------------
+~~~~~~~~~~~~~~~~~~~~~
The source element operates in push mode. It can reopen a server connection requesting
a new byte or time position and a new playback speed. The capabilities can be queried
client side forward trickmodes
-------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The seek happens as stated above. a NEW_SEGMENT event is sent downstream with a rate
-different from 1.0. Plugins receiving the NEW_SEGMENT can decide to perform the
+The seek happens as stated above. a SEGMENT event is sent downstream with a rate
+different from 1.0. Plugins receiving the SEGMENT can decide to perform the
rate conversion of the media data (retimestamp video frames, resample audio, ...).
-A plugin should not drop frames unless the SKIP flag is set. When the SKIP flag is
-set, a plugin can decide to drop frames in the case of fast playback or use a more
-efficient decoding algorithm (skip B frames, ...). If a plugin decides to resample
-or retimestamp, it should modify the NEW_SEGMENT with a rate of 1.0 and update the
-applied rate so that downstream elements don't resample again but are aware that the
-media has been modified.
+
+If a plugin decides to resample or retimestamp, it should modify the SEGMENT with
+a rate of 1.0 and update the applied rate so that downstream elements don't resample
+again but are aware that the media has been modified.
The GStreamer base audio and video sinks will resample automatically if they receive
-a NEW_SEGMENT event with a rate different from 1.0. The position reporting in the
+a SEGMENT event with a rate different from 1.0. The position reporting in the
base audio and video sinks will also depend on the applied rate of the segment
information.
+When the SKIP flag is set, frames can be dropped in the elements. If S is the
+speedup factor, a good algorithm for implementing frame skipping is to send audio in
+chunks of Nms (usualy 300ms is good) and then skip ((S-1) * Nns) of audio data.
+For the video we send only the keyframes in the (S * Nns) interval. In this
+case, the demuxer would scale the timestamps and would set an applied rate of S.
+
client side backwards trickmode
--------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For backwards playback the following rules apply:
- - the rate in the NEW_SEGMENT is less than 0.0.
- - the NEW_SEGMENT start position is less than the stop position, playback will
+ - the rate in the SEGMENT is less than 0.0.
+ - the SEGMENT start position is less than the stop position, playback will
however happen from stop to start in reverse.
- - the time member in the NEW_SEGMENT is set to the stream time of the start
+ - the time member in the SEGMENT is set to the stream time of the start
position.
For plugins the following rules apply:
forward continuous with the previous buffer.
- A video decoder decodes and accumulates all decoded frames. If a buffer with
- a DISCONT, accumulate NEWSEGMENT or EOS is received, all accumulated frames
- are sent downsteam in reverse.
+ a DISCONT, SEGMENT or EOS is received, all accumulated frames are sent
+ downsteam in reverse.
- An audio decoder decodes and accumulates all decoded audio. If a buffer with
- a DISCONT, accumulate NEWSEGMENT or EOS is received, all accumulated audio
- is sent downstream in reverse order. Some audio codecs need the previous
+ a DISCONT, SEGMENT or EOS is received, all accumulated audio is sent
+ downstream in reverse order. Some audio codecs need the previous
data buffer to decode the current one, in that case, the previous DISCONT
buffer needs to be combined with the last non-DISCONT buffer to generate the
last bit of output.
- for transcoding, audio and video resamplers can be used to reverse, resample
and retimestamp the buffers. Any rate adjustments performed on the media must
be added to the applied_rate and subtracted from the rate members in the
- NEWSEGMENT event.
+ SEGMENT event.
+
+ In SKIP mode, the same algorithm as for forward SKIP mode can be used.
+
-Notes:
+Notes
+~~~~~
- The clock/running_time keeps running forward.
- backwards playback potentially uses a lot of memory as frames and undecoded
projects / platform / upstream / gstreamer.git / blobdiff