+2006-03-13 Wim Taymans <wim@fluendo.com>
+
+ * docs/design/draft-klass.txt:
+ * docs/design/part-clocks.txt:
+ * docs/design/part-events.txt:
+ * docs/design/part-gstbin.txt:
+ * docs/design/part-gstpipeline.txt:
+ * docs/design/part-messages.txt:
+ * docs/design/part-negotiation.txt:
+ * docs/design/part-overview.txt:
+ * docs/design/part-preroll.txt:
+ * docs/design/part-seeking.txt:
+ * docs/design/part-states.txt:
+ * docs/design/part-streams.txt:
+ Documentation updates.
+
2006-03-12 Julien MOUTTE <julien@moutte.net>
* gst/gsttaglist.c: Fix rubbish docs that are encouraging
<keyword>['/'<keyword]*
- The string consists of an unordered list of keywords separated with a '/'
- character. While the / suggests a hierarchy, this is not the case.
+ The string consists of an _unordered_ list of keywords separated with a '/'
+ character. While the / suggests a hierarchy, this is _not_ the case.
2) keyword categories
filter can decide on input and output caps independently
of the stream contents (GstBaseTransform).
* Effect : applies an effect to some data, changes to data are
- intended.
+ intended. Examples are colorbalance, volume. These
+ elements can also be implemented with GstBaseTransform.
* Demuxer : splits audio, video, ... from a stream
* Muxer : interleave audio, video, ... into one stream, this is
like mixing but without losing or degrading each separate
* Decoder : decodes encoded data into a raw format, there is typically
no relation between input caps and output caps. The output
caps are defined in the stream data. This separates the
- decoder from the filter.
+ Decoder from the Filter and Effect.
* Encoder : encodes raw data into an encoded format.
* Mixer : combine audio, video, .. this is like muxing but with
applying some algorithm so that the individual streams
are not extractable anymore, there is therefore no
- reverse operation to mixing.
+ reverse operation to mixing. (audio mixer, video mixer, ...)
* Converter : convert audio into video, text to audio, ... The converter
typically works on raw types only. The source media type
is listed first.
id3demux : Extracter/Metadata
udpsrc : Source/Network/Protocol/Device
videomixer : Mixer/Video
- ffmpegcolorspace : Filter/Video (intended use to convert video with as little
+ ffmpegcolorspace : Filter/Video (intended use to convert video with as little
visible change as possible)
vertigotv : Effect/Video (intended use is to change the video)
volume : Effect/Audio (intended use is to change the audio data)
- demuxers are easier if they can just copy the timestamps as encoded in
the file. The initial NEWSEGMENT event would contain the lowest timestamp in
- the stream which makes the stream time start from 0.
+ the stream which makes the buffer stream time start from 0.
- pipelines requiring retimestamping of buffers can efficiently adjust
the timestamp in the NEWSEGMENT events and have all buffers retimestamped
automatically.
is usually generated by a demuxer or source. The event is generated before
pushing the first buffer and after a seek, right before pushing the new buffer.
-The NEWSEGMENT event can be send from both the application and the streaming
+The NEWSEGMENT event can be sent from both the application and the streaming
thread and should be serialized with the buffers.
Buffers should be clipped within the range indicated by the newsegment event
A QOS, or quality of service message, is generated in an element to report
to the upstream elements about the current quality of the stream. This
is typically done by the sinks that measure the amount of framedrops they
-have.
+have. (see part-qos.txt)
SEEK
We cannot lock the STREAM_LOCK before doing this since it might
cause a deadlock.
- 2) lock the STREAM_LOCK. This will work since the chain/loop function
+ 2) acquire the STREAM_LOCK. This will work since the chain/loop function
was unlocked in step 1).
3) perform the seek. since the STREAM_LOCK is held, the streaming thread
The application can retrieve the external GstBus and integrate it in the
mainloop or it can just _pop() messages off in its own thread.
-When a bin goes from READY into NULL state, it will set its bus to flushing,
-ie. the bus will drop all existing and new messages on the bus. This is
-necessary because bus messages hold references to the bin or its elements,
-so there are circular references that need to be broken if one ever wants
-to be able to destroy the bin properly.
+When a bin goes to READY it will clear all cached messages.
EOS
SEGMENT_START/DONE
------------------
-* not implemented.
-
A bin collects SEGMENT_START messages but does not post them to the application.
It counts the number of SEGMENT_START messages and posts a SEGMENT_STOP message
to the application when an equal number of SEGMENT_STOP messages where received.
+The cached SEGMENT_START/STOP messages are cleared when going to READY.
+
+
+DURATION
+--------
+
Subclassing
-----------
It is a good idea to update the GList of children so that the _iterate() functions
can still be used if the custom bin allows acces to its children.
+Any bin subclass can also implement a custom message handler by overriding the
+default message handler.
+
- reset the stream time to 0
- PAUSED -> PLAYING:
- - Select and set a clock.
+ - Select and a clock.
- calculate base time using the stream time.
+ - set clock and base time on all elements before performing the
+ state change.
- PAUSED -> PLAYING:
- calculate the stream time when the pipeline was stopped.
- READY -> NULL:
- - set the bus to flushing
+ - set the bus to flushing (when auto-flushing is enabled)
Clock selection
The default clock selection algorithm works as follows:
- If the application selected a clock, use that clock. (see below)
- - use clock of source elements (*)
- - use clock of other element, starting from the sinks going upstream.
- (+)
- - use GstSystemClock.
-(*) currently not implemented.
-(+) traversing the graph upstream to find the best clock is not implemented,
- currently the first element found that provides a clock is used.
+ - Use the clock of most upstream element that can provide a clock. This
+ selection is performed by iterating the element starting from the
+ sinks going upstream.
+
+ * since this selection procedure happens in the PAUSED->PLAYING
+ state change, all the sinks are prerolled and we can thus be sure
+ that each sink is linked to some upstream element.
+ * in the case of a live pipeline (NO_PREROLL), the sink will not
+ be prerolled and the selection process will select the clock of
+ a more upstream element.
+
+ - use GstSystemClock, this only happens when no element provides a
+ clock.
The application can influence this clock selection with two methods:
gst_pipeline_use_clock() and gst_pipeline_auto_clock().
The _auto_clock() method removes the fixed clock and reactivates the auto-
matic clock selection algorithm described above.
+
Seeking
-------
- record the current state of the pipeline.
- set the pipeline to paused if a FLUSHING seek is requested
- send the seek event to all sinks
- - when a FLUSH seek is done, the stream_time is set 0 again.
+ - when a FLUSH seek was performed successfully, the stream_time is set 0 again.
- restore old state of the pipeline.
GST_MESSAGE_BUFFERING:
- An element is buffering data and that could potentially take some time.
+ An element is buffering data and that could potentially take some time. This
+ message is typically emited by elements that perform some sort of network
+ buffering. While the pipeline is buffering it should remain in the PAUSED
+ state. When the buffering is finished, it can resume PLAYING.
GST_MESSAGE_STATE_CHANGED:
new format before processing the buffer data. If the data type on
the buffer is not acceptable, the element should refuse the buffer by
returning an appropriate return value from the chain function.
+ The core will automatically call the set_caps function for this purpose
+ when it is installed on the sink or source pad.
- When requesting a buffer from a bufferpool, the prefered type should
be passed to the buffer allocation function. After receiving a buffer
vorbisdec element decodes the compressed data and sends it to the alsasink
element. The alsasink element sends the samples to the audio card for playback.
- Downstream and upstream are the term used to describe the direction in the
+ Downstream and upstream are the terms used to describe the direction in the
Pipeline. From source to sink is called "downstream" and "upstream" is
from sink to source.
level functions on the pipeline object such as PLAY/PAUSE/STOP.
The application also receives messages and notifications from the pipeline such
- as metadata, warning or error messages.
+ as metadata, warning, error and EOS messages.
If the application needs more control over the graph it is possible to directly
access the elements and pads in the pipeline.
An element state change to PAUSED will activate the pads of the element. First the
source pads are activated, then the sinkpads. When the pads are activated, the
- pad activate function is called. Some pads will start a thread or some other
- mechanism to start producing or consuming data.
+ pad activate function is called. Some pads will start a thread (GstTask) or some
+ other mechanism to start producing or consuming data.
The PAUSED state is special as it is used to preroll data in the pipeline. The purpose
is to fill all connected elements in the pipeline with data so that the subsequent
- demuxer elements, these elements parse a stream and produce several output streams.
- mixer/muxer elements, combine several input streams into one output stream.
- Other categories of elements can be constructed.
+ Other categories of elements can be constructed (see part-klass.txt).
Bins
A pipeline is a special bin subclass that provides the following features to its
children:
- - Select and manage a clock
- - Provide means for elements to comunicate with the application by the Bus.
+ - Select and manage a clock.
+ - Manage stream time based on the selected clock.
+ - Provide means for elements to comunicate with the application by the GstBus.
- Manage the global state of the elements such as Errors and end-of-stream.
Normally the application creates one pipeline that will manage all the elements
the events are used to denote special conditions in the dataflow such as EOS or
to inform plugins of special events such as flushing or seeking.
+ Some events must be serialized with the buffer flow, others don't.
+
Pipeline construction
---------------------
finished. When all sinks have reported EOS, the pipeline forwards the EOS message
to the application.
- When in EOS, the pipeline remains in the playing state, if is the application
+ When in EOS, the pipeline remains in the playing state, it is the applications
responsability to PAUSE or READY the pipeline. The application can also issue
a seek, for example.
WRONG_STATE return value to the peer element. The sinkpad is
deactivated and becomes unusable for sending more data.
mp3dec to READY: the pads are deactivated and the state change completes when
- mp3dec leaves its _chain() function.
+ mp3dec leaves its _chain() function.
filesrc to READY: the pads are deactivated and the thread is paused.
The upstream elements finish their chain() function because the downstream element
- returned an error code from the _push() functions. These error codes are eventually
- returned to the element that started the streaming thread (filesrc), which pauses
- the thread and completes the state change.
+ returned an error code (WRONG_STATE) from the _push() functions. These error codes
+ are eventually returned to the element that started the streaming thread (filesrc),
+ which pauses the thread and completes the state change.
This sequence of events ensure that all elements are unblocked and all streaming
threads stopped.
message.
Several things can happen that require the preroll lock to be unlocked. This
-include state changes or flush events.
+include state changes or flush events. The prerolling is implemented in
+sinks (see part-element-sink.txt)
Committing the state
post a SEGMENT_DONE message on the bus. This message is posted by the
earliest element in the pipeline, typically a demuxer. After receiving
the message, the application can reconnect the pipeline or issue other
-seek events in the pipeline.
+seek events in the pipeline. Since the message is posted as early as
+possible in the pipeline, the application has some time to issue a new
+seek to make the transition seemless. Typically the allowed delay is
+defined by the buffer sizes of the sinks as well as the size of any
+queues in the pipeline.
The seek can also change the playback speed of the configured segment.
A speed of 1.0 is normal speed, 2.0 is double speed. Negative values
This seek type is typically performed after issuing segment seeks to finish
the playback of the pipeline.
+Performing a non-flushing seek in a PAUSED pipeline blocks until the pipeline
+is set to playing again since all data passing is blocked in the prerolled
+sinks.
+
segment seeking with FLUSH
--------------------------
and the base time. It stores this information to continue playback when
going back to the PLAYING state.
- sinks unblock any clock wait calls.
- - sinks return ASYNC from this state change and complete the state change
- when they receive a buffer or an EOS event.
+ - when a sink did not have a pending buffer to play, it returns ASYNC from
+ this state change and complete the state change when they receive a new buffer
+ or an EOS event.
- any queued EOS messages are removed since they will be reposted when going
- back to the PLAYING state.
+ back to the PLAYING state. The EOS messages are queued in GstBins.
- live sources stop generating data and return NO_PREROLL.
PAUSED -> READY
- sinks unblock any waits in the preroll.
- elements unblock any waits on devices
+ - chain or get_range functions return WRONG_STATE.
- the element pads are deactivated so that streaming becomes impossible and
all streaming threads are stopped.
GST_STATE_NO_PREROLL: The state change is completed successfully but the element
will not be able to produce data in the PAUSED state.
-In the case of an async state change, it is possible to proceed to the next
+In the case of an ASYNC state change, it is possible to proceed to the next
state before the current state change completed, however, the element will only
get to this next state before completing the previous ASYNC state change.
After receiving an ASYNC return value, you can use _element_get_state() to poll
If after calling the state function on all children, one of the children returned
NO_PREROLL, the function returns NO_PREROLL as well.
+If both NO_PREROLL and ASYNC children are present, NO_PREROLL is returned.
+
The current state of the bin can be retrieved with _get_state().
If the bin is performing an ASYNC state change, it will automatically update its
A->B: some elements ASYNC
- no commit state
- listen for commit messages on bus
- - for each commit message, poll elements
+ - for each commit message, poll elements, this happens in another
+ thread.
- if no ASYNC elements, commit state, continue state change
to STATE_PENDING
- STATE_LOCK is taken in set_state
- change state is called and returns ASYNC
- ASYNC returned to the caller.
- - element takes STATE_LOCK in streaming thread.
+ - element takes LOCK in streaming thread.
- element calls commit_state in streaming thread.
- commit state calls change_state to next state change.
+--+ +-++-+ +-+ +---+
1) NEW_SEGMENT, rate, start/stop, time
- - marks valid buffer timestamp range
- - marks stream_time of buffers in NEW_SEGMENT
- - marks playback rate
+ - marks valid buffer timestamp range (start, stop)
+ - marks stream_time of buffers (time)
+ - marks playback rate (rate)
2) N buffers
- displayable buffers are between start/stop of the NEW_SEGMENT
+
- display_time: (B.timestamp - NS.start) * NS.abs_rate
+ * used to calculate stream_time and sync_time
+
- stream_time: display_time + NS.time
- - sync_time: display_time + base_time
+ * current position in stream between 0 and duration
+
+ - sync_time: display_time + NS.accum + base_time
+ * used to synchronize against the clock.
3) EOS
- marks the end of data, nothing is to be expected after EOS
+
+
projects / platform / upstream / gstreamer.git / commitdiff