+2007-09-24 Wim Taymans <wim.taymans@gmail.com>
+
+ * docs/design/draft-latency.txt:
+ * docs/design/draft-push-pull.txt:
+ * docs/design/draft-tagreading.txt:
+ * docs/design/part-MT-refcounting.txt:
+ * docs/design/part-activation.txt:
+ * docs/design/part-block.txt:
+ * docs/design/part-element-source.txt:
+ * docs/design/part-events.txt:
+ * docs/design/part-gstbin.txt:
+ * docs/design/part-gstelement.txt:
+ * docs/design/part-gstobject.txt:
+ * docs/design/part-gstpipeline.txt:
+ * docs/design/part-messages.txt:
+ * docs/design/part-preroll.txt:
+ * docs/design/part-push-pull.txt:
+ * docs/design/part-qos.txt:
+ * docs/design/part-query.txt:
+ * docs/design/part-scheduling.txt:
+ * docs/design/part-seeking.txt:
+ * docs/design/part-segments.txt:
+ * docs/design/part-states.txt:
+ Documentation updates and typo fixes.
+
2007-09-23 Tim-Philipp Müller <tim at centricular dot net>
* plugins/elements/gstfakesink.c:
never change.
PAUSED->PLAYING:
- asrc clock selected because it is the most upstream clock provider.
- asink: PAUSED:->PLAYING, returns ASYNC because it is not prerolled
- asrc: PAUSED->PLAYING: sets pending state to PLAYING, returns ASYNC. The sink
- will commit state to PLAYING when it prerolls.
+ asrc clock selected because it is the most upstream clock provider. asink can
+ only provide a clock when it received the first buffer and configured the
+ device with the samplerate in the caps.
+
+ asink: PAUSED:->PLAYING, sets pending state to PLAYING, returns ASYNC becaus
+ it is not prerolled. The sink will commit state to
+ PLAYING when it prerolls.
+ asrc: PAUSED->PLAYING: starts pushing buffers.
* since the sink is still performing a state change from READY -> PAUSED, it
remains ASYNC. The pending state will be set to PLAYING.
- perform a latency query on all sinks.
- latency = MAX (all min latencies)
- if MIN (all max latencies) < latency we have an impossible situation and we
- must generate an error indicating that this pipeline cannot be played.
+ must generate an error indicating that this pipeline cannot be played. This
+ usually means that there is not enough buffering in some chain of the
+ pipeline. A queue can be added to those chains.
The sinks gather this information with a LATENCY query upstream. Intermediate
elements pass the query upstream and add the amount of latency they add to the
Problems:
- - core makes a touch desicion without knowing anything about the
+ - core makes a tough desicion without knowing anything about the
element. Some elements are able to deal with a pull_range()
without offset while others need full random access.
how it can use the peer element pull_range. This includes if the
peer can operate with or without offset/size. This also means that
the core does not need to select the scheduling method anymore and
- allows for more afficient scheduling methods adjusted for the
+ allows for more efficient scheduling methods adjusted for the
particular element.
The tagreading (metadata reading) use case for mediacenter applications is not
to well supported by the current GStreamer architecture. It uses demuxers on the
-files, which generelly said takes too long. what we wan't is secialized elements
+files, which generelly said takes too long. what we want is secialized elements
that just do the tag-reading.
The idea is to have one plugin 'tagread' simmilar to 'typefind' that provides
threads to perform the data passing and various asynchronous services such as
the clock can also use threads.
-This design decision has implication for the usage of the API and the objects
+This design decision has implications for the usage of the API and the objects
which this document explains.
MT safety techniques
on CPU features and instructions.
The advantages are mostly speed related since there are no heavyweight locks
- involved. Most of this instructions also do not cause a context switch in case
+ involved. Most of these instructions also do not cause a context switch in case
of concurrent access but use a retry mechanism or spinlocking.
Disadvantages are that each of these instructions usually cause a cache flush
All properties requiring a lock can change after releasing the associated
lock. This means that as long as you hold the lock, the state of the
object regarding the locked properties is consistent with the information
- obtained. As soon as the lock is released, any values required from the
+ obtained. As soon as the lock is released, any values acquired from the
properties might not be valid anymore and can as best be described as a
snapshot of the state when the lock was held.
The following code is equivalent to the above but with using the functions
to access object properties.
- peer = gst_pad_get_parent (pad);
+ peer = gst_pad_get_peer (pad);
if (peer) {
... use peer ...
function should g_free() the name after usage.
GST_OBJECT_LOCK (object)
- name = g_strdup (object->name);
+ name = g_strdup (GST_OBJECT_NAME (object));
GST_OBJECT_UNLOCK (object)
... use name ...
start tasks to drive the data flow.
An element activates its pads from sourcepads to sinkpads. This to make
-sure that when the sinkpads are activated an ready to accept data, the
+sure that when the sinkpads are activated and ready to accept data, the
sourcepads are already active to pass the data downstream.
Pads can be activated in one of two modes, PUSH and PULL. PUSH pads are
the FLUSH_START event further downstream.
The FLUSH_STOP will set the srcpad to non-flushing again and is dropped
- fr the same reason. From then on, the new data after the flushing seek
+ for the same reason. From then on, the new data after the flushing seek
will be queued when the pad block is taken again.
* Case where the stream is blocking downstream
Flushing, scheduling and sync is all handled by this base class.
+
+Timestamps
+----------
+
+A non-live source should timestamp the buffers it produces starting from 0. If
+it is not possible to timestamp every buffer (filesrc), the source is allowed to
+only timestamp the first buffer (as 0).
+
+Live sources only produce data in the PLAYING state, when the clock is running.
+They should timestamp each buffer they produce with the current running_time of
+the pipeline, which is expressed as:
+
+ absolute_time - base_time
+
+With absolute_time the time obtained from the global pipeline with
+gst_clock_get_time() and base_time being the time of that clock when the
+pipeline was last set to PLAYING.
Flushing happens in two stages.
- 1) a source filter sends the FLUSH_START event to the downstream peer element. The
- downstream element starts rejecting buffers from the upstream elements. It
+ 1) a source element sends the FLUSH_START event to the downstream peer element.
+ The downstream element starts rejecting buffers from the upstream elements. It
sends the flush event further downstream and discards any buffers it is
holding as well as return from the chain function as soon as possible.
This makes sure that all upstream elements get unblocked.
This event is not synchronized with the STREAM_LOCK and can be done in the
application thread.
- 2) a source filter sends the FLUSH_STOP event to indicate
+ 2) a source element sends the FLUSH_STOP event to indicate
that the downstream element can accept buffers again. The downstream
element sends the flush event to its peer elements. After this step dataflow
continues. The FLUSH_STOP call is synchronized with the STREAM_LOCK so any
all the pads before forwarding the event.
The EOS event should always be interleaved with the data flow, therefore the
-STREAM_LOCK should be taken.
+GStreamer core will take the STREAM_LOCK.
Sometimes the EOS event is generated by another element than the source, for
example a demuxer element can generate an EOS event before the source element.
An element can suggest a buffersize for downstream elements. This is
typically done by elements that produce data on multiple source pads
-such as demuxers. This event is currently not yet defined or used.
+such as demuxers. This event is currently not yet defined nor used.
QOS
A QOS, or quality of service message, is generated in an element to report
to the upstream elements about the current quality of real-time performance
-the stream. This is typically done by the sinks that measure the amount of
-framedrops they have. (see part-qos.txt)
+of the stream. This is typically done by the sinks that measure the amount
+of framedrops they have. (see part-qos.txt)
SEEK
external message bus, except for EOS and SEGMENT_START/DONE which are
handled specially.
-STATE_CHANGED messages received from the children are used to trigger a
-recalculation of the current state of the bin, as described in
+ASYNC_START/ASYNC_STOP messages received from the children are used to
+trigger a recalculation of the current state of the bin, as described in
part-states.txt.
The application can retrieve the external GstBus and integrate it in the
Pads
----
-GstPads are the property of a given GstElement. They provide the connection capability, with allowing arbitrary
-structure in the graph. For any Element but a source or sink, there will be at least 2 Pads owned by the Element.
-These pads are stored in a single GList within the Element. Several counters are kept in order to allow quicker
-determination of the type and properties of a given Element.
-
-Pads may be added to an element with _add_pad. Retrieval is via _get_pad(), which operates on the name of the Pad (the
-unique key). This means that all Pads owned by a given Element must have unique names (FIXME we don't verify this at
-_add time). A pointer to the GList of pads may be obtained with _get_pad_list. As with the element's name,
-precaution must be taken with all these pointers, as they are the same pointer that the Element uses internally. One
-must be especially careful not to manipulate the list of pads.
+GstPads are the property of a given GstElement. They provide the connection
+capability, with allowing arbitrary structure in the graph. For any Element
+but a source or sink, there will be at least 2 Pads owned by the Element.
+These pads are stored in a single GList within the Element. Several counters
+are kept in order to allow quicker determination of the type and properties of
+a given Element.
+
+Pads may be added to an element with _add_pad. Retrieval is via _get_pad(),
+which operates on the name of the Pad (the unique key). This means that all
+Pads owned by a given Element must have unique names.
+A pointer to the GList of pads may be obtained with _iterate_pads.
gst_element_add_pad(element,pads):
- Sets the element as the parent of the pad, then adds the pad to the element's list of pads, keeping the counts
- of total, src, and sink pads up to date. Emits the "new_pad" signal with the pad as argument. Fails if either
- the element or pad are either NULL or not what they claim to be. Should fail if the pad already has a parent.
- Should fail if the pad is already owned by the element. Should fail if there's already a pad by that name in
- the the list of pads.
+ Sets the element as the parent of the pad, then adds the pad to the
+ element's list of pads, keeping the counts of total, src, and sink pads
+ up to date. Emits the "new_pad" signal with the pad as argument.
+ Fails if either the element or pad are either NULL or not what they
+ claim to be. Should fail if the pad already has a parent. Should fail
+ if the pad is already owned by the element. Should fail if there's
+ already a pad by that name in the the list of pads.
pad = gst_element_get_pad(element,"padname"):
Searches through the list of pads
---------
A pointer is available to store the current parent of the object. This is one
-of the two fundamental requires for a hierarchical system such as GStreamer
+of the two fundamental requirements for a hierarchical system such as GStreamer
(for the other, read up on GstBin). Three functions are provided:
_set_parent(), _get_parent(), and _unparent(). The third is required because
there is an explicit check in _set_parent(): an object must not already have a
GstPad (inside an element)
-Refcounting
------------
-- GObject refcount is not threadsafe. This will be changed in the future.
- GStreamer for now sets it to a constant value on each _ref/_unref()
- and uses an atomic int "refcount" instead for threadsafe refcounting
- This implies you should always use gst_object_ref() and gst_object_unref() !
-
-
Naming
------
- names of objects cannot be changed when they are parented
Each object in the GStreamer object hierarchy can have flags associated with it,
which are used to describe a state or a feature of the object.
-GstObject has flags to mark its lifecycle: FLOATING, DISPOSING and DESTROYED.
+GstObject has flags to mark its lifecycle: FLOATING and DISPOSING.
Class signals
state change.
- PAUSED -> PLAYING:
- - calculate the running_time when the pipeline was stopped.
+ - calculate the running_time when the pipeline was PAUSED.
- READY -> NULL:
- set the bus to flushing (when auto-flushing is enabled)
The running_time represents the total elapsed time, measured in clock units,
that the pipeline spent in the PLAYING state (see part-synchronisation.txt).
+The running_time is set to 0 after a flushing seek.
Clock selection
An internal message used to instruct a pipeline hierarchy that a state
recalculation must be performed because of an ASYNC state change completed.
+ This message is not used anymore.
GST_MESSAGE_STEP_DONE:
GST_MESSAGE_APPLICATION:
- The application posted a message.
+ The application posted a message. This message must be used when the
+ application posts a message on the bus.
GST_MESSAGE_ELEMENT:
After receiving a buffer (or EOS) on a pad the chain/event function should
wait to render the buffers or in the EOS case, wait to post the EOS
-message.
+message. While waiting, the sink will wait for the preroll cond to be signalled.
-Several things can happen that require the preroll lock to be unlocked. This
+Several things can happen that require the preroll cond to be signalled. This
include state changes or flush events. The prerolling is implemented in
sinks (see part-element-sink.txt)
The GstBin sets the state of all the sink elements. These are the elements
without source pads.
-Setting the state on an element will first activate all the sinkpads. For
-each of the sinkpads, gst_pad_check_pull_range() is performed. If the
-sinkpad supports a loopfunction and the peer pad returns TRUE from the
-GstPadCheckPullRange function, then the peer pad is activated first as
+Setting the state on an element will first activate all the srcpads and then
+the sinkpads. For each of the sinkpads, gst_pad_check_pull_range() is
+performed. If the sinkpad supports a loopfunction and the peer pad returns TRUE
+from the GstPadCheckPullRange function, then the peer pad is activated first as
it must be in the right state to handle a _pull_range(). Note that the
state change of the element is not yet performed, just the activate function
is called on the source pad. This means that elements that implement a
QoS event
---------
-The QoS event travels upstream and contains the following fields:
+The QoS event is generated by an element that synchronizes against the clock. It
+travels upstream and contains the following fields:
- timestamp: The timestamp on the buffer that generated the QoS
event. These timestamps are expressed in total running_time in
In case of a positive jitter we can therefore send a QoS message with
a timestamp B1, jitter J1 and proportion given by (4).
-This allows an upstream element to not generate any data with a timestamps
+This allows an upstream element to not generate any data with timestamps
B2 < T1, where the element can derive T1 as B1 + J1.
This will effectively result in frame drops.
- upstream running average of processing time (5) in stream time.
- running average of buffer durations.
- - upstream running average of processing time in system time.
- running average of render time (in system time)
- rendered/dropped buffers
element.
some transforms can reduce the complexity of their algorithms. Depending on the
-algorith, the changes in quality may have disturbing visual or audible effect
+algorithm, the changes in quality may have disturbing visual or audible effect
that should be avoided.
Proposition
- - define GstQuery extending GstData and containing a GstStructure (see GstMessage)
+ - define GstQuery extending GstMiniObject and containing a GstStructure (see GstMessage)
- define standard query types (see proposed types)
- define methods to create a parse the results in the GstQuery.
- define pad method:
Deciding the scheduling mode
----------------------------
-When the core performs the pad activate function, it will select a scheduling mode
-for the pads. Sinkpads that expose a loop function are prefered over source pads
-with a loop function so that the pull mode is selected when possible. Selecting the
-pull mode is more efficient because it allows for arbitrary seeking and random access
-to the data.
+When tha pad is activated, the _activate() function is called. The pad can then
+choose to activate itself in push or pull mode depending on upstream
+capabilities.
+
+The GStreamer core will by default activate pads in push mode when there is no
+activate function for the pad.
The chain function
------------------
or samples.
Seeking can be performed to an absolute position or relative to the
-current configured segment.
+currently configured segment.
For seeking to work reliably, all plugins in the pipeline need to follow
the well-defined rules in this document.
followed by the previously decoded P frame. If the P frame is outside of the
segment, the decoder knows it should not send the P frame.
- Avidemux stops sending data after pushing a frame with timestamp 5 and pushes
- an EOS event downstream to finish playback.
+ Avidemux stops sending data after pushing a frame with timestamp 5 and
+ returns GST_FLOW_UNEXPECTED from the chain function to make the upstream
+ elements perform the EOS logic.
Use case: live stream
- READY: The element should be prepared to go to PAUSED.
- PAUSED: The element should be ready to accept and process data. Sink
elements however only accept one buffer and then block.
- - PLAYING: The same as PAUSED except for sinks, who are now accepting
- and rendering data.
+ - PLAYING: The same as PAUSED except for live sources and sinks. Sinks accept
+ and rendering data. Live sources produce data.
We call the sequence NULL->PLAYING an upwards state change and PLAYING->NULL
a downwards state change.
projects / platform / upstream / gstreamer.git / commitdiff