From 46d2c100f8aa9cf7948c2b2dd6aa1eb5c8d1ad81 Mon Sep 17 00:00:00 2001 From: Wim Taymans Date: Thu, 15 Feb 2007 11:32:02 +0000 Subject: [PATCH] docs/design/: Some doc updates. Start renaming from stream_time to running_time where it was used wrongly. Original commit message from CVS: * docs/design/part-TODO.txt: * docs/design/part-activation.txt: * docs/design/part-block.txt: * docs/design/part-buffering.txt: * docs/design/part-clocks.txt: * docs/design/part-element-source.txt: * docs/design/part-events.txt: * docs/design/part-gstbin.txt: * docs/design/part-gstbus.txt: * docs/design/part-gstpipeline.txt: * docs/design/part-live-source.txt: * docs/design/part-messages.txt: * docs/design/part-overview.txt: * docs/design/part-qos.txt: * docs/design/part-query.txt: * docs/design/part-states.txt: * docs/design/part-trickmodes.txt: Some doc updates. Start renaming from stream_time to running_time where it was used wrongly. --- ChangeLog | 22 ++++++++++++ docs/design/part-TODO.txt | 6 ++-- docs/design/part-activation.txt | 16 ++++++--- docs/design/part-block.txt | 15 +++++++- docs/design/part-buffering.txt | 5 +++ docs/design/part-clocks.txt | 50 +++++++++++++++++--------- docs/design/part-element-source.txt | 50 +++++++++++++++++++++++--- docs/design/part-events.txt | 72 ++++++++++++++++++++++++------------- docs/design/part-gstbin.txt | 16 +++++++-- docs/design/part-gstbus.txt | 3 +- docs/design/part-gstpipeline.txt | 36 ++++++------------- docs/design/part-live-source.txt | 24 ++++--------- docs/design/part-messages.txt | 34 +++++++++++++++--- docs/design/part-overview.txt | 23 ++++++------ docs/design/part-qos.txt | 8 ++--- docs/design/part-query.txt | 36 +++++++++---------- docs/design/part-states.txt | 12 +++---- docs/design/part-trickmodes.txt | 2 +- 18 files changed, 288 insertions(+), 142 deletions(-) diff --git a/ChangeLog b/ChangeLog index 778320a..c939e62 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,27 @@ 2007-02-15 Wim Taymans + * docs/design/part-TODO.txt: + * docs/design/part-activation.txt: + * docs/design/part-block.txt: + * docs/design/part-buffering.txt: + * docs/design/part-clocks.txt: + * docs/design/part-element-source.txt: + * docs/design/part-events.txt: + * docs/design/part-gstbin.txt: + * docs/design/part-gstbus.txt: + * docs/design/part-gstpipeline.txt: + * docs/design/part-live-source.txt: + * docs/design/part-messages.txt: + * docs/design/part-overview.txt: + * docs/design/part-qos.txt: + * docs/design/part-query.txt: + * docs/design/part-states.txt: + * docs/design/part-trickmodes.txt: + Some doc updates. Start renaming from stream_time to running_time where + it was used wrongly. + +2007-02-15 Wim Taymans + * libs/gst/base/gstbasesrc.c: (gst_base_src_default_query): Answer LATENCY query. diff --git a/docs/design/part-TODO.txt b/docs/design/part-TODO.txt index cdcb404..01ec91a 100644 --- a/docs/design/part-TODO.txt +++ b/docs/design/part-TODO.txt @@ -22,11 +22,13 @@ API/ABI - use | instead of + as divider in serialization of Flags (gstvalue/gststructure) +- with the addition the PREROLLED message, we can probably get rid of the + STATE_DIRTY message. + + IMPLEMENTATION -------------- -- implement latency calculation for live sources. - - implement more QOS, see part-qos.txt. - implement BUFFERSIZE. diff --git a/docs/design/part-activation.txt b/docs/design/part-activation.txt index f2aa75c..4255055 100644 --- a/docs/design/part-activation.txt +++ b/docs/design/part-activation.txt @@ -6,6 +6,10 @@ sink-to-source order. As elements undergo the READY->PAUSED transition, their pads are activated so as to prepare for data flow. Some pads will start tasks to drive the data flow. +An element activates its pads from sourcepads to sinkpads. This to make +sure that when the sinkpads are activated an ready to accept data, the +sourcepads are already active to pass the data downstream. + Pads can be activated in one of two modes, PUSH and PULL. PUSH pads are the normal case, where the source pad in a link sends data to the sink pad via gst_pad_push(). PULL pads instead have sink pads request data @@ -71,12 +75,14 @@ pushing data. Deactivation ------------ -Pad deactivation occurs when a pad is unlinked, or when its parent goes -into the READY state. gst_pad_set_active() is called with a FALSE -argument, which then calls activate_push() or activate_pull() with a -FALSE argument, depending on the activation mode of the pad. +Pad deactivation occurs when its parent goes into the READY state or when the +pad is deactivated explicitly by the application or element. +gst_pad_set_active() is called with a FALSE argument, which then calls +activate_push() or activate_pull() with a FALSE argument, depending on the +activation mode of the pad. Mode switching -------------- -Changing from push to pull modes needs a bit of thought. +Changing from push to pull modes needs a bit of thought. This is actually +possible and implemented but not yet documented here. diff --git a/docs/design/part-block.txt b/docs/design/part-block.txt index b4cc52f..a30b1c6 100644 --- a/docs/design/part-block.txt +++ b/docs/design/part-block.txt @@ -137,7 +137,7 @@ Use cases: 5a) optionally element2 can now be set to NULL. 6) link element4 and element3 7) link element1 and element4 (FIXME, how about letting element4 know - about the currently running segment?) + about the currently running segment?, see issues.) 8) unblock element1 src The same flow can be used to replace an element in a PAUSED pipeline. Only @@ -150,5 +150,18 @@ Use cases: be implemented with a helper function in the future. +Issues +------ +When an EOS event has passed a pad and the pad is set to blocked, the block will +never happen because no data is going to flow anymore. One possibility is to +keep track of the pad's EOS state and make the block succeed immediatly. This is +not yet implemenented. + +When dynamically reconnecting pads, some events (like NEWSEGMENT, EOS, +TAGS, ...) are not yet retransmitted to the newly connected element. It's +unclear if this can be done by core automatically by caching those events and +resending them on a relink. It might also be possible that this needs a +GstFlowReturn value from the event function, in which case the implementation +must be delayed for after 0.11, when we can break API/ABI. diff --git a/docs/design/part-buffering.txt b/docs/design/part-buffering.txt index 622d211..67a4fe4 100644 --- a/docs/design/part-buffering.txt +++ b/docs/design/part-buffering.txt @@ -51,3 +51,8 @@ application should not set the pipeline to PLAYING before a BUFFERING message with 100 percent value is received, which might only happen after the pipeline prerolled. + +Incremental download +-------------------- + + diff --git a/docs/design/part-clocks.txt b/docs/design/part-clocks.txt index 486528e..c5e157c 100644 --- a/docs/design/part-clocks.txt +++ b/docs/design/part-clocks.txt @@ -1,13 +1,6 @@ Clocks ------ -To synchronize the different elements, the GstPipeline is responsible for -selecting and distributing a global GstClock for all the elements in it. - -This selection happens whenever an element is added or removed from the -pipeline. Whever the clock changes in a pipeline, a NEW_CLOCK message is -posted on the bus signaling the new clock to the application. - The GstClock returns a monotonically increasing time with the method _get_time(). Its accuracy and base time depends on the specific clock implementation but time is always expessed in nanoseconds. Since the @@ -17,28 +10,53 @@ times. The time reported by the clock is called the absolute time. +Clock Selection +--------------- + +To synchronize the different elements, the GstPipeline is responsible for +selecting and distributing a global GstClock for all the elements in it. + +This selection happens whenever the pipeline goes to PLAYING. Whenever an +element is added/removed from the pipeline, this selection will be redone in the +next state change to PLAYING. Adding an element that can provide a clock will +post a GST_MESSAGE_CLOCK_PROVIDE message on the bus to inform parent bins of the +fact that a clock recalculation is needed. + +When a clock is selected, a NEW_CLOCK message is posted on the bus signaling the +clock to the application. + +When the element that provided the clock is removed from the pipeline, a +CLOCK_LOST message is posted. The application must then set the pipeline to +PAUSED and PLAYING again in order to let the pipeline select a new clock +and distribute a new base time. + + Time in GStreamer ----------------- -The absolute time is used to calculate the stream time. The stream time +The absolute time is used to calculate the running time. The running time is defined as follows: - - If the pipeline is NULL/READY, the stream time is undefined. - - In PAUSED, the stream time remains at the time when it was last - PAUSED. When the stream is PAUSED for the first time, the stream time + - If the pipeline is NULL/READY, the running time is undefined. + - In PAUSED, the running time remains at the time when it was last + PAUSED. When the stream is PAUSED for the first time, the running time is 0. - - In PLAYING, the stream time is the delta between the absolute time + - In PLAYING, the running time is the delta between the absolute time and the base time. The base time is defined as the absolute time minus - the stream time at the time when the pipeline is set to PLAYING. - - after a seek, the stream time is set to 0 (see part-seeking.txt) + the running time at the time when the pipeline is set to PLAYING. + - after a flushing seek, the running time is set to 0 (see part-seeking.txt) -The stream time is completely managed by the GstPipeline object using the -GstClock absolute time. +The running time is completely managed by the GstPipeline object using the +GstClock absolute time. It basically represents the elapsed time that the +pipeline spend in the PLAYING state. Timestamps ---------- +NOTE: this info is inaccurate, see part-synchronisation.txt for how the clock +time is used to synchronize buffer timestamps. + The combination of the last NEWSEGMENT event and the buffer timestamps express the presentation stream time of the buffer. The stream time of a buffer is calculated as follows: diff --git a/docs/design/part-element-source.txt b/docs/design/part-element-source.txt index fc929de..346d59a 100644 --- a/docs/design/part-element-source.txt +++ b/docs/design/part-element-source.txt @@ -7,10 +7,53 @@ does typically not have any sink (input) pads. Typical source elements include: - file readers - - network elements + - network elements (live or not) - capture elements (video/audio/...) - generators (signals/video/audio/...) + +Live sources +------------ + +A source is said to be a live source when it has the following property: + + * temporarily stopping reading from the source causes data to be lost. + +In general when this property holds, the source also produces data at a fixed +rate. Most sources have a limit to the rate at which they can deliver data, which +might be faster or slower than the consumption rate. this property however does +not make them a live source. + +Let's look at some example sources. + + - file readers: you can PAUSE without losing data. There is however a limit to + how fast you can read from this source. This limit is usually much higher + than the consumption rate. In some cases it might be slower (an NFS share, + for example) in which case you might need to use some buffering + (see part-buffering.txt). + + - http network element: you can PAUSE without data loss. Depending on the + available network bandwidth, consumption rate might be higher than production + rate in which case buffering should be used (see part-buffering.txt). + + - audio source: pausing the audio capture will lead to lost data. this source + is therefore definatly live. In addition, an audio source will produce data + at a fixed rate (the samplerate). Also depending on the buffersize, this + source will introduce a latency (see part-latency.txt). + + - udp network source: Pausing the receiving part will lead to lost data. This + source is therefore a live source. Also in a typical case the udp packets + will be received at a certain rate, which might be difficult to guess because + of network jitter. This source does not necessarily introduce latency on its + own. + + - dvb source: PAUSING this element will lead to data loss, it's a live source + similar to a UDP source. + + +Source types +------------ + A source element can operate in three ways: - it is fully seekable, this means that random access can be performed @@ -22,15 +65,14 @@ A source element can operate in three ways: A video source always provides the same amount of data (one video frame). Note that this is not a fully seekable source. - - it is a live source, this means that data arrives when it is ready. - An example of this is a video or network source. + - it is a live source, see above. When writing a source, one has to look at how the source can operate to decide on the scheduling methods to implement on the source. - fully seekable sources implement a getrange function on the source pad. - - sources that can give N bytes but cannot do seeking also implement a + - sources that can give N bytes but cannot do seeking also implement a getrange function but state that they cannot do random access. - sources that are purely live sources implement a task to push out diff --git a/docs/design/part-events.txt b/docs/design/part-events.txt index 6bb627c..8f2e64f 100644 --- a/docs/design/part-events.txt +++ b/docs/design/part-events.txt @@ -19,8 +19,12 @@ Different types of events exist to implement various functionalities. GST_EVENT_QOS: A notification of the quality of service of the stream GST_EVENT_SEEK: A seek should be performed to a new position in the stream GST_EVENT_NAVIGATION: A navigation event. - GST_EVENT_DRAIN: Play all data downstream before returning. + GST_EVENT_LATENCY: Configure the latency in a pipeline + * GST_EVENT_DRAIN: Play all data downstream before returning. + +* not yet implemented, under investigation, might be needed to do still frames + in DVD. FLUSH_START/STOP ---------------- @@ -53,6 +57,12 @@ For elements that use the pullrange function, they send both flush events to the upstream pads in the same way to make sure that the pullrange function unlocks and any pending buffers are cleared in the upstream elements. +A FLUSH_STOP event will also clear any configured synchronisation information +like NEWSEGMENT events. After a FLUSH_STOP, any element that performs +synchronisation to the clock will therefore need a NEWSEGMENT event (which makes +the running_time start from 0 again) and will therefore also need a new +base_time (see part-clocks.txt and part-synchronisation.txt). + EOS --- @@ -64,7 +74,7 @@ in the streaming thread but can also be sent from the application thread. An EOS event sent on a srcpad returns GST_FLOW_UNEXPECTED. The downstream element should forward the EOS event to its downstream peer -elements. This way the event will eventually reach the renderers which should +elements. This way the event will eventually reach the sinks which should then post an EOS message on the bus when in PLAYING. An element might want to flush its internally queued data before forwarding @@ -86,14 +96,14 @@ sending data. An element that sends EOS on a pad should stop sending data on that pad. Source elements typically pause() their task for that purpose. -By default, a GstBin collects all EOS events from all its sinks before +By default, a GstBin collects all EOS messages from all its sinks before posting the EOS message to its parent. The EOS is only posted on the bus by the sink elements in the PLAYING state. If the EOS event is received in the PAUSED state, it is queued until the element goes to PLAYING. -A seek event on an element flushes all pending EOS messages. +A FLUSH_STOP event on an element flushes the EOS state and all pending EOS messages. NEWSEGMENT @@ -148,56 +158,57 @@ BUFFERSIZE An element can suggest a buffersize for downstream elements. This is typically done by elements that produce data on multiple source pads -such as demuxers. +such as demuxers. This event is currently not yet defined or used. QOS --- A QOS, or quality of service message, is generated in an element to report -to the upstream elements about the current quality of the stream. This -is typically done by the sinks that measure the amount of framedrops they -have. (see part-qos.txt) +to the upstream elements about the current quality of real-time performance +the stream. This is typically done by the sinks that measure the amount of +framedrops they have. (see part-qos.txt) SEEK ---- -A seek event is issued by the application to start playback of a new -position in the stream. It is called form the application thread and -travels upstream. +A seek event is issued by the application to configure the playback range +of a stream. It is called form the application thread and travels upstream. -The seek event contains the new start and end position of playback -after the seek is performed. Optionally the end position can be left +The seek event contains the new start and stop position of playback +after the seek is performed. Optionally the stop position can be left at -1 to continue playback to the end of the stream. The seek event also contains the new playback rate of the stream, 1.0 is normal playback, 2.0 double speed and negative values mean backwards playback. A seek usually flushes the graph to minimize latency after the seek. This -behaviour is triggered by using the SEEK_FLUSH flag. +behaviour is triggered by using the SEEK_FLUSH flag on the seek event. -The seek event is passed along from element to element until it reaches -an element that can perform the seek. No intermediate element is allowed -to assume that a seek to this location will happen. It is allowed to -modify the start and stop times if it needs to do so. this is typically -the case if a seek is requested for a non-time position. +The seek event usually starts from the sink elements and travels upstream +from element to element until it reaches an element that can perform the +seek. No intermediate element is allowed to assume that a seek to this +location will happen. It is allowed to modify the start and stop times if it +needs to do so. this is typically the case if a seek is requested for a +non-time position. The actual seek is performed in the application thread so that success or failure can be reported as a return value of the seek event. It is therefore important that before executing the seek, the element acquires -the STREAM_LOCK so that the streaming thread and the seek gets serialized. +the STREAM_LOCK so that the streaming thread and the seek get serialized. The general flow of executing the seek with FLUSH is as follows: 1) unblock the streaming threads, they could be blocked in a chain - function. This is done by sending a FLUSH_START on all srcpads. + function. This is done by sending a FLUSH_START on all srcpads or by pausing + the streaming task, depending on the seek FLUSH flag. The flush will make sure that all downstream elements unlock and that control will return to this element chain/loop function. We cannot lock the STREAM_LOCK before doing this since it might cause a deadlock. 2) acquire the STREAM_LOCK. This will work since the chain/loop function - was unlocked in step 1). + was unlocked/paused in step 1). 3) perform the seek. since the STREAM_LOCK is held, the streaming thread will wait for the seek to complete. Most likely, the stream thread @@ -205,8 +216,8 @@ The general flow of executing the seek with FLUSH is as follows: 4) send a FLUSH_STOP event to all peer elements to allow streaming again. - 5) send a NEWSEGMENT event to signal the new buffer timestamp base time. - This can also be done from the streaming thread. + 5) create a NEWSEGMENT event to signal the new buffer timestamp base time. + This event must be queued to be send by the streaming thread. 6) start stopped tasks and unlock the STREAM_LOCK, dataflow will continue now from the new position. @@ -223,9 +234,22 @@ of a navigation event such as a mouse movement or button click. Navigation events travel upstream. +LATENCY +------- + +A latency event is used to configure a certain latency in the pipeline. It +contains a single GstClockTime with the required latency. The latency value is +calculated by the pipeline and distributed to all sink elements before they are +set to PLAYING. The sinks will add the configured latency value to the +timestamps of the buffer in order to delay their presentation. +See also part-latency.txt. + + DRAIN ----- +This event is not yet implemented. + Drain event indicates that upstream is about to perform a real-time event, such as pausing to present an interactive menu or such, and needs to wait for all data it has sent to be played-out in the sink. diff --git a/docs/design/part-gstbin.txt b/docs/design/part-gstbin.txt index 3ba0e57..b380cf3 100644 --- a/docs/design/part-gstbin.txt +++ b/docs/design/part-gstbin.txt @@ -65,8 +65,8 @@ When a bin goes to READY it will clear all cached messages. EOS --- -The sink elements will post an EOS event on the bus when they reach EOS. The -EOS message is only posted to the bus when the element is in PLAYING. +The sink elements will post an EOS message on the bus when they reach EOS. The +EOS message is only posted to the bus when the sink element is in PLAYING. The bin collects all EOS messages and forwards it to the application as soon as all the sinks have posted an EOS. @@ -89,6 +89,18 @@ The cached SEGMENT_START/STOP messages are cleared when going to READY. DURATION -------- +When a DURATION query is performed on a bin, it will forward the query to all +its sink elements. The bin will calculate the total duration as the MAX of all +returned durations and will then cache the result so that any further query can +use the cached version. The reason for caching the result is because the +duration of a stream typically does not change that often. + +A GST_MESSAGE_DURATION posted by an element will clear the cached duration value +so that the bin will query the sinks again. This message is typically posted by +elements that calculate the duration of the stream based on some average +bitrate, which might change while playing the stream. The DURATION message is +posted to the application, which can then fetch the updated DURATION. + Subclassing ----------- diff --git a/docs/design/part-gstbus.txt b/docs/design/part-gstbus.txt index e1e83ec..61c7306 100644 --- a/docs/design/part-gstbus.txt +++ b/docs/design/part-gstbus.txt @@ -7,7 +7,8 @@ a first-in first-out way from the streaming threads to the application. Since the application typically only wants to deal with delivery of these messages from one thread, the GstBus will marshall the messages between different threads. This is important since the actual streaming of media -is done in another thread than the application. +is done in another thread than the application. It is also important to not +block the streaming thread while the application deals with the message. The GstBus provides support for GSource based notifications. This makes it possible to handle the delivery in the glib mainloop. Different GSources diff --git a/docs/design/part-gstpipeline.txt b/docs/design/part-gstpipeline.txt index 7945929..8bb5c09 100644 --- a/docs/design/part-gstpipeline.txt +++ b/docs/design/part-gstpipeline.txt @@ -6,11 +6,9 @@ children with a clock. A GstPipeline also provides a toplevel GstBus (see part-gstbus.txt) -The pipeline also calculates the stream time based on the selected +The pipeline also calculates the running_time based on the selected clock (see part-clocks.txt). -The pipeline manages the seek operation for the application. - State changes ------------- @@ -22,26 +20,30 @@ GstBin, the pipeline performs the following actions during a state change: - set the bus to non-flushing - READY -> PAUSED: - - reset the stream time to 0 + - reset the running_time to 0 - PAUSED -> PLAYING: - Select and a clock. - - calculate base time using the stream time. - - set clock and base time on all elements before performing the + - calculate base_time using the running_time. + - set clock and base_time on all elements before performing the state change. - PAUSED -> PLAYING: - - calculate the stream time when the pipeline was stopped. + - calculate the running_time when the pipeline was stopped. - READY -> NULL: - set the bus to flushing (when auto-flushing is enabled) +The running_time represents the total elapsed time, measured in clock units, +that the pipeline spent in the PLAYING state. + Clock selection --------------- Since all of the children of a GstPipeline must use the same clock, the -pipeline must select a clock. +pipeline must select a clock. This clock selection happens when the pipeline +gopes to the PLAYING state. The default clock selection algorithm works as follows: @@ -54,7 +56,7 @@ The default clock selection algorithm works as follows: * since this selection procedure happens in the PAUSED->PLAYING state change, all the sinks are prerolled and we can thus be sure that each sink is linked to some upstream element. - * in the case of a live pipeline (NO_PREROLL), the sink will not + * in the case of a live pipeline (NO_PREROLL), the sink will not yet be prerolled and the selection process will select the clock of a more upstream element. @@ -72,19 +74,3 @@ possible. The _auto_clock() method removes the fixed clock and reactivates the auto- matic clock selection algorithm described above. - -Seeking -------- - -When performing a seek on the pipeline element using gst_element_send_event(), -the pipeline performs the following actions: - - - record the current state of the pipeline. - - set the pipeline to paused if a FLUSHING seek is requested - - send the seek event to all sinks - - when a FLUSH seek was performed successfully, the stream_time is set 0 again. - - restore old state of the pipeline. - - - - diff --git a/docs/design/part-live-source.txt b/docs/design/part-live-source.txt index ae198e3..ada1660 100644 --- a/docs/design/part-live-source.txt +++ b/docs/design/part-live-source.txt @@ -1,6 +1,9 @@ Live sources ------------ +A live source is a source that cannot be arbitrarily PAUSED without losing +data. + A live source such as an element capturing audio or video need to be handled in a special way. It does not make sense to start the dataflow in the PAUSED state for those devices as the user might wait a long time between going from @@ -26,7 +29,7 @@ a blocking wait. Scheduling ---------- -Live sources can not produce data in the paused state. They block in the +Live sources will not produce data in the paused state. They block in the getrange function or in the loop function until they go to PLAYING. @@ -39,19 +42,6 @@ the first sample of data and the last sample. This means that when the buffer arrives at the sink, it will already be late and will be dropped. The latency is the time it takes to construct one buffer of data. This latency -could be exposed by latency queries. - -Theses latency queries could to be done by the managing pipeline for all sinks. -In that case they can only be done after the meassurements have been taken (all -are prerolled). Thus in pipeline:state_changed:PAUSED_TO_PLAYING we need -get the max-latency and set this as a sync-offset in all sinks. The problem is -that in a live pipeline, we set the pipeline to PLAYING before waiting for the -sinks to preroll. - -Another possibility would be to configure a fixed latency in a pipeline that -would automatically be configured on any sink in the case of a NO_PREROLL -element. For decoupled elements this is practically the only viable way to -introduce enough latency that does not starve the sinks. - -The current latency can also be measured in the sinks and the pipeline could -optimize it to the lowest possible latency. +is exposed with a LATENCY query. + +See part-latency.txt. diff --git a/docs/design/part-messages.txt b/docs/design/part-messages.txt index c804be2..904f4da 100644 --- a/docs/design/part-messages.txt +++ b/docs/design/part-messages.txt @@ -55,9 +55,14 @@ Message types An element changed state in the pipeline. The message carries the old, new and pending state of the element. + GST_MESSAGE_STATE_DIRTY: + + An internal message used to instruct a pipeline hierarchy that a state + recalculation must be performed because of an ASYNC state change completed. + GST_MESSAGE_STEP_DONE: - An element stepping frames has finished. + An element stepping frames has finished. This is currently not used. GST_MESSAGE_CLOCK_PROVIDE: @@ -75,12 +80,15 @@ Message types GST_MESSAGE_STRUCTURE_CHANGE: The pipeline changed of structure, This means elements were added or removed or - pads were linked or unlinked. + pads were linked or unlinked. This messages is not yet used. GST_MESSAGE_STREAM_STATUS: - An element posted information about the stream it is handling. This could include - information about the length of the stream. + Posted by an element when it start/stop/pauses a streaming task. It + contains information about the reason why the stream state changed along + with the thread id. The application can use this information to detect + failures in streaming threads. It can also be used to adjust streaming + thread priorities by the application. GST_MESSAGE_APPLICATION: @@ -100,4 +108,22 @@ Message types An element or bin completed playback of a segment. This message is only posted on the bus if a SEGMENT seek is performed on a pipeline. + GST_MESSAGE_DURATION: + + An element posts this message when it has detected or updated the stream duration. + + GST_MESSAGE_LOST_PREROLL: + + Posted by sinks when they lose their preroll buffer in the PAUSED or PLAYING + state caused by a FLUSH_START event. + + GST_MESSAGE_PREROLLED: + + Posted by sinks when they receive the first data buffer. + + GST_MESSAGE_LATENCY: + + Posted by elements when the latency in a pipeline changed and a new global + latency should be calculated by the pipeline or application. + diff --git a/docs/design/part-overview.txt b/docs/design/part-overview.txt index 1232f14..5e700a3 100644 --- a/docs/design/part-overview.txt +++ b/docs/design/part-overview.txt @@ -47,7 +47,7 @@ Introduction Downstream and upstream are the terms used to describe the direction in the Pipeline. From source to sink is called "downstream" and "upstream" is - from sink to source. + from sink to source. Dataflow always happens downstream. The task of the application is to construct a pipeline as above using existing elements. This is further explained in the pipeline building topic. @@ -162,8 +162,10 @@ Pipeline A pipeline is a special bin subclass that provides the following features to its children: - - Select and manage a clock. - - Manage stream time based on the selected clock. + - Select and manage a global clock for all its children. + - Manage running_time based on the selected clock. Running_time is the elapsed + time the pipeline spent in the PLAYING state and is used for + synchronisation. - Provide means for elements to comunicate with the application by the GstBus. - Manage the global state of the elements such as Errors and end-of-stream. @@ -226,6 +228,7 @@ Dataflow and buffers The process of selecting a media type and attaching it to the buffers is called caps negotiation. + Caps ---- @@ -243,7 +246,7 @@ Dataflow and events ------------------- Parallel to the dataflow is a flow of events. Unlike the buffers, events can pass - upstream and downstream. Some events only travel upstream others only downstream. + both upstream and downstream. Some events only travel upstream others only downstream. the events are used to denote special conditions in the dataflow such as EOS or to inform plugins of special events such as flushing or seeking. @@ -311,13 +314,13 @@ Pipeline clock a sink element such as an audio sink. In capture pipelines, this will typically select the clock of the data producer, which - can in most cases not control the rate at which it produces data. + in most cases can not control the rate at which it produces data. Pipeline states --------------- - When all the pads are linked or signals have been connected, the pipeline can + When all the pads are linked and signals have been connected, the pipeline can be put in the PAUSED state to start dataflow. When a bin (and hence a pipeline) performs a state change, it will change the state @@ -361,7 +364,7 @@ Pipeline states the pipeline can be put in the PLAYING state. Before going to PLAYING, the pipeline select a clock and samples the current time of - the clock. This is the base time. It then distributes this time to all elements. + the clock. This is the base_time. It then distributes this time to all elements. Elements can then synchronize against the clock using the buffer timestamp+base time. The following chain of state changes then takes place: @@ -404,14 +407,14 @@ Pipeline EOS data after receiving an EOS event on a sinkpad. The element providing the streaming thread stops sending data after sending the - EOS message. + EOS event. - The EOS even will eventually arrive in the sink element. The sink will then post + The EOS event will eventually arrive in the sink element. The sink will then post an EOS message on the bus to inform the pipeline that a particular stream has finished. When all sinks have reported EOS, the pipeline forwards the EOS message to the application. - When in EOS, the pipeline remains in the playing state, it is the applications + When in EOS, the pipeline remains in the PLAYING state, it is the applications responsability to PAUSE or READY the pipeline. The application can also issue a seek, for example. diff --git a/docs/design/part-qos.txt b/docs/design/part-qos.txt index 91f16a8..2956ba0 100644 --- a/docs/design/part-qos.txt +++ b/docs/design/part-qos.txt @@ -31,7 +31,7 @@ QoS event The QoS event travels upstream and contains the following fields: - timestamp: The timestamp on the buffer that generated the QoS - event. These timestamps are expressed in total running time in + event. These timestamps are expressed in total running_time in the sink so that the value is ever increasing. - jitter: The difference of that timestamp against the current clock time. @@ -223,11 +223,11 @@ values: The processing time and the average buffer durations will be used to calculate a proportion. -the processing time in system time is compared to render time to decide if +The processing time in system time is compared to render time to decide if the majority of the time is spend upstream or in the sink itself. This value is used to decide flood or starvation. -the number of rendered and dropped buffers is used to query stats on the sink. +The number of rendered and dropped buffers is used to query stats on the sink. A QoS message with the most current values is sent upstream for each buffer that was received by the sink. @@ -293,5 +293,5 @@ less demanding source material. In case of a network stream, a switch could be d to a lower or higher quality stream or additional enhancement layers could be used or ignored. -Live sources will automatically drop data when it takes too long to prcess the data +Live sources will automatically drop data when it takes too long to process the data that the element pushes out. diff --git a/docs/design/part-query.txt b/docs/design/part-query.txt index 2e6b445..7163ebf 100644 --- a/docs/design/part-query.txt +++ b/docs/design/part-query.txt @@ -1,10 +1,6 @@ DRAFT Query ----------- -Status - - Implemented, move me to design... - Purpose Queries are used to get information about the stream. @@ -61,33 +57,33 @@ Proposed types - GST_QUERY_SEEKING: - - get info on how seeking can be done - - getrange, with/without offset/size - - ranges where seeking is efficient (for caching network sources) - - flags describing seeking behaviour (forward, backward, segments, + get info on how seeking can be done + - getrange, with/without offset/size + - ranges where seeking is efficient (for caching network sources) + - flags describing seeking behaviour (forward, backward, segments, play backwards, ...) - GST_QUERY_POSITION: - - get info on current position of the stream - - start position - - current position - - end position - - length + get info on current position of the stream in stream_time. + + - GST_QUERY_DURATION: + + get info on the total duration of the stream. - GST_QUERY_LATENCY: - - get amount of buffering + get amount of latency introduced in the pipeline. - - GST_QUERY_CONVERT: + - GST_QUERY_SEGMENT: - - convert format/value to another format/value pair. + get info about the currently configured playback segment. - - GST_QUERY_FORMATS: + - GST_QUERY_CONVERT: - - return list of supported formats. + convert format/value to another format/value pair. -Also???? + - GST_QUERY_FORMATS: - - GST_QUERY_CAPS: + return list of supported formats that can be used for GST_QUERY_CONVERT. diff --git a/docs/design/part-states.txt b/docs/design/part-states.txt index a53ff04..373b0ad 100644 --- a/docs/design/part-states.txt +++ b/docs/design/part-states.txt @@ -41,7 +41,7 @@ the following state changes are possible: when they have enough information. It is a requirement for sinks to return ASYNC and complete the state change when they receive the first buffer or EOS event (prerol). Sinks also block the dataflow when in PAUSED. - - a pipeline resets the stream time to 0. + - a pipeline resets the running_time to 0. - live sources return NO_PREROLL and don't generate data. PAUSED -> PLAYING @@ -49,8 +49,8 @@ the following state changes are possible: - The pipeline selects a clock and distributes this to all the children before setting them to PLAYING. This means that it is only alowed to synchronize on the clock in the PLAYING state. - - The pipeline uses the clock and the stream time to calculate the base time. - The base time is distributed to all children when performing the state + - The pipeline uses the clock and the running_time to calculate the base_time. + The base_time is distributed to all children when performing the state change. - sink elements stop blocking on the preroll buffer or event and start rendering the data. @@ -62,8 +62,8 @@ the following state changes are possible: PLAYING -> PAUSED - most elements ignore this state change. - - The pipeline calculates the stream time based on the last selected clock - and the base time. It stores this information to continue playback when + - The pipeline calculates the running_time based on the last selected clock + and the base_time. It stores this information to continue playback when going back to the PLAYING state. - sinks unblock any clock wait calls. - when a sink did not have a pending buffer to play, it returns ASYNC from @@ -202,7 +202,7 @@ and which were not affected by iterating the elements and calling _get_state() on the elements. If after calling the state function on all children, one of the children returned -ASYNC, the function returns ASYNC as well. +ASYNC, the function returns ASYNC as well. If after calling the state function on all children, one of the children returned NO_PREROLL, the function returns NO_PREROLL as well. diff --git a/docs/design/part-trickmodes.txt b/docs/design/part-trickmodes.txt index de09ad5..253550b 100644 --- a/docs/design/part-trickmodes.txt +++ b/docs/design/part-trickmodes.txt @@ -195,7 +195,7 @@ For plugins the following rules apply: Notes: - - The clock/stream_time keeps running forward. + - The clock/running_time keeps running forward. - backwards playback potentially uses a lot of memory as frames and undecoded data gets buffered. -- 2.7.4