The QoS event is generated by an element that synchronizes against the
clock. It travels upstream and contains the following fields:
-* **`type`**: `GST\_TYPE\_QOS\_TYPE:` The type of the QoS event, we have the
-following types and the default type is `GST\_QOS\_TYPE\_UNDERFLOW`:
+* **`type`**: `GST_TYPE_QOS_TYPE:` The type of the QoS event, we have the
+following types and the default type is `GST_QOS_TYPE_UNDERFLOW`:
- * GST_QOS_TYPE_OVERFLOW: an element is receiving buffers too fast and can't
+ * `GST_QOS_TYPE_OVERFLOW`: an element is receiving buffers too fast and can't
keep up processing them. Upstream should reduce the rate.
- * GST_QOS_TYPE_UNDERFLOW: an element is receiving buffers too slowly
+ * `GST_QOS_TYPE_UNDERFLOW`: an element is receiving buffers too slowly
and has to drop them because they are too late. Upstream should
increase the processing rate.
- * GST_QOS_TYPE_THROTTLE: the application is asking to add extra delay
+ * `GST_QOS_TYPE_THROTTLE`: the application is asking to add extra delay
between buffers, upstream is allowed to drop buffers
-* **`timestamp`**: G\_TYPE\_UINT64: The timestamp on the buffer that
+* **`timestamp`**: `G_TYPE_UINT64`: The timestamp on the buffer that
generated the QoS event. These timestamps are expressed in total
-running\_time in the sink so that the value is ever increasing.
+`running_time` in the sink so that the value is ever increasing.
-* **`jitter`**: G\_TYPE\_INT64: The difference of that timestamp against the
+* **`jitter`**: `G_TYPE_INT64`: The difference of that timestamp against the
current clock time. Negative values mean the timestamp was on time.
Positive values indicate the timestamp was late by that amount. When
buffers are received in time and throttling is not enabled, the QoS
the throttling delay added by the application and the type is set to
THROTTLE.
-* **`proportion`**: G\_TYPE\_DOUBLE: Long term prediction of the ideal rate
+* **`proportion`**: `G_TYPE_DOUBLE`: Long term prediction of the ideal rate
relative to normal rate to get optimal quality.
The rest of this document deals with how these values can be calculated
that performs QoS because of QOS events received from a downstream
element (\!live).
-The `GST\_MESSAGE\_QOS` contains at least the following info:
+The `GST_MESSAGE_QOS` contains at least the following info:
-* **`live`**: G\_TYPE\_BOOLEAN: If the QoS message was dropped by a live
+* **`live`**: `G_TYPE_BOOLEAN`: If the QoS message was dropped by a live
element such as a sink or a live source. If the live property is
FALSE, the QoS message was generated as a response to a QoS event in
a non-live element.
-* **`running-time`**: G\_TYPE\_UINT64: The running\_time of the buffer that
+* **`running-time`**: `G_TYPE_UINT64`: The `running_time` of the buffer that
generated the QoS message.
-* **`stream-time`**: G\_TYPE\_UINT64: The stream\_time of the buffer that
+* **`stream-time`**: `G_TYPE_UINT64`: The `stream_time` of the buffer that
generated the QoS message.
-* **`timestamp`**: G\_TYPE\_UINT64: The timestamp of the buffer that
+* **`timestamp`**: `G_TYPE_UINT64`: The timestamp of the buffer that
generated the QoS message.
-* **`duration`**: G\_TYPE\_UINT64: The duration of the buffer that generated
+* **`duration`**: `G_TYPE_UINT64`: The duration of the buffer that generated
the QoS message.
-* **`jitter`**: G\_TYPE\_INT64: The difference of the running-time against
+* **`jitter`**: `G_TYPE_INT64`: The difference of the running-time against
the deadline. Negative values mean the timestamp was on time.
Positive values indicate the timestamp was late (and dropped) by
-that amount. The deadline can be a realtime running\_time or an
-estimated running\_time.
+that amount. The deadline can be a realtime `running_time` or an
+estimated `running_time`.
-* **`proportion`**: G\_TYPE\_DOUBLE: Long term prediction of the ideal rate
+* **`proportion`**: `G_TYPE_DOUBLE`: Long term prediction of the ideal rate
relative to normal rate to get optimal quality.
-* **`quality`**: G\_TYPE\_INT: An element dependent integer value that
+* **`quality`**: `G_TYPE_INT`: An element dependent integer value that
specifies the current quality level of the element. The default
maximum quality is 1000000.
-* **`format`**: GST\_TYPE\_FORMAT Units of the *processed* and *dropped*
-fields. Video sinks and video filters will use GST\_FORMAT\_BUFFERS
+* **`format`**: `GST_TYPE_FORMAT` Units of the *processed* and *dropped*
+fields. Video sinks and video filters will use `GST_FORMAT_BUFFERS`
(frames). Audio sinks and audio filters will likely use
-GST\_FORMAT\_DEFAULT (samples).
+`GST_FORMAT_DEFAULT` (samples).
-* **`processed`**: G\_TYPE\_UINT64: Total number of units correctly
+* **`processed`**: `G_TYPE_UINT64`: Total number of units correctly
processed since the last state change to READY or a flushing
operation.
-* **`dropped`**: G\_TYPE\_UINT64: Total number of units dropped since the
+* **`dropped`**: `G_TYPE_UINT64`: Total number of units dropped since the
last state change to READY or a flushing operation.
The *running-time* and *processed* fields can be used to estimate the
Where CT is the clock time when the entry arrives in the sink. This
value is calculated inside the clock when we perform
-`gst\_clock\_id\_wait()`.
+`gst_clock_id_wait()`.
If the jitter is negative, the entry arrived in time and can be rendered
after waiting for the clock to reach time B1 (which is also CT - J1).
Using the jitter we can calculate the time when the buffer arrived in
the sink:
+```
T1 = B1 + J1. (1)
+```
The time the buffer leaves the sink after synchronisation is measured
as:
+```
T2 = B1 + (J1 < 0 ? 0 : J1) (2)
+```
For buffers that arrive in time (J1 \< 0) the buffer leaves after
synchronisation which is exactly B1. Late buffers (J1 \>= 0) leave the
-sink when they arrive, whithout any synchronisation, which is T2 = T1 =
-B1 + J1.
+sink when they arrive, whithout any synchronisation, which is `T2 = T1 =
+B1 + J1`.
Using a previous T0 and a new T1, we can calculate the time it took for
upstream to generate a buffer with timestamp B1.
+```
PT1 = T1 - T0 (3)
+```
We call PT1 the processing time needed to generate buffer with timestamp
B1.
when it receives too much data it cannot process in time. The element is
then said to be overflowed.
-# Short term correction
+## Short term correction
The timestamp and jitter serve as short term correction information for
upstream elements. Indeed, given arrival time T1 as given in (1) we can
B2 = B1 + 2 * J1 + D1 (5)
```
-# Long term correction
+## Long term correction
The datarate used to calculate (5) for the short term prediction is
based on a single observation. A more accurate datarate can be obtained
given by the proportion member. Failure to do so will certainly lead to
more dropped frames and a generally worse QoS.
-# Throttling
+## Throttling
In throttle mode, the time distance between buffers is kept to a
configurable throttle interval. This means that effectively the buffer
type, the proportion and the jitter member to tune their
implementations.
-# QoS strategies
+## QoS strategies
Several strategies exist to reduce processing delay that might affect
real time performance.
- assign more CPU(s) to critical pipeline parts
-# QoS implementations
+## QoS implementations
Here follows a small overview of how QoS can be implemented in a range
of different types of elements.
-# GstBaseSink
+### GstBaseSink
The primary implementor of QoS is GstBaseSink. It will calculate the
following values:
A QoS message is posted whenever a (part of a) buffer is dropped.
In throttle mode, the sink sends QoS event upstream with the timestamp
-set to the running\_time of the latest buffer and the jitter set to the
+set to the `running_time` of the latest buffer and the jitter set to the
throttle interval. If the throttled buffer is late, the lateness is
subtracted from the throttle interval in order to keep the desired
throttle interval.
-# GstBaseTransform
+### GstBaseTransform
Transform elements can entirely skip the transform based on the
timestamp and jitter values of recent QoS event since these buffers will
quality of the filter is reduced. The quality member in the QOS message
should reflect the quality setting of the filter.
-# Video Decoders
+### Video Decoders
A video decoder can, based on the codec in use, decide to not decode
intermediate frames. A typical codec can for example skip the decoding
- skipping P/B frames: quality = 55555 (for I-frame spacing of 18
frames)
-# Demuxers
+### Demuxers
Demuxers usually cannot do a lot regarding QoS except for skipping
frames to the next keyframe when a lateness QoS event arrives on a
QoS events on all the pads and derive an aggregated QoS event for the
upstream element.
-# Sources
+### Sources
The QoS events only apply to push based sources since pull based sources
are entirely controlled by another downstream element.
projects / platform / upstream / gstreamer.git / commitdiff