2 title: Different scheduling modes
5 # Different scheduling modes
7 The scheduling mode of a pad defines how data is retrieved from (source)
8 or given to (sink) pads. GStreamer can operate in two scheduling mode,
9 called push- and pull-mode. GStreamer supports elements with pads in any
10 of the scheduling modes where not all pads need to be operating in the
13 So far, we have only discussed `_chain ()`-operating elements, i.e.
14 elements that have a chain-function set on their sink pad and push
15 buffers on their source pad(s). We call this the push-mode because a
16 peer element will use `gst_pad_push ()` on a srcpad, which will cause
17 our `_chain ()`-function to be called, which in turn causes our element
18 to push out a buffer on the source pad. The initiative to start the
19 dataflow happens somewhere upstream when it pushes out a buffer and all
20 downstream elements get scheduled when their `_chain ()`-functions are
23 Before we explain pull-mode scheduling, let's first understand how the
24 different scheduling modes are selected and activated on a pad.
26 ## The pad activation stage
28 During the element state change of READY-\>PAUSED, the pads of an
29 element will be activated. This happens first on the source pads and
30 then on the sink pads of the element. GStreamer calls the `_activate ()`
31 of a pad. By default this function will activate the pad in push-mode by
32 calling `gst_pad_activate_mode ()` with the GST\_PAD\_MODE\_PUSH
33 scheduling mode. It is possible to override the `_activate ()` of a pad
34 and decide on a different scheduling mode. You can know in what
35 scheduling mode a pad is activated by overriding the `_activate_mode
38 GStreamer allows the different pads of an element to operate in
39 different scheduling modes. This allows for many different possible
40 use-cases. What follows is an overview of some typical use-cases.
42 - If all pads of an element are activated in push-mode scheduling, the
43 element as a whole is operating in push-mode. For source elements
44 this means that they will have to start a task that pushes out
45 buffers on the source pad to the downstream elements. Downstream
46 elements will have data pushed to them by upstream elements using
47 the sinkpads `_chain ()`-function which will push out buffers on the
48 source pads. Prerequisites for this scheduling mode are that a
49 chain-function was set for each sinkpad using
50 `gst_pad_set_chain_function ()` and that all downstream elements
51 operate in the same mode.
53 - Alternatively, sinkpads can be the driving force behind a pipeline
54 by operating in pull-mode, while the sourcepads of the element still
55 operate in push-mode. In order to be the driving force, those pads
56 start a `GstTask` when they are activated. This task is a thread,
57 which will call a function specified by the element. When called,
58 this function will have random data access (through
59 `gst_pad_pull_range ()`) over all sinkpads, and can push data over
60 the sourcepads, which effectively means that this element controls
61 data flow in the pipeline. Prerequisites for this mode are that all
62 downstream elements can act in push mode, and that all upstream
63 elements operate in pull-mode (see below).
65 Source pads can be activated in PULL mode by a downstream element
66 when they return GST\_PAD\_MODE\_PULL from the
67 GST\_QUERY\_SCHEDULING query. Prerequisites for this scheduling mode
68 are that a getrange-function was set for the source pad using
69 `gst_pad_set_getrange_function ()`.
71 - Lastly, all pads in an element can be activated in PULL-mode.
72 However, contrary to the above, this does not mean that they start a
73 task on their own. Rather, it means that they are pull slave for the
74 downstream element, and have to provide random data access to it
75 from their `_get_range ()`-function. Requirements are that the a
77 ()`-function was set on this pad using the function
78 `gst_pad_set_getrange_function ()`. Also, if the element has any
79 sinkpads, all those pads (and thereby their peers) need to operate
80 in PULL access mode, too.
82 When a sink element is activated in PULL mode, it should start a
83 task that calls `gst_pad_pull_range ()` on its sinkpad. It can only
84 do this when the upstream SCHEDULING query returns support for the
85 GST\_PAD\_MODE\_PULL scheduling mode.
87 In the next two sections, we will go closer into pull-mode scheduling
88 (elements/pads driving the pipeline, and elements/pads providing random
89 access), and some specific use cases will be given.
91 ## Pads driving the pipeline
93 Sinkpads operating in pull-mode, with the sourcepads operating in
94 push-mode (or it has no sourcepads when it is a sink), can start a task
95 that will drive the pipeline data flow. Within this task function, you
96 have random access over all of the sinkpads, and push data over the
97 sourcepads. This can come in useful for several different kinds of
100 - Demuxers, parsers and certain kinds of decoders where data comes in
101 unparsed (such as MPEG-audio or video streams), since those will
102 prefer byte-exact (random) access from their input. If possible,
103 however, such elements should be prepared to operate in push-mode
106 - Certain kind of audio outputs, which require control over their
107 input data flow, such as the Jack sound server.
109 First you need to perform a SCHEDULING query to check if the upstream
110 element(s) support pull-mode scheduling. If that is possible, you can
111 activate the sinkpad in pull-mode. Inside the activate\_mode function
112 you can then start the task.
118 static gboolean gst_my_filter_activate (GstPad * pad,
120 static gboolean gst_my_filter_activate_mode (GstPad * pad,
124 static void gst_my_filter_loop (GstMyFilter * filter);
126 G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
130 gst_my_filter_init (GstMyFilter * filter)
135 gst_pad_set_activate_function (filter->sinkpad, gst_my_filter_activate);
136 gst_pad_set_activatemode_function (filter->sinkpad,
137 gst_my_filter_activate_mode);
146 gst_my_filter_activate (GstPad * pad, GstObject * parent)
151 /* first check what upstream scheduling is supported */
152 query = gst_query_new_scheduling ();
154 if (!gst_pad_peer_query (pad, query)) {
155 gst_query_unref (query);
159 /* see if pull-mode is supported */
160 pull_mode = gst_query_has_scheduling_mode_with_flags (query,
161 GST_PAD_MODE_PULL, GST_SCHEDULING_FLAG_SEEKABLE);
162 gst_query_unref (query);
167 /* now we can activate in pull-mode. GStreamer will also
168 * activate the upstream peer in pull-mode */
169 return gst_pad_activate_mode (pad, GST_PAD_MODE_PULL, TRUE);
173 /* something not right, we fallback to push-mode */
174 return gst_pad_activate_mode (pad, GST_PAD_MODE_PUSH, TRUE);
179 gst_my_filter_activate_pull (GstPad * pad,
185 GstMyFilter *filter = GST_MY_FILTER (parent);
188 case GST_PAD_MODE_PUSH:
191 case GST_PAD_MODE_PULL:
194 res = gst_pad_start_task (pad,
195 (GstTaskFunction) gst_my_filter_loop, filter, NULL);
197 res = gst_pad_stop_task (pad);
201 /* unknown scheduling mode */
210 Once started, your task has full control over input and output. The most
211 simple case of a task function is one that reads input and pushes that
212 over its source pad. It's not all that useful, but provides some more
213 flexibility than the old push-mode case that we've been looking at so
217 #define BLOCKSIZE 2048
220 gst_my_filter_loop (GstMyFilter * filter)
224 GstFormat fmt = GST_FORMAT_BYTES;
225 GstBuffer *buf = NULL;
227 if (!gst_pad_query_duration (filter->sinkpad, fmt, &len)) {
228 GST_DEBUG_OBJECT (filter, "failed to query duration, pausing");
232 if (filter->offset >= len) {
233 GST_DEBUG_OBJECT (filter, "at end of input, sending EOS, pausing");
234 gst_pad_push_event (filter->srcpad, gst_event_new_eos ());
238 /* now, read BLOCKSIZE bytes from byte offset filter->offset */
239 ret = gst_pad_pull_range (filter->sinkpad, filter->offset,
242 if (ret != GST_FLOW_OK) {
243 GST_DEBUG_OBJECT (filter, "pull_range failed: %s", gst_flow_get_name (ret));
247 /* now push buffer downstream */
248 ret = gst_pad_push (filter->srcpad, buf);
250 buf = NULL; /* gst_pad_push() took ownership of buffer */
252 if (ret != GST_FLOW_OK) {
253 GST_DEBUG_OBJECT (filter, "pad_push failed: %s", gst_flow_get_name (ret));
257 /* everything is fine, increase offset and wait for us to be called again */
258 filter->offset += BLOCKSIZE;
262 GST_DEBUG_OBJECT (filter, "pausing task");
263 gst_pad_pause_task (filter->sinkpad);
267 ## Providing random access
269 In the previous section, we have talked about how elements (or pads)
270 that are activated to drive the pipeline using their own task, must use
271 pull-mode scheduling on their sinkpads. This means that all pads linked
272 to those pads need to be activated in pull-mode. Source pads activated
273 in pull-mode must implement a `_get_range ()`-function set using
274 `gst_pad_set_getrange_function ()`, and that function will be called
275 when the peer pad requests some data with `gst_pad_pull_range ()`. The
276 element is then responsible for seeking to the right offset and
277 providing the requested data. Several elements can implement random
280 - Data sources, such as a file source, that can provide data from any
281 offset with reasonable low latency.
283 - Filters that would like to provide a pull-mode scheduling over the
286 - Parsers who can easily provide this by skipping a small part of
287 their input and are thus essentially "forwarding" getrange requests
288 literally without any own processing involved. Examples include tag
289 readers (e.g. ID3) or single output parsers, such as a WAVE parser.
291 The following example will show how a `_get_range
292 ()`-function can be implemented in a source element:
296 gst_my_filter_get_range (GstPad * pad,
302 G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
307 gst_my_filter_init (GstMyFilter * filter)
312 gst_pad_set_getrange_function (filter->srcpad,
313 gst_my_filter_get_range);
319 gst_my_filter_get_range (GstPad * pad,
326 GstMyFilter *filter = GST_MY_FILTER (parent);
328 [.. here, you would fill *buf ..]
333 In practice, many elements that could theoretically do random access,
334 may in practice often be activated in push-mode scheduling anyway, since
335 there is no downstream element able to start its own task. Therefore, in
336 practice, those elements should implement both a `_get_range
337 ()`-function and a `_chain
338 ()`-function (for filters and parsers) or a `_get_range
339 ()`-function and be prepared to start their own task by providing
340 `_activate_* ()`-functions (for source elements).