1 <chapter id="chapter-scheduling" xreflabel="Different scheduling modes">
2 <title>Different scheduling modes</title>
4 Scheduling is, in short, a method for making sure that every element gets
5 called once in a while to process data and prepare data for the next
6 element. Likewise, a kernel has a scheduler to for processes, and your
7 brain is a very complex scheduler too in a way.
8 Randomly calling elements' chain functions won't bring us far, however, so
9 you'll understand that the schedulers in &GStreamer; are a bit more complex
10 than this. However, as a start, it's a nice picture.
13 So far, we have only discussed <function>_chain ()</function>-operating
14 elements, i.e. elements that have a chain-function set on their sinkpad
15 and push buffers on their sinkpad. Pads (or elements) can also operate
16 in two other scheduling modes, however. In this chapter, we will discuss
17 what those scheduling modes are, how they can be enabled and in what
18 cases they are useful. The other two scheduling modes are random access
19 (<function>_getrange ()</function>-based) or task-runner (which means
20 that this element is the driving force in the pipeline) mode.
23 <sect1 id="section-scheduling-activation"
24 xreflabel="The pad actication stage">
25 <title>The pad activation stage</title>
27 The stage in which &GStreamer; decides in what scheduling mode the
28 various elements will operate, is called the pad-activation stage. In
29 this stage, &GStreamer; will query the scheduling capabilities (i.e.
30 it will see in what modes each particular element/pad can operate) and
31 decide on the optimal scheduling composition for the pipeline. Next,
32 each pad will be notified of the scheduling mode that was assigned to
33 it, and after that the pipeline will start running.
36 Pads can be assigned one of three modes, each mode putting several
37 prerequisites on the pads. Pads should implement a notification
38 function (<function>gst_pad_set_activatepull_function ()</function> and
39 <function>gst_pad_set_activatepush_function ()</function>) to be
40 notified of the scheduling mode assignment. Also, sinkpads assigned
41 to do pull-based scheduling mode should start and stop their task
47 If all pads of an element are assigned to do
48 <quote>push</quote>-based scheduling, then this means that data
49 will be pushed by upstream elements to this element using the
50 sinkpads <function>_chain ()</function>-function. Prerequisites
51 for this scheduling mode are that a chain-function was set for
52 each sinkpad using<function>gst_pad_set_chain_function ()</function>
53 and that all downstream elements operate in the same mode. Pads are
54 assigned to do push-based scheduling in sink-to-source element
55 order, and within an element first sourcepads and then sinkpads.
56 Sink elements can operate in this mode if their sinkpad is activated
57 for push-based scheduling. Source elements cannot be chain-based.
62 Alternatively, sinkpads can be the driving force behind a pipeline
63 by operating in <quote>pull</quote>-based mode, while the sourcepads
64 of the element still operate in push-based mode. In order to be the
65 driving force, those pads start a <classname>GstTask</classname>
66 when their pads are being activated. This task is a thread, which
67 will call a function specified by the element. When called, this
68 function will have random data access (through
69 <function>gst_pad_get_range ()</function>) over all sinkpads, and
70 can push data over the sourcepads, which effectively means that
71 this element controls dataflow in the pipeline. Prerequisites for
72 this mode are that all downstream elements can act in chain-based
73 mode, and that all upstream elements allow random access (see below).
74 Source elements can be told to act in this mode if their sourcepads
75 are activated in push-based fashion. Sink elements can be told to
76 act in this mode when their sinkpads are activated in pull-mode.
81 lastly, all pads in an element can be assigned to act in pull-mode.
82 too. However, contrary to the above, this does not mean that they
83 start a task on their own. Rather, it means that they are pull
84 slave for the downstream element, and have to provide random data
85 access to it from their <function>_get_range ()</function>-function.
86 Requiremenents are that the a <function>_get_range
87 ()</function>-function was set on this pad using the function
88 <function>gst_pad_set_getrange_function ()</function>. Also, if
89 the element has any sinkpads, all those pads (and thereby their
90 peers) need to operate in random access mode, too. Note that the
91 element is supposed to activate those elements itself! &GStreamer;
92 will not do that for you.
97 In the next two sections, we will go closer into pull-based scheduling
98 (elements/pads driving the pipeline, and elements/pads providing random
99 access), and some specific use cases will be given.
103 <sect1 id="section-scheduling-loop" xreflabel="Pads driving the pipeline">
104 <title>Pads driving the pipeline</title>
106 Sinkpads assigned to operate in pull-based mode, while none of its
107 sourcepads operate in pull-based mode (or it has no sourcepads), can
108 start a task that will drive the pipeline dataflow. Within this
109 function, those elements have random access over all of their sinkpads,
110 and push data over their sourcepads. This can come in useful for
111 several different kinds of elements:
116 Demuxers, parsers and certain kinds of decoders where data comes
117 in unparsed (such as MPEG-audio or video streams), since those will
118 prefer byte-exact (random) access from their input. If possible,
119 however, such elements should be prepared to operate in chain-based
125 Certain kind of audio outputs, which require control over their
126 input dataflow, such as the Jack sound server.
131 In order to start this task, you will need to create it in the
134 <programlisting><!-- example-begin task.c a -->
136 #include <string.h>
138 static gboolean gst_my_filter_activate (GstPad * pad);
139 static gboolean gst_my_filter_activate_pull (GstPad * pad,
141 static void gst_my_filter_loop (GstMyFilter * filter);
143 GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT);
144 <!-- example-end task.c a -->
145 <!-- example-begin task.c b --><!--
146 static gboolean gst_my_filter_setcaps (GstPad *pad,
148 static GstCaps *gst_my_filter_getcaps (GstPad *pad);
151 gst_my_filter_base_init (gpointer klass)
153 GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
154 static GstElementDetails my_filter_details = {
156 "Example/FirstExample",
157 "Shows the basic structure of a plugin",
158 "your name <your.name@your.isp>"
160 static GstStaticPadTemplate sink_factory =
161 GST_STATIC_PAD_TEMPLATE (
165 GST_STATIC_CAPS ("ANY")
167 static GstStaticPadTemplate src_factory =
168 GST_STATIC_PAD_TEMPLATE (
172 GST_STATIC_CAPS ("ANY")
175 gst_element_class_set_details (element_class, &my_filter_details);
176 gst_element_class_add_pad_template (element_class,
177 gst_static_pad_template_get (&src_factory));
178 gst_element_class_add_pad_template (element_class,
179 gst_static_pad_template_get (&sink_factory));
183 gst_my_filter_class_init (GstMyFilterClass * klass)
186 --><!-- example-begin task.c b -->
187 <!-- example-begin task.c c -->
189 gst_my_filter_init (GstMyFilter * filter)
191 <!-- example-end task.c c -->
192 [..]<!-- example-begin task.c d --><!--
193 GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
195 filter->sinkpad = gst_pad_new_from_template (
196 gst_element_class_get_pad_template (klass, "sink"), "sink");
197 gst_pad_set_setcaps_function (filter->sinkpad, gst_my_filter_setcaps);
198 gst_pad_set_getcaps_function (filter->sinkpad, gst_my_filter_getcaps);
199 --><!-- example-end task.c d -->
200 <!-- example-begin task.c e -->
201 gst_pad_set_activate_function (filter->sinkpad, gst_my_filter_activate);
202 gst_pad_set_activatepull_function (filter->sinkpad,
203 gst_my_filter_activate_pull);
204 <!-- example-end task.c e -->
205 <!-- example-begin task.c f --><!--
206 gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);
208 filter->srcpad = gst_pad_new_from_template (
209 gst_element_class_get_pad_template (klass, "src"), "src");
210 gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
211 --><!-- example-end task.c f -->
212 [..]<!-- example-begin task.c g -->
214 <!-- example-end task.c g -->
215 [..]<!-- example-begin task.c h --><!--
217 --><!-- example-end task.c h -->
218 <!-- example-begin task.c i -->
220 gst_my_filter_activate (GstPad * pad)
222 if (gst_pad_check_pull_range (pad)) {
223 return gst_pad_activate_pull (pad, TRUE);
230 gst_my_filter_activate_pull (GstPad *pad,
233 GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad));
237 return gst_pad_start_task (pad,
238 (GstTaskFunction) gst_my_filter_loop, filter);
240 return gst_pad_stop_task (pad);
243 <!-- example-end task.c i --></programlisting>
245 Once started, your task has full control over input and output. The
246 most simple case of a task function is one that reads input and pushes
247 that over its source pad. It's not all that useful, but provides some
248 more flexibility than the old chain-based case that we've been looking
251 <programlisting><!-- example-begin task.c j -->
252 #define BLOCKSIZE 2048
255 gst_my_filter_loop (GstMyFilter * filter)
259 GstFormat fmt = GST_FORMAT_BYTES;
260 GstBuffer *buf = NULL;
262 if (!gst_pad_query_duration (filter->sinkpad, &fmt, &len)) {
263 GST_DEBUG_OBJECT (filter, "failed to query duration, pausing");
267 if (filter->offset >= len) {
268 GST_DEBUG_OBJECT (filter, "at end of input, sending EOS, pausing");
269 gst_pad_push_event (filter->srcpad, gst_event_new_eos ());
273 /* now, read BLOCKSIZE bytes from byte offset filter->offset */
274 ret = gst_pad_pull_range (filter->sinkpad, filter->offset,
275 BLOCKSIZE, &buf);
277 if (ret != GST_FLOW_OK) {
278 GST_DEBUG_OBJECT (filter, "pull_range failed: %s", gst_flow_get_name (ret));
282 /* now push buffer downstream */
283 ret = gst_pad_push (filter->srcpad, buf);
285 buf = NULL; /* gst_pad_push() took ownership of buffer */
287 if (ret != GST_FLOW_OK) {
288 GST_DEBUG_OBJECT (filter, "pad_push failed: %s", gst_flow_get_name (ret));
292 /* everything is fine, increase offset and wait for us to be called again */
293 filter->offset += BLOCKSIZE;
297 GST_DEBUG_OBJECT (filter, "pausing task");
298 gst_pad_pause_task (filter->sinkpad);
300 <!-- example-end task.c j -->
301 <!-- example-begin task.c k --><!--
302 #include "register.func"
303 --><!-- example-end task.c k --></programlisting>
306 <sect1 id="section-scheduling-randomxs" xreflabel="Providing random access">
307 <title>Providing random access</title>
309 In the previous section, we have talked about how elements (or pads)
310 that are assigned to drive the pipeline using their own task, have
311 random access over their sinkpads. This means that all elements linked
312 to those pads (recursively) need to provide random access functions.
313 Requesting random access is done using the function
314 <function>gst_pad_pull_range ()</function>, which requests a buffer of
315 a specified size and offset. Source pads implementing and assigned to
316 do random access will have a <function>_get_range ()</function>-function
317 set using <function>gst_pad_set_getrange_function ()</function>, and
318 that function will be called when the peer pad requests some data. The
319 element is then responsible for seeking to the right offset and
320 providing the requested data. Several elements can implement random
326 Data sources, such as a file source, that can provide data from any
327 offset with reasonable low latency.
332 Filters that would like to provide a pull-based-like scheduling
333 mode over the whole pipeline. Note that elements assigned to do
334 random access-based scheduling are themselves responsible for
335 assigning this scheduling mode to their upstream peers! &GStreamer;
336 will not do that for you.
341 Parsers who can easily provide this by skipping a small part of
342 their input and are thus essentially "forwarding" random access
343 requests literally without any own processing involved. Examples
344 include tag readers (e.g. ID3) or single output parsers, such as
350 The following example will show how a <function>_get_range
351 ()</function>-function can be implemented in a source element:
353 <programlisting><!-- example-begin range.c a -->
356 gst_my_filter_get_range (GstPad * pad,
361 GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT);
362 <!-- example-end range.c a -->
363 <!-- example-begin range.c b --><!--
365 gst_my_filter_base_init (gpointer klass)
367 GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
368 static GstElementDetails my_filter_details = {
370 "Example/FirstExample",
371 "Shows the basic structure of a plugin",
372 "your name <your.name@your.isp>"
374 static GstStaticPadTemplate src_factory =
375 GST_STATIC_PAD_TEMPLATE (
379 GST_STATIC_CAPS ("ANY")
382 gst_element_class_set_details (element_class, &my_filter_details);
383 gst_element_class_add_pad_template (element_class,
384 gst_static_pad_template_get (&src_factory));
388 gst_my_filter_class_init (GstMyFilterClass * klass)
391 --><!-- example-begin range.c b -->
392 <!-- example-begin range.c c -->
394 gst_my_filter_init (GstMyFilter * filter)
396 GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
398 filter->srcpad = gst_pad_new_from_template (
399 gst_element_class_get_pad_template (klass, "src"), "src");
400 gst_pad_set_getrange_function (filter->srcpad,
401 gst_my_filter_get_range);
402 gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
403 <!-- example-end range.c c -->
404 [..]<!-- example-begin range.c d -->
408 gst_my_filter_get_range (GstPad * pad,
413 <!-- example-end range.c d -->
414 GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad));
416 [.. here, you would fill *buf ..]
417 <!-- example-begin range.c e -->
420 <!-- example-end range.c e -->
421 <!-- example-begin range.c f --><!--
422 #include "register.func"
423 --><!-- example-end range.c f --></programlisting>
425 In practice, many elements that could theoretically do random access,
426 may in practice often be assigned to do push-based scheduling anyway,
427 since there is no downstream element able to start its own task.
428 Therefore, in practice, those elements should implement both a
429 <function>_get_range ()</function>-function and a <function>_chain
430 ()</function>-function (for filters and parsers) or a <function>_get_range
431 ()</function>-function and be prepared to start their own task by
432 providing <function>_activate_* ()</function>-functions (for
433 source elements), so that &GStreamer; can decide for the optimal
434 scheduling mode and have it just work fine in practice.