1 <chapter id="chapter-elements" xreflabel="Elements">
2 <title>Elements</title>
4 The most important object in &GStreamer; for the application programmer
5 is the <ulink type="http"
6 url="../../gstreamer/html/GstElement.html"><classname>GstElement</classname></ulink>
7 object. An element is the basic building block for a media pipeline. All
8 the different high-level components you will use are derived from
9 <classname>GstElement</classname>. Every decoder, encoder, demuxer, video
10 or audio output is in fact a <classname>GstElement</classname>
13 <sect1 id="section-elements-design" xreflabel="What are elements?">
14 <title>What are elements?</title>
16 For the application programmer, elements are best visualized as black
17 boxes. On the one end, you might put something in, the element does
18 something with it and something else comes out at the other side. For
19 a decoder element, for example, you'd put in encoded data, and the
20 element would output decoded data. In the next chapter (see <xref
21 linkend="chapter-pads"/>), you will learn more about data input and
22 output in elements, and how you can set that up in your application.
25 <sect2 id="section-elements-src">
26 <title>Source elements</title>
28 Source elements generate data for use by a pipeline, for example
29 reading from disk or from a sound card. <xref
30 linkend="section-element-srcimg"/> shows how we will visualise
31 a source element. We always draw a source pad to the right of
34 <figure float="1" id="section-element-srcimg">
35 <title>Visualisation of a source element</title>
38 <imagedata scale="75" fileref="images/src-element.ℑ"
44 Source elements do not accept data, they only generate data. You can
45 see this in the figure because it only has a source pad (on the
46 right). A source pad can only generate data.
50 <sect2 id="section-elements-filter">
51 <title>Filters, convertors, demuxers, muxers and codecs</title>
53 Filters and filter-like elements have both input and outputs pads.
54 They operate on data that they receive on their input (sink) pads,
55 and will provide data on their output (source) pads. Examples of
56 such elements are a volume element (filter), a video scaler
57 (convertor), an Ogg demuxer or a Vorbis decoder.
60 Filter-like elements can have any number of source or sink pads. A
61 video demuxer, for example, would have one sink pad and several
62 (1-N) source pads, one for each elementary stream contained in the
63 container format. Decoders, on the other hand, will only have one
66 <figure float="1" id="section-element-filterimg">
67 <title>Visualisation of a filter element</title>
70 <imagedata scale="75" fileref="images/filter-element.ℑ"
76 <xref linkend="section-element-filterimg"/> shows how we will
77 visualise a filter-like element. This specific element has one source
78 and one sink element. Sink pads, receiving input data, are depicted
79 at the left of the element; source pads are still on the right.
81 <figure float="1" id="section-element-multifilterimg">
82 <title>Visualisation of a filter element with
83 more than one output pad</title>
86 <imagedata scale="75" fileref="images/filter-element-multi.ℑ"
92 <xref linkend="section-element-multifilterimg"/> shows another
93 filter-like element, this one having more than one output (source)
94 pad. An example of one such element could, for example, be an Ogg
95 demuxer for an Ogg stream containing both audio and video. One
96 source pad will contain the elementary video stream, another will
97 contain the elementary audio stream. Demuxers will generally fire
98 signals when a new pad is created. The application programmer can
99 then handle the new elementary stream in the signal handler.
103 <sect2 id="section-elements-sink">
104 <title>Sink elements</title>
106 Sink elements are end points in a media pipeline. They accept
107 data but do not produce anything. Disk writing, soundcard playback,
108 and video output would all be implemented by sink elements.
109 <xref linkend="section-element-sinkimg"/> shows a sink element.
111 <figure float="1" id="section-element-sinkimg">
112 <title>Visualisation of a sink element</title>
115 <imagedata scale="75" fileref="images/sink-element.ℑ"
123 <sect1 id="section-elements-create">
124 <title>Creating a <classname>GstElement</classname></title>
126 The simplest way to create an element is to use <ulink type="http"
127 url="&URLAPI;GstElementFactory.html#gst-element-factory-make"><function>gst_element_factory_make
128 ()</function></ulink>. This function takes a factory name and an
129 element name for the newly created element. The name of the element
130 is something you can use later on to look up the element in a bin,
131 for example. The name will also be used in debug output. You can
132 pass <symbol>NULL</symbol> as the name argument to get a unique,
136 When you don't need the element anymore, you need to unref it using
138 url="&URLAPI;GstObject.html#gst-object-unref"><function>gst_object_unref
139 ()</function></ulink>. This decreases the reference count for the
140 element by 1. An element has a refcount of 1 when it gets created.
141 An element gets destroyed completely when the refcount is decreased
145 The following example &EXAFOOT; shows how to create an element named
146 <emphasis>source</emphasis> from the element factory named
147 <emphasis>fakesrc</emphasis>. It checks if the creation succeeded.
148 After checking, it unrefs the element.
150 <programlisting><!-- example-begin elementmake.c --><![CDATA[
160 gst_init (&argc, &argv);
163 element = gst_element_factory_make ("fakesrc", "source");
165 g_print ("Failed to create element of type 'fakesrc'\n");
169 gst_object_unref (GST_OBJECT (element));
173 ]]><!-- example-end elementmake.c --></programlisting>
175 <function>gst_element_factory_make</function> is actually a shorthand
176 for a combination of two functions. A <ulink type="http"
177 url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink>
178 object is created from a factory. To create the element, you have to
179 get access to a <ulink type="http"
180 url="&URLAPI;GstElementFactory.html"><classname>GstElementFactory</classname></ulink>
181 object using a unique factory name. This is done with <ulink type="http"
182 url="&URLAPI;GstElementFactory.html#gst-element-factory-find"><function>gst_element_factory_find
183 ()</function></ulink>.
186 The following code fragment is used to get a factory that can be used
187 to create the <emphasis>fakesrc</emphasis> element, a fake data source.
188 The function <ulink type="http"
189 url="&URLAPI;GstElementFactory.html#gst-element-factory-create"><function>gst_element_factory_create
190 ()</function></ulink> will use the element factory to create an
191 element with the given name.
193 <programlisting><!-- example-begin elementcreate.c --><![CDATA[
200 GstElementFactory *factory;
201 GstElement * element;
204 gst_init (&argc, &argv);
206 /* create element, method #2 */
207 factory = gst_element_factory_find ("fakesrc");
209 g_print ("Failed to find factory of type 'fakesrc'\n");
212 element = gst_element_factory_create (factory, "source");
214 g_print ("Failed to create element, even though its factory exists!\n");
218 gst_object_unref (GST_OBJECT (element));
222 ]]><!-- example-end elementcreate.c --></programlisting>
225 <sect1 id="section-elements-properties">
226 <title>Using an element as a <classname>GObject</classname></title>
229 url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink>
230 can have several properties which are implemented using standard
231 <classname>GObject</classname> properties. The usual
232 <classname>GObject</classname> methods to query, set and get
233 property values and <classname>GParamSpecs</classname> are
237 Every <classname>GstElement</classname> inherits at least one
238 property from its parent <classname>GstObject</classname>: the
239 "name" property. This is the name you provide to the functions
240 <function>gst_element_factory_make ()</function> or
241 <function>gst_element_factory_create ()</function>. You can get
242 and set this property using the functions
243 <function>gst_object_set_name</function> and
244 <function>gst_object_get_name</function> or use the
245 <classname>GObject</classname> property mechanism as shown below.
247 <programlisting><!-- example-begin elementget.c --><![CDATA[
258 gst_init (&argc, &argv);
261 element = gst_element_factory_make ("fakesrc", "source");
264 g_object_get (G_OBJECT (element), "name", &name, NULL);
265 g_print ("The name of the element is '%s'.\n", name);
268 gst_object_unref (GST_OBJECT (element));
272 ]]><!-- example-end elementget.c --></programlisting>
274 Most plugins provide additional properties to provide more information
275 about their configuration or to configure the element.
276 <command>gst-inspect</command> is a useful tool to query the properties
277 of a particular element, it will also use property introspection to give
278 a short explanation about the function of the property and about the
279 parameter types and ranges it supports. See
280 <xref linkend="section-applications-inspect"/>
281 in the appendix for details about <command>gst-inspect</command>.
284 For more information about <classname>GObject</classname>
285 properties we recommend you read the <ulink
286 url="http://developer.gnome.org/doc/API/2.0/gobject/index.html"
287 type="http">GObject manual</ulink> and an introduction to <ulink
288 url="http://developer.gnome.org/doc/API/2.0/gobject/pr01.html" type="http">
289 The Glib Object system</ulink>.
292 A <ulink type="http" url="&URLAPI;GstElementFactory.html">
293 <classname>GstElement</classname></ulink> also provides various
294 <classname>GObject</classname> signals that can be used as a flexible
295 callback mechanism. Here, too, you can use <command>gst-inspect</command>
296 to see which signals a specific element supports. Together, signals
297 and properties are the most basic way in which elements and
298 applications interact.
302 <sect1 id="section-elements-factories">
303 <title>More about element factories</title>
305 In the previous section, we briefly introduced the <ulink type="http"
306 url="&URLAPI;GstElement.html"><classname>GstElementFactory</classname></ulink>
307 object already as a way to create instances of an element. Element
308 factories, however, are much more than just that. Element factories
309 are the basic types retrieved from the &GStreamer; registry, they
310 describe all plugins and elements that &GStreamer; can create. This
311 means that element factories are useful for automated element
312 instancing, such as what autopluggers do, and for creating lists
313 of available elements, such as what pipeline editing applications
314 (e.g. <ulink type="http"
315 url="http://gstreamer.freedesktop.org/modules/gst-editor.html">&GStreamer;
319 <sect2 id="section-elements-factories-details">
320 <title>Getting information about an element using a factory</title>
322 Tools like <command>gst-inspect</command> will provide some generic
323 information about an element, such as the person that wrote the
324 plugin, a descriptive name (and a shortname), a rank and a category.
325 The category can be used to get the type of the element that can
326 be created using this element factory. Examples of categories include
327 <classname>Codec/Decoder/Video</classname> (video decoder),
328 <classname>Codec/Encoder/Video</classname> (video encoder),
329 <classname>Source/Video</classname> (a video generator),
330 <classname>Sink/Video</classname> (a video output), and all these
331 exist for audio as well, of course. Then, there's also
332 <classname>Codec/Demuxer</classname> and
333 <classname>Codec/Muxer</classname> and a whole lot more.
334 <command>gst-inspect</command> will give a list of all factories, and
335 <command>gst-inspect <factory-name></command> will list all
336 of the above information, and a lot more.
338 <programlisting><!-- example-begin elementfactory.c --><![CDATA[
345 GstElementFactory *factory;
348 gst_init (&argc, &argv);
351 factory = gst_element_factory_find ("fakesrc");
353 g_print ("You don't have the 'fakesrc' element installed!\n");
357 /* display information */
358 g_print ("The '%s' element is a member of the category %s.\n"
360 gst_plugin_feature_get_name (GST_PLUGIN_FEATURE (factory)),
361 gst_element_factory_get_klass (factory),
362 gst_element_factory_get_description (factory));
366 ]]><!-- example-end elementfactory.c --></programlisting>
368 You can use <function>gst_registry_pool_feature_list (GST_TYPE_ELEMENT_FACTORY)</function>
369 to get a list of all the element factories that &GStreamer; knows
374 <sect2 id="section-elements-factories-padtemplates">
375 <title>Finding out what pads an element can contain</title>
377 Perhaps the most powerful feature of element factories is that
378 they contain a full description of the pads that the element
379 can generate, and the capabilities of those pads (in layman words:
380 what types of media can stream over those pads), without actually
381 having to load those plugins into memory. This can be used
382 to provide a codec selection list for encoders, or it can be used
383 for autoplugging purposes for media players. All current
384 &GStreamer;-based media players and autopluggers work this way.
385 We'll look closer at these features as we learn about
386 <classname>GstPad</classname> and <classname>GstCaps</classname>
387 in the next chapter: <xref linkend="chapter-pads"/>
392 <sect1 id="section-elements-link" xreflabel="Linking elements">
393 <title>Linking elements</title>
395 By linking a source element with zero or more filter-like
396 elements and finally a sink element, you set up a media
397 pipeline. Data will flow through the elements. This is the
398 basic concept of media handling in &GStreamer;.
400 <figure float="1" id="section-link">
401 <title>Visualisation of three linked elements</title>
404 <imagedata scale="75" fileref="images/linked-elements.ℑ"
410 By linking these three elements, we have created a very simple
411 chain of elements. The effect of this will be that the output of
412 the source element (<quote>element1</quote>) will be used as input
413 for the filter-like element (<quote>element2</quote>). The
414 filter-like element will do something with the data and send the
415 result to the final sink element (<quote>element3</quote>).
418 Imagine the above graph as a simple Ogg/Vorbis audio decoder. The
419 source is a disk source which reads the file from disc. The second
420 element is a Ogg/Vorbis audio decoder. The sink element is your
421 soundcard, playing back the decoded audio data. We will use this
422 simple graph to construct an Ogg/Vorbis player later in this manual.
425 In code, the above graph is written like this:
427 <programlisting><!-- example-begin elementlink.c a -->
428 #include <gst/gst.h>
434 GstElement *pipeline;
435 GstElement *source, *filter, *sink;
438 gst_init (&argc, &argv);
440 /* create pipeline */
441 pipeline = gst_pipeline_new ("my-pipeline");
443 /* create elements */
444 source = gst_element_factory_make ("fakesrc", "source");
445 filter = gst_element_factory_make ("identity", "filter");
446 sink = gst_element_factory_make ("fakesink", "sink");
448 /* must add elements to pipeline before linking them */
449 gst_bin_add_many (GST_BIN (pipeline), source, filter, sink, NULL);
452 if (!gst_element_link_many (source, filter, sink, NULL)) {
453 g_warning ("Failed to link elements!");
455 <!-- example-end elementlink.c a -->
456 [..]<!-- example-begin elementlink.c b --><!--
458 --><!-- example-end elementlink.c b -->
459 <!-- example-begin elementlink.c c -->
461 <!-- example-end elementlink.c c --></programlisting>
463 For more specific behaviour, there are also the functions
464 <function>gst_element_link ()</function> and
465 <function>gst_element_link_pads ()</function>. You can also obtain
466 references to individual pads and link those using various
467 <function>gst_pad_link_* ()</function> functions. See the API
468 references for more details.
471 Important: you must add elements to a bin or pipeline
472 <emphasis>before</emphasis> linking them, since adding an element to
473 a bin will disconnect any already existing links. Also, you cannot
474 directly link elements that are not in the same bin or pipeline; if
475 you want to link elements or pads at different hierarchy levels, you
476 will need to use ghost pads (more about ghost pads later).
480 <sect1 id="section-elements-states">
481 <title>Element States</title>
483 After being created, an element will not actually perform any actions
484 yet. You need to change elements state to make it do something.
485 &GStreamer; knows four element states, each with a very specific
486 meaning. Those four states are:
491 <classname>GST_STATE_NULL</classname>: this is the default state.
492 No resources are allocated in this state, so, transitioning to it
493 will free all resources. The element must be in this state when
494 its refcount reaches 0 and it is freed.
499 <classname>GST_STATE_READY</classname>: in the ready state, an
500 element has allocated all of its global resources, that is,
501 resources that can be kept within streams. You can think about
502 opening devices, allocating buffers and so on. However, the
503 stream is not opened in this state, so the stream positions is
504 automatically zero. If a stream was previously opened, it should
505 be closed in this state, and position, properties and such should
511 <classname>GST_STATE_PAUSED</classname>: in this state, an
512 element has opened the stream, but is not actively processing
513 it. An element is allowed to modify a stream's position, read
514 and process data and such to prepare for playback as soon as
515 state is changed to PLAYING, but it is <emphasis>not</emphasis>
516 allowed to play the data which would make the clock run.
517 In summary, PAUSED is the same as PLAYING but without a running
521 Elements going into the PAUSED state should prepare themselves
522 for moving over to the PLAYING state as soon as possible. Video
523 or audio outputs would, for example, wait for data to arrive and
524 queue it so they can play it right after the state change. Also,
525 video sinks can already play the first frame (since this does
526 not affect the clock yet). Autopluggers could use this same
527 state transition to already plug together a pipeline. Most other
528 elements, such as codecs or filters, do not need to explicitly
529 do anything in this state, however.
534 <classname>GST_STATE_PLAYING</classname>: in the PLAYING state,
535 an element does exactly the same as in the PAUSED state, except
536 that the clock now runs.
541 You can change the state of an element using the function
542 <function>gst_element_set_state ()</function>. If you set an element
543 to another state, &GStreamer; will internally traverse all intermediate
544 states. So if you set an element from NULL to PLAYING, &GStreamer;
545 will internally set the element to READY and PAUSED in between.
548 When moved to <classname>GST_STATE_PLAYING</classname>, pipelines
549 will process data automatically. They do not need to be iterated in
550 any form. Internally, &GStreamer; will start threads that take this
551 task on to them. &GStreamer; will also take care of switching
552 messages from the pipeline's thread into the application's own
553 thread, by using a <ulink type="http"
554 url="&URLAPI;GstBus.html"><classname>GstBus</classname></ulink>. See
555 <xref linkend="chapter-bus"/> for details.
558 When you set a bin or pipeline to a certain target state, it will usually
559 propagate the state change to all elements within the bin or pipeline
560 automatically, so it's usually only necessary to set the state of the
561 top-level pipeline to start up the pipeline or shut it down. However,
562 when adding elements dynamically to an already-running pipeline, e.g.
563 from within a "pad-added" or "new-decoded-pad" signal callback, you
564 need to set it to the desired target state yourself using
565 <function>gst_element_set_state ()</function> or
566 <function>gst_element_sync_state_with_parent ()</function>.