1 <chapter id="chapter-pads" xreflabel="Pads and capabilities">
2 <title>Pads and capabilities</title>
4 As we have seen in <xref linkend="chapter-elements"/>, the pads are
5 the element's interface to the outside world. Data streams from one
6 element's source pad to another element's sink pad. The specific
7 type of media that the element can handle will be exposed by the
8 pad's capabilities. We will talk more on capabilities later in this
9 chapter (see <xref linkend="section-caps"/>).
12 <sect1 id="section-pads">
15 A pad type is defined by two properties: its direction and its
16 availability. As we've mentioned before, &GStreamer; defines two
17 pad directions: source pads and sink pads. This terminology is
18 defined from the view of within the element: elements receive data
19 on their sink pads and generate data on their source pads.
20 Schematically, sink pads are drawn on the left side of an element,
21 whereas source pads are drawn on the right side of an element. In
22 such graphs, data flows from left to right.
25 In reality, there is no objection to data flowing from a
26 source pad to the sink pad of an element upstream (to the
27 left of this element in drawings). Data will, however, always
28 flow from a source pad of one element to the sink pad of
35 Pad directions are very simple compared to pad availability. A pad
36 can have any of three availabilities: always, sometimes and on
37 request. The meaning of those three types is exactly as it says:
38 always pads always exist, sometimes pad exist only in certain
39 cases (and can disappear randomly), and on-request pads appear
40 only if explicitely requested by applications.
43 <sect2 id="section-pads-dynamic">
44 <title>Dynamic (or sometimes) pads</title>
46 Some elements might not have all of their pads when the element is
47 created. This can happen, for example, with an Ogg demuxer element.
48 The element will read the Ogg stream and create dynamic pads for
49 each contained elementary stream (vorbis, theora) when it detects
50 such a stream in the Ogg stream. Likewise, it will delete the pad
51 when the stream ends. This principle is very useful for demuxer
52 elements, for example.
55 Running <application>gst-inspect oggdemux</application> will show
56 that the element has only one pad: a sink pad called 'sink'. The
57 other pads are <quote>dormant</quote>. You can see this in the pad
58 template because there is an <quote>Exists: Sometimes</quote>
59 property. Depending on the type of Ogg file you play, the pads will
60 be created. We will see that this is very important when you are
61 going to create dynamic pipelines. You can attach a signal handler
62 to an element to inform you when the element has created a new pad
63 from one of its <quote>sometimes</quote> pad templates. The
64 following piece of code is an example of how to do this:
66 <programlisting><!-- example-begin pad.c a -->
67 #include <gst/gst.h>
70 cb_new_pad (GstElement *element,
76 name = gst_pad_get_name (pad);
77 g_print ("A new pad %s was created\n", name);
80 /* here, you would setup a new pad link for the newly created pad */
81 <!-- example-end pad.c a -->[..]
82 <!-- example-begin pad.c b -->
89 GstElement *pipeline, *source, *demux;
93 gst_init (&argc, &argv);
96 pipeline = gst_pipeline_new ("my_pipeline");
97 source = gst_element_factory_make ("filesrc", "source");
98 g_object_set (source, "location", argv[1], NULL);
99 demux = gst_element_factory_make ("oggdemux", "demuxer");
101 /* you would normally check that the elements were created properly */
103 /* put together a pipeline */
104 gst_bin_add_many (GST_BIN (pipeline), source, demux, NULL);
105 gst_element_link_pads (source, "src", demux, "sink");
107 /* listen for newly created pads */
108 g_signal_connect (demux, "pad-added", G_CALLBACK (cb_new_pad), NULL);
110 /* start the pipeline */
111 gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);
112 loop = g_main_loop_new (NULL, FALSE);
113 g_main_loop_run (loop);
114 <!--example-end pad.c b -->
115 [..]<!-- example-begin pad.c c --><!--
117 --><!-- example-end pad.c c -->
118 <!-- example-begin pad.c d -->
120 <!-- example-end pad.c d --></programlisting>
123 <sect2 id="section-pads-request">
124 <title>Request pads</title>
126 An element can also have request pads. These pads are not created
127 automatically but are only created on demand. This is very useful
128 for multiplexers, aggregators and tee elements. Aggregators are
129 elements that merge the content of several input streams together
130 into one output stream. Tee elements are the reverse: they are
131 elements that have one input stream and copy this stream to each
132 of their output pads, which are created on request. Whenever an
133 application needs another copy of the stream, it can simply request
134 a new output pad from the tee element.
137 The following piece of code shows how you can request a new output
138 pad from a <quote>tee</quote> element:
142 some_function (GstElement *tee)
147 pad = gst_element_get_request_pad (tee, "src%d");
148 name = gst_pad_get_name (pad);
149 g_print ("A new pad %s was created\n", name);
152 /* here, you would link the pad */
155 /* and, after doing that, free our reference */
156 gst_object_unref (GST_OBJECT (pad));
160 The <function>gst_element_get_request_pad ()</function> method
161 can be used to get a pad from the element based on the name of
162 the pad template. It is also possible to request a pad that is
163 compatible with another pad template. This is very useful if
164 you want to link an element to a multiplexer element and you
165 need to request a pad that is compatible. The method
166 <function>gst_element_get_compatible_pad ()</function> can be
167 used to request a compatible pad, as shown in the next example.
168 It will request a compatible pad from an Ogg multiplexer from
173 link_to_multiplexer (GstPad *tolink_pad,
177 gchar *srcname, *sinkname;
179 srcname = gst_pad_get_name (tolink_pad);
180 pad = gst_element_get_compatible_pad (mux, tolink_pad);
181 gst_pad_link (tolinkpad, pad);
182 sinkname = gst_pad_get_name (pad);
183 gst_object_unref (GST_OBJECT (pad));
185 g_print ("A new pad %s was created and linked to %s\n", srcname, sinkname);
193 <sect1 id="section-caps">
194 <title>Capabilities of a pad</title>
196 Since the pads play a very important role in how the element is
197 viewed by the outside world, a mechanism is implemented to describe
198 the data that can flow or currently flows through the pad by using
199 capabilities. Here, we will briefly describe what capabilities are
200 and how to use them, enough to get an understanding of the concept.
201 For an in-depth look into capabilities and a list of all capabilities
202 defined in &GStreamer;, see the <ulink type="http"
203 url="http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/index.html">Plugin
204 Writers Guide</ulink>.
207 Capabilities are attached to pad templates and to pads. For pad
208 templates, it will describe the types of media that may stream
209 over a pad created from this template. For pads, it can either
210 be a list of possible caps (usually a copy of the pad template's
211 capabilities), in which case the pad is not yet negotiated, or it
212 is the type of media that currently streams over this pad, in
213 which case the pad has been negotiated already.
216 <sect2 id="section-caps-structure">
217 <title>Dissecting capabilities</title>
219 A pad's capabilities are described in a <classname>GstCaps</classname>
220 object. Internally, a <ulink type="http"
221 url="../../gstreamer/html/gstreamer-GstCaps.html"><classname>GstCaps</classname></ulink>
222 will contain one or more <ulink type="http"
223 url="../../gstreamer/html/gstreamer-GstStructure.html"><classname>GstStructure</classname></ulink>
224 that will describe one media type. A negotiated pad will have
225 capabilities set that contain exactly <emphasis>one</emphasis>
226 structure. Also, this structure will contain only
227 <emphasis>fixed</emphasis> values. These constraints are not
228 true for unnegotiated pads or pad templates.
231 As an example, below is a dump of the capabilities of the
232 <quote>vorbisdec</quote> element, which you will get by running
233 <command>gst-inspect vorbisdec</command>. You will see two pads:
234 a source and a sink pad. Both of these pads are always available,
235 and both have capabilities attached to them. The sink pad will
236 accept vorbis-encoded audio data, with the mime-type
237 <quote>audio/x-vorbis</quote>. The source pad will be used
238 to send raw (decoded) audio samples to the next element, with
239 a raw audio mime-type (in this case,
240 <quote>audio/x-raw-float</quote>). The source pad will also
241 contain properties for the audio samplerate and the amount of
242 channels, plus some more that you don't need to worry about
251 rate: [ 8000, 50000 ]
257 SINK template: 'sink'
264 <sect2 id="section-caps-props">
265 <title>Properties and values</title>
267 Properties are used to describe extra information for
268 capabilities. A property consists of a key (a string) and
269 a value. There are different possible value types that can be used:
274 Basic types, this can be pretty much any
275 <classname>GType</classname> registered with Glib. Those
276 properties indicate a specific, non-dynamic value for this
277 property. Examples include:
282 An integer value (<classname>G_TYPE_INT</classname>):
283 the property has this exact value.
288 A boolean value (<classname>G_TYPE_BOOLEAN</classname>):
289 the property is either TRUE or FALSE.
294 A float value (<classname>G_TYPE_FLOAT</classname>):
295 the property has this exact floating point value.
300 A string value (<classname>G_TYPE_STRING</classname>):
301 the property contains a UTF-8 string.
306 A fraction value (<classname>GST_TYPE_FRACTION</classname>):
307 contains a fraction expressed by an integer numerator and
315 Range types are <classname>GType</classname>s registered by
316 &GStreamer; to indicate a range of possible values. They are
317 used for indicating allowed audio samplerate values or
318 supported video sizes. The two types defined in &GStreamer;
324 An integer range value
325 (<classname>GST_TYPE_INT_RANGE</classname>): the property
326 denotes a range of possible integers, with a lower and an
327 upper boundary. The <quote>vorbisdec</quote> element, for
328 example, has a rate property that can be between 8000 and
335 (<classname>GST_TYPE_FLOAT_RANGE</classname>): the property
336 denotes a range of possible floating point values, with a
337 lower and an upper boundary.
342 A fraction range value
343 (<classname>GST_TYPE_FRACTION_RANGE</classname>): the property
344 denotes a range of possible fraction values, with a
345 lower and an upper boundary.
352 A list value (<classname>GST_TYPE_LIST</classname>): the
353 property can take any value from a list of basic values
357 Example: caps that express that either
358 a sample rate of 44100 Hz and a sample rate of 48000 Hz
359 is supported would use a list of integer values, with
360 one value being 44100 and one value being 48000.
365 An array value (<classname>GST_TYPE_ARRAY</classname>): the
366 property is an array of values. Each value in the array is a
367 full value on its own, too. All values in the array should be
368 of the same elementary type. This means that an array can
369 contain any combination of integers, lists of integers, integer
370 ranges together, and the same for floats or strings, but it can
371 not contain both floats and ints at the same time.
374 Example: for audio where there are more than two channels involved
375 the channel layout needs to be specified (for one and two channel
376 audio the channel layout is implicit unless stated otherwise in the
377 caps). So the channel layout would be an array of integer enum
378 values where each enum value represents a loudspeaker position.
379 Unlike a <classname>GST_TYPE_LIST</classname>, the values in an
380 array will be interpreted as a whole.
387 <sect1 id="section-caps-api">
388 <title>What capabilities are used for</title>
390 Capabilities (short: caps) describe the type of data that is streamed
391 between two pads, or that one pad (template) supports. This makes them
392 very useful for various purposes:
397 Autoplugging: automatically finding elements to link to a
398 pad based on its capabilities. All autopluggers use this
404 Compatibility detection: when two pads are linked, &GStreamer;
405 can verify if the two pads are talking about the same media
406 type. The process of linking two pads and checking if they
407 are compatible is called <quote>caps negotiation</quote>.
412 Metadata: by reading the capabilities from a pad, applications
413 can provide information about the type of media that is being
414 streamed over the pad, which is information about the stream
415 that is currently being played back.
420 Filtering: an application can use capabilities to limit the
421 possible media types that can stream between two pads to a
422 specific subset of their supported stream types. An application
423 can, for example, use <quote>filtered caps</quote> to set a
424 specific (fixed or non-fixed) video size that should stream
425 between two pads. You will see an example of filtered caps
426 later in this manual, in <xref linkend="section-data-spoof"/>.
427 You can do caps filtering by inserting a capsfilter element into
428 your pipeline and setting its <quote>caps</quote> property. Caps
429 filters are often placed after converter elements like audioconvert,
430 audioresample, ffmpegcolorspace or videoscale to force those
431 converters to convert data to a specific output format at a
432 certain point in a stream.
437 <sect2 id="section-caps-metadata">
438 <title>Using capabilities for metadata</title>
440 A pad can have a set (i.e. one or more) of capabilities attached
441 to it. Capabilities (<classname>GstCaps</classname>) are represented
442 as an array of one or more <classname>GstStructure</classname>s, and
443 each <classname>GstStructure</classname> is an array of fields where
444 each field consists of a field name string (e.g. "width") and a
445 typed value (e.g. <classname>G_TYPE_INT</classname> or
446 <classname>GST_TYPE_INT_RANGE</classname>).
449 Note that there is a distinct difference between the
450 <emphasis>possible</emphasis> capabilities of a pad (ie. usually what
451 you find as caps of pad templates as they are shown in gst-inspect),
452 the <emphasis>allowed</emphasis> caps of a pad (can be the same as
453 the pad's template caps or a subset of them, depending on the possible
454 caps of the peer pad) and lastly <emphasis>negotiated</emphasis> caps
455 (these describe the exact format of a stream or buffer and contain
456 exactly one structure and have no variable bits like ranges or lists,
457 ie. they are fixed caps).
460 You can get values of properties in a set of capabilities
461 by querying individual properties of one structure. You can get
462 a structure from a caps using
463 <function>gst_caps_get_structure ()</function> and the number of
464 structures in a <classname>GstCaps</classname> using
465 <function>gst_caps_get_size ()</function>.
468 Caps are called <emphasis>simple caps</emphasis> when they contain
469 only one structure, and <emphasis>fixed caps</emphasis> when they
470 contain only one structure and have no variable field types (like
471 ranges or lists of possible values). Two other special types of caps
472 are <emphasis>ANY caps</emphasis> and <emphasis>empty caps</emphasis>.
475 Here is an example of how to extract the width and height from
476 a set of fixed video caps:
479 read_video_props (GstCaps *caps)
482 const GstStructure *str;
484 g_return_if_fail (gst_caps_is_fixed (caps));
486 str = gst_caps_get_structure (caps, 0);
487 if (!gst_structure_get_int (str, "width", &width) ||
488 !gst_structure_get_int (str, "height", &height)) {
489 g_print ("No width/height available\n");
493 g_print ("The video size of this set of capabilities is %dx%d\n",
500 <sect2 id="section-caps-filter">
501 <title>Creating capabilities for filtering</title>
503 While capabilities are mainly used inside a plugin to describe the
504 media type of the pads, the application programmer often also has
505 to have basic understanding of capabilities in order to interface
506 with the plugins, especially when using filtered caps. When you're
507 using filtered caps or fixation, you're limiting the allowed types of
508 media that can stream between two pads to a subset of their supported
509 media types. You do this using a <classname>capsfilter</classname>
510 element in your pipeline. In order to do this, you also need to
511 create your own <classname>GstCaps</classname>. The easiest way to
512 do this is by using the convenience function
513 <function>gst_caps_new_simple ()</function>:
518 link_elements_with_filter (GstElement *element1, GstElement *element2)
523 caps = gst_caps_new_simple ("video/x-raw-yuv",
524 "format", GST_TYPE_FOURCC, GST_MAKE_FOURCC ('I', '4', '2', '0'),
525 "width", G_TYPE_INT, 384,
526 "height", G_TYPE_INT, 288,
527 "framerate", GST_TYPE_FRACTION, 25, 1,
530 link_ok = gst_element_link_filtered (element1, element2, caps);
531 gst_caps_unref (caps);
534 g_warning ("Failed to link element1 and element2!");
540 This will force the data flow between those two elements to
541 a certain video format, width, height and framerate (or the linking
542 will fail if that cannot be achieved in the context of the elements
543 involved). Keep in mind that when you use <function>
544 gst_element_link_filtered ()</function> it will automatically create
545 a <classname>capsfilter</classname> element for you and insert it into
546 your bin or pipeline between the two elements you want to connect (this
547 is important if you ever want to disconnect those elements because then
548 you will have to disconnect both elements from the capsfilter instead).
551 In some cases, you will want to create a more elaborate set of
552 capabilities to filter a link between two pads. Then, this function
553 is too simplistic and you'll want to use the method
554 <function>gst_caps_new_full ()</function>:
558 link_elements_with_filter (GstElement *element1, GstElement *element2)
563 caps = gst_caps_new_full (
564 gst_structure_new ("video/x-raw-yuv",
565 "width", G_TYPE_INT, 384,
566 "height", G_TYPE_INT, 288,
567 "framerate", GST_TYPE_FRACTION, 25, 1,
569 gst_structure_new ("video/x-raw-rgb",
570 "width", G_TYPE_INT, 384,
571 "height", G_TYPE_INT, 288,
572 "framerate", GST_TYPE_FRACTION, 25, 1,
576 link_ok = gst_element_link_filtered (element1, element2, caps);
577 gst_caps_unref (caps);
580 g_warning ("Failed to link element1 and element2!");
587 See the API references for the full API of
589 url="&URLAPI;GstStructure.html"><classname>GstStructure</classname></ulink>
590 and <ulink type="http"
591 url="&URLAPI;GstCaps.html"><classname>GstCaps</classname></ulink>.
596 <sect1 id="section-pads-ghost">
597 <title>Ghost pads</title>
599 You can see from <xref linkend="section-bin-noghost-img"/> how a bin
600 has no pads of its own. This is where "ghost pads" come into play.
602 <figure float="1" id="section-bin-noghost-img">
603 <title>Visualisation of a <ulink type="http"
604 url="../../gstreamer/html/GstBin.html"><classname>GstBin</classname></ulink>
605 element without ghost pads</title>
608 <imagedata scale="75" fileref="images/bin-element-noghost.ℑ"
614 A ghost pad is a pad from some element in the bin that can be
615 accessed directly from the bin as well. Compare it to a symbolic
616 link in UNIX filesystems. Using ghost pads on bins, the bin also
617 has a pad and can transparently be used as an element in other
621 <figure float="1" id="section-bin-ghost-img">
622 <title>Visualisation of a <ulink type="http"
623 url="../../gstreamer/html/GstBin.html"><classname>GstBin</classname></ulink>
624 element with a ghost pad</title>
627 <imagedata scale="75" fileref="images/bin-element-ghost.ℑ"
633 <xref linkend="section-bin-ghost-img"/> is a representation of a
634 ghost pad. The sink pad of element one is now also a pad of the bin.
635 Because ghost pads look and work like any other pads, they can be added
636 to any type of elements, not just to a <classname>GstBin</classname>,
637 just like ordinary pads.
640 A ghostpad is created using the function
641 <function>gst_ghost_pad_new ()</function>:
643 <programlisting><!-- example-begin ghostpad.c a -->
644 #include <gst/gst.h>
650 GstElement *bin, *sink;
654 gst_init (&argc, &argv);
656 /* create element, add to bin */
657 sink = gst_element_factory_make ("fakesink", "sink");
658 bin = gst_bin_new ("mybin");
659 gst_bin_add (GST_BIN (bin), sink);
662 pad = gst_element_get_static_pad (sink, "sink");
663 gst_element_add_pad (bin, gst_ghost_pad_new ("sink", pad));
664 gst_object_unref (GST_OBJECT (pad));
665 <!-- example-end ghostpad.c a -->
666 [..]<!-- example-begin ghostpad.c b --><!--
668 --><!-- example-end ghostpad.c b -->
669 <!-- example-begin ghostpad.c c -->
671 <!-- example-end ghostpad.c c --></programlisting>
673 In the above example, the bin now also has a pad: the pad called
674 <quote>sink</quote> of the given element. The bin can, from here
675 on, be used as a substitute for the sink element. You could, for
676 example, link another element to the bin.