2 title: Pads and capabilities
5 # Pads and capabilities
7 As we have seen in [Elements](manual/building/elements.md), the pads are the
8 element's interface to the outside world. Data streams from one
9 element's source pad to another element's sink pad. The specific type of
10 media that the element can handle will be exposed by the pad's
11 capabilities. We will talk more on capabilities later in this chapter
12 (see [Capabilities of a pad](#capabilities-of-a-pad)).
16 A pad type is defined by two properties: its direction and its
17 availability. As we've mentioned before, GStreamer defines two pad
18 directions: source pads and sink pads. This terminology is defined from
19 the view of within the element: elements receive data on their sink pads
20 and generate data on their source pads. Schematically, sink pads are
21 drawn on the left side of an element, whereas source pads are drawn on
22 the right side of an element. In such graphs, data flows from left to
25 Pad directions are very simple compared to pad availability. A pad can
26 have any of three availabilities: always, sometimes and on request. The
27 meaning of those three types is exactly as it says: always pads always
28 exist, sometimes pad exist only in certain cases (and can disappear
29 randomly), and on-request pads appear only if explicitly requested by
32 ### Dynamic (or sometimes) pads
34 Some elements might not have all of their pads when the element is
35 created. This can happen, for example, with an Ogg demuxer element. The
36 element will read the Ogg stream and create dynamic pads for each
37 contained elementary stream (vorbis, theora) when it detects such a
38 stream in the Ogg stream. Likewise, it will delete the pad when the
39 stream ends. This principle is very useful for demuxer elements, for
42 Running gst-inspect oggdemux will show that the element has only one
43 pad: a sink pad called 'sink'. The other pads are “dormant”. You can see
44 this in the pad template because there is an “Exists: Sometimes”
45 property. Depending on the type of Ogg file you play, the pads will be
46 created. We will see that this is very important when you are going to
47 create dynamic pipelines. You can attach a signal handler to an element
48 to inform you when the element has created a new pad from one of its
49 “sometimes” pad templates. The following piece of code is an example
56 cb_new_pad (GstElement *element,
62 name = gst_pad_get_name (pad);
63 g_print ("A new pad %s was created\n", name);
66 /* here, you would setup a new pad link for the newly created pad */
75 GstElement *pipeline, *source, *demux;
79 gst_init (&argc, &argv);
82 pipeline = gst_pipeline_new ("my_pipeline");
83 source = gst_element_factory_make ("filesrc", "source");
84 g_object_set (source, "location", argv[1], NULL);
85 demux = gst_element_factory_make ("oggdemux", "demuxer");
87 /* you would normally check that the elements were created properly */
89 /* put together a pipeline */
90 gst_bin_add_many (GST_BIN (pipeline), source, demux, NULL);
91 gst_element_link_pads (source, "src", demux, "sink");
93 /* listen for newly created pads */
94 g_signal_connect (demux, "pad-added", G_CALLBACK (cb_new_pad), NULL);
96 /* start the pipeline */
97 gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);
98 loop = g_main_loop_new (NULL, FALSE);
99 g_main_loop_run (loop);
107 It is not uncommon to add elements to the pipeline only from within the
108 "pad-added" callback. If you do this, don't forget to set the state of
109 the newly-added elements to the target state of the pipeline using
110 `gst_element_set_state ()` or `gst_element_sync_state_with_parent ()`.
114 An element can also have request pads. These pads are not created
115 automatically but are only created on demand. This is very useful for
116 multiplexers, aggregators and tee elements. Aggregators are elements
117 that merge the content of several input streams together into one output
118 stream. Tee elements are the reverse: they are elements that have one
119 input stream and copy this stream to each of their output pads, which
120 are created on request. Whenever an application needs another copy of
121 the stream, it can simply request a new output pad from the tee element.
123 The following piece of code shows how you can request a new output pad
124 from a “tee” element:
126 {{ snippets.c#some_function }}
128 The `gst_element_get_request_pad ()` method can be used to get a pad
129 from the element based on the name of the pad template. It is also
130 possible to request a pad that is compatible with another pad template.
131 This is very useful if you want to link an element to a multiplexer
132 element and you need to request a pad that is compatible. The method
133 `gst_element_get_compatible_pad ()` can be used to request a compatible
134 pad, as shown in the next example. It will request a compatible pad from
135 an Ogg multiplexer from any input.
137 {{ snippets.c#link_to_multiplexer }}
139 ## Capabilities of a pad
141 Since the pads play a very important role in how the element is viewed
142 by the outside world, a mechanism is implemented to describe the data
143 that can flow or currently flows through the pad by using capabilities.
144 Here, we will briefly describe what capabilities are and how to use
145 them, enough to get an understanding of the concept. For an in-depth
146 look into capabilities and a list of all capabilities defined in
147 GStreamer, see the [Plugin Writers
148 Guide](http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/index.html).
150 Capabilities are attached to pad templates and to pads. For pad
151 templates, it will describe the types of media that may stream over a
152 pad created from this template. For pads, it can either be a list of
153 possible caps (usually a copy of the pad template's capabilities), in
154 which case the pad is not yet negotiated, or it is the type of media
155 that currently streams over this pad, in which case the pad has been
158 ### Dissecting capabilities
160 A pad's capabilities are described in a `GstCaps` object. Internally, a
161 [`GstCaps`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstCaps.html)
162 will contain one or more
163 [`GstStructure`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstStructure.html)
164 that will describe one media type. A negotiated pad will have
165 capabilities set that contain exactly *one* structure. Also, this
166 structure will contain only *fixed* values. These constraints are not
167 true for unnegotiated pads or pad templates.
169 As an example, below is a dump of the capabilities of the “vorbisdec”
170 element, which you will get by running `gst-inspect vorbisdec`. You will
171 see two pads: a source and a sink pad. Both of these pads are always
172 available, and both have capabilities attached to them. The sink pad
173 will accept vorbis-encoded audio data, with the media type
174 “audio/x-vorbis”. The source pad will be used to send raw (decoded)
175 audio samples to the next element, with a raw audio media type (in this
176 case, “audio/x-raw”). The source pad will also contain properties for
177 the audio samplerate and the amount of channels, plus some more that you
178 don't need to worry about for now.
188 rate: [ 1, 2147483647 ]
191 SINK template: 'sink'
198 ### Properties and values
200 Properties are used to describe extra information for capabilities. A
201 property consists of a key (a string) and a value. There are different
202 possible value types that can be used:
204 - Basic types, this can be pretty much any `GType` registered with
205 Glib. Those properties indicate a specific, non-dynamic value for
206 this property. Examples include:
208 - An integer value (`G_TYPE_INT`): the property has this exact
211 - A boolean value (`G_TYPE_BOOLEAN`): the property is either TRUE
214 - A float value (`G_TYPE_FLOAT`): the property has this exact
215 floating point value.
217 - A string value (`G_TYPE_STRING`): the property contains a UTF-8
220 - A fraction value (`GST_TYPE_FRACTION`): contains a fraction
221 expressed by an integer numerator and denominator.
223 - Range types are `GType`s registered by GStreamer to indicate a range
224 of possible values. They are used for indicating allowed audio
225 samplerate values or supported video sizes. The two types defined in
228 - An integer range value (`GST_TYPE_INT_RANGE`): the property
229 denotes a range of possible integers, with a lower and an upper
230 boundary. The “vorbisdec” element, for example, has a rate
231 property that can be between 8000 and 50000.
233 - A float range value (`GST_TYPE_FLOAT_RANGE`): the property
234 denotes a range of possible floating point values, with a lower
235 and an upper boundary.
237 - A fraction range value (`GST_TYPE_FRACTION_RANGE`): the property
238 denotes a range of possible fraction values, with a lower and an
241 - A list value (`GST_TYPE_LIST`): the property can take any value from
242 a list of basic values given in this list.
244 Example: caps that express that either a sample rate of 44100 Hz and
245 a sample rate of 48000 Hz is supported would use a list of integer
246 values, with one value being 44100 and one value being 48000.
248 - An array value (`GST_TYPE_ARRAY`): the property is an array of
249 values. Each value in the array is a full value on its own, too. All
250 values in the array should be of the same elementary type. This
251 means that an array can contain any combination of integers, lists
252 of integers, integer ranges together, and the same for floats or
253 strings, but it can not contain both floats and ints at the same
256 Example: for audio where there are more than two channels involved
257 the channel layout needs to be specified (for one and two channel
258 audio the channel layout is implicit unless stated otherwise in the
259 caps). So the channel layout would be an array of integer enum
260 values where each enum value represents a loudspeaker position.
261 Unlike a `GST_TYPE_LIST`, the values in an array will be interpreted
264 ## What capabilities are used for
266 Capabilities (short: caps) describe the type of data that is streamed
267 between two pads, or that one pad (template) supports. This makes them
268 very useful for various purposes:
270 - Autoplugging: automatically finding elements to link to a pad based
271 on its capabilities. All autopluggers use this method.
273 - Compatibility detection: when two pads are linked, GStreamer can
274 verify if the two pads are talking about the same media type. The
275 process of linking two pads and checking if they are compatible is
276 called “caps negotiation”.
278 - Metadata: by reading the capabilities from a pad, applications can
279 provide information about the type of media that is being streamed
280 over the pad, which is information about the stream that is
281 currently being played back.
283 - Filtering: an application can use capabilities to limit the possible
284 media types that can stream between two pads to a specific subset of
285 their supported stream types. An application can, for example, use
286 “filtered caps” to set a specific (fixed or non-fixed) video size
287 that should stream between two pads. You will see an example of
288 filtered caps later in this manual, in [Manually adding or removing
290 pipeline](manual/advanced/dataaccess.md#manually-adding-or-removing-data-fromto-a-pipeline).
291 You can do caps filtering by inserting a capsfilter element into
292 your pipeline and setting its “caps” property. Caps filters are
293 often placed after converter elements like audioconvert,
294 audioresample, videoconvert or videoscale to force those converters
295 to convert data to a specific output format at a certain point in a
298 ### Using capabilities for metadata
300 A pad can have a set (i.e. one or more) of capabilities attached to it.
301 Capabilities (`GstCaps`) are represented as an array of one or more
302 `GstStructure`s, and each `GstStructure` is an array of fields where
303 each field consists of a field name string (e.g. "width") and a typed
304 value (e.g. `G_TYPE_INT` or `GST_TYPE_INT_RANGE`).
306 Note that there is a distinct difference between the *possible*
307 capabilities of a pad (ie. usually what you find as caps of pad
308 templates as they are shown in gst-inspect), the *allowed* caps of a pad
309 (can be the same as the pad's template caps or a subset of them,
310 depending on the possible caps of the peer pad) and lastly *negotiated*
311 caps (these describe the exact format of a stream or buffer and contain
312 exactly one structure and have no variable bits like ranges or lists,
313 ie. they are fixed caps).
315 You can get values of properties in a set of capabilities by querying
316 individual properties of one structure. You can get a structure from a
317 caps using `gst_caps_get_structure ()` and the number of structures in a
318 `GstCaps` using `gst_caps_get_size ()`.
320 Caps are called *simple caps* when they contain only one structure, and
321 *fixed caps* when they contain only one structure and have no variable
322 field types (like ranges or lists of possible values). Two other special
323 types of caps are *ANY caps* and *empty caps*.
325 Here is an example of how to extract the width and height from a set of
330 read_video_props (GstCaps *caps)
333 const GstStructure *str;
335 g_return_if_fail (gst_caps_is_fixed (caps));
337 str = gst_caps_get_structure (caps, 0);
338 if (!gst_structure_get_int (str, "width", &width) ||
339 !gst_structure_get_int (str, "height", &height)) {
340 g_print ("No width/height available\n");
344 g_print ("The video size of this set of capabilities is %dx%d\n",
350 ### Creating capabilities for filtering
352 While capabilities are mainly used inside a plugin to describe the media
353 type of the pads, the application programmer often also has to have
354 basic understanding of capabilities in order to interface with the
355 plugins, especially when using filtered caps. When you're using filtered
356 caps or fixation, you're limiting the allowed types of media that can
357 stream between two pads to a subset of their supported media types. You
358 do this using a `capsfilter` element in your pipeline. In order to do
359 this, you also need to create your own `GstCaps`. The easiest way to do
360 this is by using the convenience function `gst_caps_new_simple ()`:
364 link_elements_with_filter (GstElement *element1, GstElement *element2)
369 caps = gst_caps_new_simple ("video/x-raw",
370 "format", G_TYPE_STRING, "I420",
371 "width", G_TYPE_INT, 384,
372 "height", G_TYPE_INT, 288,
373 "framerate", GST_TYPE_FRACTION, 25, 1,
376 link_ok = gst_element_link_filtered (element1, element2, caps);
377 gst_caps_unref (caps);
380 g_warning ("Failed to link element1 and element2!");
388 This will force the data flow between those two elements to a certain
389 video format, width, height and framerate (or the linking will fail if
390 that cannot be achieved in the context of the elements involved). Keep
391 in mind that when you use `
392 gst_element_link_filtered ()` it will automatically create a
393 `capsfilter` element for you and insert it into your bin or pipeline
394 between the two elements you want to connect (this is important if you
395 ever want to disconnect those elements because then you will have to
396 disconnect both elements from the capsfilter instead).
398 In some cases, you will want to create a more elaborate set of
399 capabilities to filter a link between two pads. Then, this function is
400 too simplistic and you'll want to use the method `gst_caps_new_full ()`:
404 link_elements_with_filter (GstElement *element1, GstElement *element2)
409 caps = gst_caps_new_full (
410 gst_structure_new ("video/x-raw",
411 "width", G_TYPE_INT, 384,
412 "height", G_TYPE_INT, 288,
413 "framerate", GST_TYPE_FRACTION, 25, 1,
415 gst_structure_new ("video/x-bayer",
416 "width", G_TYPE_INT, 384,
417 "height", G_TYPE_INT, 288,
418 "framerate", GST_TYPE_FRACTION, 25, 1,
422 link_ok = gst_element_link_filtered (element1, element2, caps);
423 gst_caps_unref (caps);
426 g_warning ("Failed to link element1 and element2!");
434 See the API references for the full API of
435 [`GstStructure`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstStructure.html)
437 [`GstCaps`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/gstreamer-GstCaps.html).
441 You can see from [Visualisation of a GstBin element without ghost
442 pads](#visualisation-of-a-gstbin-------element-without-ghost-pads) how a
443 bin has no pads of its own. This is where "ghost pads" come into play.
446 [`GstBin`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstBin.html)
447 element without ghost pads](images/bin-element-noghost.png "fig:")
449 A ghost pad is a pad from some element in the bin that can be accessed
450 directly from the bin as well. Compare it to a symbolic link in UNIX
451 filesystems. Using ghost pads on bins, the bin also has a pad and can
452 transparently be used as an element in other parts of your code.
455 [`GstBin`](http://gstreamer.freedesktop.org/data/doc/gstreamer/stable/gstreamer/html/GstBin.html)
456 element with a ghost pad](images/bin-element-ghost.png "fig:")
458 [Visualisation of a GstBin element with a ghost
459 pad](#visualisation-of-a-gstbin-------element-with-a-ghost-pad) is a
460 representation of a ghost pad. The sink pad of element one is now also a
461 pad of the bin. Because ghost pads look and work like any other pads,
462 they can be added to any type of elements, not just to a `GstBin`, just
465 A ghostpad is created using the function `gst_ghost_pad_new ()`:
474 GstElement *bin, *sink;
478 gst_init (&argc, &argv);
480 /* create element, add to bin */
481 sink = gst_element_factory_make ("fakesink", "sink");
482 bin = gst_bin_new ("mybin");
483 gst_bin_add (GST_BIN (bin), sink);
486 pad = gst_element_get_static_pad (sink, "sink");
487 gst_element_add_pad (bin, gst_ghost_pad_new ("sink", pad));
488 gst_object_unref (GST_OBJECT (pad));
496 In the above example, the bin now also has a pad: the pad called “sink”
497 of the given element. The bin can, from here on, be used as a substitute
498 for the sink element. You could, for example, link another element to
501 1. In reality, there is no objection to data flowing from a source pad
502 to the sink pad of an element upstream (to the left of this element
503 in drawings). Data will, however, always flow from a source pad of
504 one element to the sink pad of another.