1 # Basic tutorial 6: Media formats and Pad Capabilities
5 Pad Capabilities are a fundamental element of GStreamer, although most
6 of the time they are invisible because the framework handles them
7 automatically. This somewhat theoretical tutorial shows:
9 - What are Pad Capabilities.
11 - How to retrieve them.
13 - When to retrieve them.
15 - Why you need to know about them.
21 As it has already been shown, Pads allow information to enter and leave
22 an element. The *Capabilities* (or *Caps*, for short) of a Pad, then,
23 specify what kind of information can travel through the Pad. For
24 example, “RGB video with a resolution of 320x200 pixels and 30 frames
25 per second”, or “16-bits per sample audio, 5.1 channels at 44100 samples
26 per second”, or even compressed formats like mp3 or h264.
28 Pads can support multiple Capabilities (for example, a video sink can
29 support video in different types of RGB or YUV formats) and Capabilities can be
30 specified as *ranges* (for example, an audio sink can support samples
31 rates from 1 to 48000 samples per second). However, the actual
32 information traveling from Pad to Pad must have only one well-specified
33 type. Through a process known as *negotiation*, two linked Pads agree on
34 a common type, and thus the Capabilities of the Pads become *fixed*
35 (they only have one type and do not contain ranges). The walkthrough of
36 the sample code below should make all this clear.
38 **In order for two elements to be linked together, they must share a
39 common subset of Capabilities** (Otherwise they could not possibly
40 understand each other). This is the main goal of Capabilities.
42 As an application developer, you will usually build pipelines by linking
43 elements together (to a lesser extent if you use all-in-all elements
44 like `playbin`). In this case, you need to know the *Pad Caps* (as they
45 are familiarly referred to) of your elements, or, at least, know what
46 they are when GStreamer refuses to link two elements with a negotiation
51 Pads are created from *Pad Templates*, which indicate all possible
52 Capabilities a Pad could ever have. Templates are useful to create several
53 similar Pads, and also allow early refusal of connections between
54 elements: If the Capabilities of their Pad Templates do not have a
55 common subset (their *intersection* is empty), there is no need to
58 Pad Templates can be viewed as the first step in the negotiation
59 process. As the process evolves, actual Pads are instantiated and their
60 Capabilities refined until they are fixed (or negotiation fails).
62 ### Capabilities examples
70 rate: [ 1, 2147483647 ]
74 rate: [ 1, 2147483647 ]
78 This pad is a sink which is always available on the element (we will not
79 talk about availability for now). It supports two kinds of media, both
80 raw audio in integer format (`audio/x-raw`): signed, 16-bit little endian and
81 unsigned 8-bit. The square brackets indicate a range: for instance, the
82 number of channels varies from 1 to 2.
89 width: [ 1, 2147483647 ]
90 height: [ 1, 2147483647 ]
91 framerate: [ 0/1, 2147483647/1 ]
92 format: { I420, NV12, NV21, YV12, YUY2, Y42B, Y444, YUV9, YVU9, Y41B, Y800, Y8, GREY, Y16 , UYVY, YVYU, IYU1, v308, AYUV, A420 }
95 `video/x-raw` indicates that this source pad outputs raw video. It
96 supports a wide range of dimensions and framerates, and a set of YUV
97 formats (The curly braces indicate a *list*). All these formats
98 indicate different packing and subsampling of the image planes.
102 You can use the `gst-inspect-1.0` tool described in [Basic tutorial 10:
103 GStreamer tools](tutorial-basic-gstreamer-tools.md) to
104 learn about the Caps of any GStreamer element.
106 Bear in mind that some elements query the underlying hardware for
107 supported formats and offer their Pad Caps accordingly (They usually do
108 this when entering the READY state or higher). Therefore, the shown caps
109 can vary from platform to platform, or even from one execution to the
110 next (even though this case is rare).
112 This tutorial instantiates two elements (this time, through their
113 factories), shows their Pad Templates, links them and sets the pipeline
114 to play. On each state change, the Capabilities of the sink element's
115 Pad are shown, so you can observe how the negotiation proceeds until the
118 ## A trivial Pad Capabilities Example
120 Copy this code into a text file named `basic-tutorial-6.c` (or find it
121 in the SDK installation).
123 **basic-tutorial-6.c**
128 /* Functions below print the Capabilities in a human-friendly format */
129 static gboolean print_field (GQuark field, const GValue * value, gpointer pfx) {
130 gchar *str = gst_value_serialize (value);
132 g_print ("%s %15s: %s\n", (gchar *) pfx, g_quark_to_string (field), str);
137 static void print_caps (const GstCaps * caps, const gchar * pfx) {
140 g_return_if_fail (caps != NULL);
142 if (gst_caps_is_any (caps)) {
143 g_print ("%sANY\n", pfx);
146 if (gst_caps_is_empty (caps)) {
147 g_print ("%sEMPTY\n", pfx);
151 for (i = 0; i < gst_caps_get_size (caps); i++) {
152 GstStructure *structure = gst_caps_get_structure (caps, i);
154 g_print ("%s%s\n", pfx, gst_structure_get_name (structure));
155 gst_structure_foreach (structure, print_field, (gpointer) pfx);
159 /* Prints information about a Pad Template, including its Capabilities */
160 static void print_pad_templates_information (GstElementFactory * factory) {
162 GstStaticPadTemplate *padtemplate;
164 g_print ("Pad Templates for %s:\n", gst_element_factory_get_longname (factory));
165 if (!gst_element_factory_get_num_pad_templates (factory)) {
170 pads = gst_element_factory_get_static_pad_templates (factory);
172 padtemplate = pads->data
173 pads = g_list_next (pads);
175 if (padtemplate->direction == GST_PAD_SRC)
176 g_print (" SRC template: '%s'\n", padtemplate->name_template);
177 else if (padtemplate->direction == GST_PAD_SINK)
178 g_print (" SINK template: '%s'\n", padtemplate->name_template);
180 g_print (" UNKNOWN!!! template: '%s'\n", padtemplate->name_template);
182 if (padtemplate->presence == GST_PAD_ALWAYS)
183 g_print (" Availability: Always\n");
184 else if (padtemplate->presence == GST_PAD_SOMETIMES)
185 g_print (" Availability: Sometimes\n");
186 else if (padtemplate->presence == GST_PAD_REQUEST) {
187 g_print (" Availability: On request\n");
189 g_print (" Availability: UNKNOWN!!!\n");
191 if (padtemplate->static_caps.string) {
193 g_print (" Capabilities:\n");
194 caps = gst_static_caps_get (&padtemplate->static_caps);
195 print_caps (caps, " ");
196 gst_caps_unref (caps);
204 /* Shows the CURRENT capabilities of the requested pad in the given element */
205 static void print_pad_capabilities (GstElement *element, gchar *pad_name) {
207 GstCaps *caps = NULL;
210 pad = gst_element_get_static_pad (element, pad_name);
212 g_printerr ("Could not retrieve pad '%s'\n", pad_name);
216 /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */
217 caps = gst_pad_get_current_caps (pad);
219 caps = gst_pad_query_caps (pad, NULL);
222 g_print ("Caps for the %s pad:\n", pad_name);
223 print_caps (caps, " ");
224 gst_caps_unref (caps);
225 gst_object_unref (pad);
228 int main(int argc, char *argv[]) {
229 GstElement *pipeline, *source, *sink;
230 GstElementFactory *source_factory, *sink_factory;
233 GstStateChangeReturn ret;
234 gboolean terminate = FALSE;
236 /* Initialize GStreamer */
237 gst_init (&argc, &argv);
239 /* Create the element factories */
240 source_factory = gst_element_factory_find ("audiotestsrc");
241 sink_factory = gst_element_factory_find ("autoaudiosink");
242 if (!source_factory || !sink_factory) {
243 g_printerr ("Not all element factories could be created.\n");
247 /* Print information about the pad templates of these factories */
248 print_pad_templates_information (source_factory);
249 print_pad_templates_information (sink_factory);
251 /* Ask the factories to instantiate actual elements */
252 source = gst_element_factory_create (source_factory, "source");
253 sink = gst_element_factory_create (sink_factory, "sink");
255 /* Create the empty pipeline */
256 pipeline = gst_pipeline_new ("test-pipeline");
258 if (!pipeline || !source || !sink) {
259 g_printerr ("Not all elements could be created.\n");
263 /* Build the pipeline */
264 gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
265 if (gst_element_link (source, sink) != TRUE) {
266 g_printerr ("Elements could not be linked.\n");
267 gst_object_unref (pipeline);
271 /* Print initial negotiated caps (in NULL state) */
272 g_print ("In NULL state:\n");
273 print_pad_capabilities (sink, "sink");
276 ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
277 if (ret == GST_STATE_CHANGE_FAILURE) {
278 g_printerr ("Unable to set the pipeline to the playing state (check the bus for error messages).\n");
281 /* Wait until error, EOS or State Change */
282 bus = gst_element_get_bus (pipeline);
284 msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS |
285 GST_MESSAGE_STATE_CHANGED);
292 switch (GST_MESSAGE_TYPE (msg)) {
293 case GST_MESSAGE_ERROR:
294 gst_message_parse_error (msg, &err, &debug_info);
295 g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
296 g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
297 g_clear_error (&err);
301 case GST_MESSAGE_EOS:
302 g_print ("End-Of-Stream reached.\n");
305 case GST_MESSAGE_STATE_CHANGED:
306 /* We are only interested in state-changed messages from the pipeline */
307 if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {
308 GstState old_state, new_state, pending_state;
309 gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
310 g_print ("\nPipeline state changed from %s to %s:\n",
311 gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
312 /* Print the current capabilities of the sink element */
313 print_pad_capabilities (sink, "sink");
317 /* We should not reach here because we only asked for ERRORs, EOS and STATE_CHANGED */
318 g_printerr ("Unexpected message received.\n");
321 gst_message_unref (msg);
323 } while (!terminate);
326 gst_object_unref (bus);
327 gst_element_set_state (pipeline, GST_STATE_NULL);
328 gst_object_unref (pipeline);
329 gst_object_unref (source_factory);
330 gst_object_unref (sink_factory);
335 > ![Information](images/icons/emoticons/information.png)
338 > If you need help to compile this code, refer to the **Building the tutorials** section for your platform: [Linux](installing-on-linux.md#InstallingonLinux-Build), [Mac OS X](installing-on-mac-osx.md#InstallingonMacOSX-Build) or [Windows](installing-on-windows.md#InstallingonWindows-Build), or use this specific command on Linux:
340 > `` gcc basic-tutorial-6.c -o basic-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0` ``
342 >If you need help to run this code, refer to the **Running the tutorials** section for your platform: [Linux](installing-on-linux.md#InstallingonLinux-Run), [Mac OS X](installing-on-mac-osx.md#InstallingonMacOSX-Run) or [Windows](installing-on-windows.md#InstallingonWindows-Run).
344 > This tutorial simply displays information regarding the Pad Capabilities in different time instants.
346 > Required libraries: `gstreamer-1.0`
350 The `print_field`, `print_caps` and `print_pad_templates` simply
351 display, in a human-friendly format, the capabilities structures. If you
352 want to learn about the internal organization of the
353 `GstCaps` structure, read the `GStreamer Documentation` regarding Pad
357 /* Shows the CURRENT capabilities of the requested pad in the given element */
358 static void print_pad_capabilities (GstElement *element, gchar *pad_name) {
360 GstCaps *caps = NULL;
363 pad = gst_element_get_static_pad (element, pad_name);
365 g_printerr ("Could not retrieve pad '%s'\n", pad_name);
369 /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */
370 caps = gst_pad_get_current_caps (pad);
372 caps = gst_pad_query_caps (pad, NULL);
375 g_print ("Caps for the %s pad:\n", pad_name);
376 print_caps (caps, " ");
377 gst_caps_unref (caps);
378 gst_object_unref (pad);
382 `gst_element_get_static_pad()` retrieves the named Pad from the given
383 element. This Pad is *static* because it is always present in the
384 element. To know more about Pad availability read the `GStreamer
385 documentation` about Pads.
387 Then we call `gst_pad_get_current_caps()` to retrieve the Pad's
388 current Capabilities, which can be fixed or not, depending on the state
389 of the negotiation process. They could even be non-existent, in which
390 case, we call `gst_pad_query_caps()` to retrieve the currently
391 acceptable Pad Capabilities. The currently acceptable Caps will be the
392 Pad Template's Caps in the NULL state, but might change in later states,
393 as the actual hardware Capabilities might be queried.
395 We then print these Capabilities.
398 /* Create the element factories */
399 source_factory = gst_element_factory_find ("audiotestsrc");
400 sink_factory = gst_element_factory_find ("autoaudiosink");
401 if (!source_factory || !sink_factory) {
402 g_printerr ("Not all element factories could be created.\n");
406 /* Print information about the pad templates of these factories */
407 print_pad_templates_information (source_factory);
408 print_pad_templates_information (sink_factory);
410 /* Ask the factories to instantiate actual elements */
411 source = gst_element_factory_create (source_factory, "source");
412 sink = gst_element_factory_create (sink_factory, "sink");
415 In the previous tutorials we created the elements directly using
416 `gst_element_factory_make()` and skipped talking about factories, but we
417 will do now. A `GstElementFactory` is in charge of instantiating a
418 particular type of element, identified by its factory name.
420 You can use `gst_element_factory_find()` to create a factory of type
421 “videotestsrc”, and then use it to instantiate multiple “videotestsrc”
422 elements using `gst_element_factory_create()`.
423 `gst_element_factory_make()` is really a shortcut for
424 `gst_element_factory_find()`+ `gst_element_factory_create()`.
426 The Pad Templates can already be accessed through the factories, so they
427 are printed as soon as the factories are created.
429 We skip the pipeline creation and start, and go to the State-Changed
433 case GST_MESSAGE_STATE_CHANGED:
434 /* We are only interested in state-changed messages from the pipeline */
435 if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {
436 GstState old_state, new_state, pending_state;
437 gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
438 g_print ("\nPipeline state changed from %s to %s:\n",
439 gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
440 /* Print the current capabilities of the sink element */
441 print_pad_capabilities (sink, "sink");
446 This simply prints the current Pad Caps every time the state of the
447 pipeline changes. You should see, in the output, how the initial caps
448 (the Pad Template's Caps) are progressively refined until they are
449 completely fixed (they contain a single type with no ranges).
453 This tutorial has shown:
455 - What are Pad Capabilities and Pad Template Capabilities.
457 - How to retrieve them
458 with `gst_pad_get_current_caps()` or `gst_pad_query_caps()`.
460 - That they have different meaning depending on the state of the
461 pipeline (initially they indicate all the possible Capabilities,
462 later they indicate the currently negotiated Caps for the Pad).
464 - That Pad Caps are important to know beforehand if two elements can
467 - That Pad Caps can be found using the `gst-inspect-1.0` tool described
468 in [Basic tutorial 10: GStreamer
469 tools](tutorial-basic-gstreamer-tools.md).
471 Next tutorial shows how data can be manually injected into and extracted
472 from the GStreamer pipeline.
474 Remember that attached to this page you should find the complete source
475 code of the tutorial and any accessory files needed to build it.
476 It has been a pleasure having you here, and see you soon!