1 <chapter id="chapter-autoplugging">
2 <title>Putting together a pipeline</title>
4 The small application we created in the previous chapter used the
5 concept of a factory to create the elements. In this chapter we will
6 show you how to use the factory concepts to create elements based
7 on what they do instead of what they are called.
11 We will first explain the concepts involved before we move on
12 to the reworked helloworld example using autoplugging.
14 <sect1 id="section-factories-helloworld-problems">
15 <title>The problems with the helloworld example</title>
17 If we take a look at how the elements were created in the previous
18 example we used a rather crude mechanism:
23 /* now it's time to get the parser */
24 decoder = gst_element_factory_make ("mad", "decoder");
29 While this mechanism is quite effective it also has some big problems:
30 The elements are created based on their name. Indeed, we create an
31 element, mad, by explicitly stating the mad element's name. Our little
32 program therefore always uses the mad decoder element to decode
33 the MP3 audio stream, even if there are three other MP3 decoders in the
34 system. We will see how we can use a more general way to create an
38 We have to introduce the concept of MIME types and capabilities
39 added to the source and sink pads.
43 <sect1 id="section-factories-mime">
44 <title>More on MIME Types</title>
46 GStreamer uses MIME types to identify the different types of data
47 that can be handled by the elements. They are the high level
48 mechanisms to make sure that everyone is talking about the right
52 A MIME (Multipurpose Internet Mail Extension) type is a pair of
53 strings that denote a certain type of data. Examples include:
57 audio/x-raw-int : raw audio samples
62 audio/mpeg : MPEG audio
67 video/mpeg : MPEG video
73 An element must associate a MIME type to its source and sink pads
74 when it is loaded into the system. GStreamer knows about the
75 different elements and what type of data they expect and emit.
76 This allows for very dynamic and extensible element creation as we
80 As we have seen in the previous chapter, MIME types are added
81 to the Capability structure of a pad.
85 <xref linkend="section-mime-img"/> shows the MIME types associated with
86 each pad from the "hello world" example.
88 <figure float="1" id="section-mime-img">
89 <title>The Hello world pipeline with MIME types</title>
92 <imagedata fileref="images/mime-world.ℑ" format="&IMAGE;" />
98 We will see how you can create an element based on the MIME types
99 of its source and sink pads. This way the end-user will have the
100 ability to choose his/her favorite audio/mpeg decoder without
101 you even having to care about it.
104 The typing of the source and sink pads also makes it possible to
105 'autoplug' a pipeline. We will have the ability to say: "construct
106 a pipeline that does an audio/mpeg to audio/x-raw-int conversion".
110 The basic GStreamer library does not try to solve all of your
111 autoplug problems. It leaves the hard decisions to the application
112 programmer, where they belong.
118 <sect1 id="section-factories-gstreamer-types">
119 <title>GStreamer types</title>
121 GStreamer assigns a unique number to all registered MIME types.
122 GStreamer also keeps a reference to
123 a function that can be used to determine if a given buffer is of
127 There is also an association between a MIME type and a file extension,
128 but the use of typefind functions (similar to file(1)) is preferred.
131 The type information is maintained in a list of
132 <classname>GstType</classname>. The definition of a
133 <classname>GstType</classname> is like:
137 typedef GstCaps (*GstTypeFindFunc) (GstBuffer *buf,gpointer *priv);
139 typedef struct _GstType GstType;
142 guint16 id; /* type id (assigned) */
144 gchar *mime; /* MIME type */
145 gchar *exts; /* space-delimited list of extensions */
147 GstTypeFindFunc typefindfunc; /* typefind function */
152 All operations on <classname>GstType</classname> occur
153 via their <classname>guint16 id</classname> numbers, with
154 the <classname>GstType</classname> structure private to the GStreamer
159 <title>MIME type to id conversion</title>
162 We can obtain the id for a given MIME type
163 with the following piece of code:
168 id = gst_type_find_by_mime ("audio/mpeg");
171 This function will return 0 if the type was not known.
176 <title>id to <classname>GstType</classname> conversion</title>
178 We can obtain the <classname>GstType</classname> for a given id
179 with the following piece of code:
184 type = gst_type_find_by_id (id);
187 This function will return NULL if the id was not associated with
188 any known <classname>GstType</classname>
193 <title>extension to id conversion</title>
195 We can obtain the id for a given file extension
196 with the following piece of code:
201 id = gst_type_find_by_ext (".mp3");
204 This function will return 0 if the extension was not known.
209 <sect1 id="section-factories-create">
210 <title>Creating elements with the factory</title>
212 In the previous section we described how you could obtain
213 an element factory using MIME types. One the factory has been
214 obtained, you can create an element using:
217 GstElementFactory *factory;
220 // obtain the factory
223 element = gst_element_factory_create (factory, "name");
226 This way, you do not have to create elements by name which
227 allows the end-user to select the elements he/she prefers for the
232 <sect1 id="section-factories-basic-types">
233 <title>GStreamer basic types</title>
235 GStreamer only has two builtin types:
240 audio/raw : raw audio samples
245 video/raw and image/raw : raw video data
250 All other MIME types are maintained by the plugin elements.
255 <sect1 id="chapter-dynamic">
256 <title>Dynamic pipelines</title>
258 In this chapter we will see how you can create a dynamic pipeline. A
259 dynamic pipeline is a pipeline that is updated or created while data
260 is flowing through it. We will create a partial pipeline first and add
261 more elements while the pipeline is playing. Dynamic pipelines cause
262 all sorts of scheduling issues and will remain a topic of research for
263 a long time in GStreamer.
266 We will show how to create an MPEG1 video player using dynamic pipelines.
267 As you have seen in the pad section, we can attach a signal to an element
268 when a pad is created. We will use this to create our MPEG1 player.
272 We'll start with a simple main function:
276 /* example-begin dynamic.c */
277 #include <string.h>
278 #include <gst/gst.h>
281 eof (GstElement *src)
283 g_print ("have eos, quitting\n");
288 idle_func (gpointer data)
290 gst_bin_iterate (GST_BIN (data));
295 new_pad_created (GstElement *parse, GstPad *pad, GstElement *pipeline)
297 GstElement *decode_video = NULL;
298 GstElement *decode_audio, *play, *color, *show;
299 GstElement *audio_queue, *video_queue;
300 GstElement *audio_thread, *video_thread;
302 g_print ("***** a new pad %s was created\n", gst_pad_get_name (pad));
304 gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PAUSED);
306 /* link to audio pad */
307 if (strncmp (gst_pad_get_name (pad), "audio_", 6) == 0) {
309 /* construct internal pipeline elements */
310 decode_audio = gst_element_factory_make ("mad", "decode_audio");
311 g_return_if_fail (decode_audio != NULL);
312 play = gst_element_factory_make ("osssink", "play_audio");
313 g_return_if_fail (play != NULL);
315 /* create the thread and pack stuff into it */
316 audio_thread = gst_thread_new ("audio_thread");
317 g_return_if_fail (audio_thread != NULL);
319 /* construct queue and link everything in the main pipeline */
320 audio_queue = gst_element_factory_make ("queue", "audio_queue");
321 g_return_if_fail (audio_queue != NULL);
323 gst_bin_add_many (GST_BIN (audio_thread),
324 audio_queue, decode_audio, play, NULL);
326 /* set up pad links */
327 gst_element_add_ghost_pad (audio_thread,
328 gst_element_get_pad (audio_queue, "sink"),
330 gst_element_link (audio_queue, decode_audio);
331 gst_element_link (decode_audio, play);
333 gst_bin_add (GST_BIN (pipeline), audio_thread);
335 gst_pad_link (pad, gst_element_get_pad (audio_thread, "sink"));
337 /* set up thread state and kick things off */
338 g_print ("setting to READY state\n");
339 gst_element_set_state (GST_ELEMENT (audio_thread), GST_STATE_READY);
342 else if (strncmp (gst_pad_get_name (pad), "video_", 6) == 0) {
344 /* construct internal pipeline elements */
345 decode_video = gst_element_factory_make ("mpeg2dec", "decode_video");
346 g_return_if_fail (decode_video != NULL);
348 color = gst_element_factory_make ("colorspace", "color");
349 g_return_if_fail (color != NULL);
352 show = gst_element_factory_make ("xvideosink", "show");
353 g_return_if_fail (show != NULL);
355 /* construct queue and link everything in the main pipeline */
356 video_queue = gst_element_factory_make ("queue", "video_queue");
357 g_return_if_fail (video_queue != NULL);
359 /* create the thread and pack stuff into it */
360 video_thread = gst_thread_new ("video_thread");
361 g_return_if_fail (video_thread != NULL);
362 gst_bin_add_many (GST_BIN (video_thread), video_queue,
363 decode_video, color, show, NULL);
365 /* set up pad links */
366 gst_element_add_ghost_pad (video_thread,
367 gst_element_get_pad (video_queue, "sink"),
369 gst_element_link (video_queue, decode_video);
370 gst_element_link_many (decode_video, color, show, NULL);
372 gst_bin_add (GST_BIN (pipeline), video_thread);
374 gst_pad_link (pad, gst_element_get_pad (video_thread, "sink"));
376 /* set up thread state and kick things off */
377 g_print ("setting to READY state\n");
378 gst_element_set_state (GST_ELEMENT (video_thread), GST_STATE_READY);
380 gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);
384 main (int argc, char *argv[])
386 GstElement *pipeline, *src, *demux;
388 gst_init (&argc, &argv);
390 pipeline = gst_pipeline_new ("pipeline");
391 g_return_val_if_fail (pipeline != NULL, -1);
393 src = gst_element_factory_make ("filesrc", "src");
394 g_return_val_if_fail (src != NULL, -1);
396 g_error ("Please specify a video file to play !");
398 g_object_set (G_OBJECT (src), "location", argv[1], NULL);
400 demux = gst_element_factory_make ("mpegdemux", "demux");
401 g_return_val_if_fail (demux != NULL, -1);
403 gst_bin_add_many (GST_BIN (pipeline), src, demux, NULL);
405 g_signal_connect (G_OBJECT (demux), "new_pad",
406 G_CALLBACK (new_pad_created), pipeline);
408 g_signal_connect (G_OBJECT (src), "eos",
409 G_CALLBACK (eof), NULL);
411 gst_element_link (src, demux);
413 gst_element_set_state (GST_ELEMENT (pipeline), GST_STATE_PLAYING);
415 g_idle_add (idle_func, pipeline);
421 /* example-end dynamic.c */
424 We create two elements: a file source and an MPEG demuxer.
425 There's nothing special about this piece of code except for
426 the signal 'new_pad' that we linked to the mpegdemux
430 g_signal_connect (G_OBJECT (demux), "new_pad",
431 G_CALLBACK (new_pad_created), pipeline);
434 When an elementary stream has been detected in the system stream,
435 mpegdemux will create a new pad that will provide the data of the
436 elementary stream. A function 'new_pad_created' will be called when
440 In the above example, we created new elements based on the name of
441 the newly created pad. We then added them to a new thread.
442 There are other possibilities to check the type of the pad, for
443 example by using the MIME type and the properties of the pad.
448 <sect1 id="chapter-typedetection">
449 <title>Type Detection</title>
451 Sometimes the capabilities of a pad are not specificied. The filesrc
452 element, for example, does not know what type of file it is reading. Before
453 you can attach an element to the pad of the filesrc, you need to determine
454 the media type in order to be able to choose a compatible element.
457 To solve this problem, a plugin can provide the <application>GStreamer</application>
458 core library with a type definition. The type definition
459 will contain the following information:
463 The MIME type we are going to define.
468 An optional string with a list of possible file extensions this
469 type usually is associated with. the list entries are separated with
470 a space. eg, ".mp3 .mpa .mpg".
475 An optional typefind function.
481 The typefind functions give a meaning to the MIME types that are used
482 in GStreamer. The typefind function is a function with the following definition:
485 typedef GstCaps *(*GstTypeFindFunc) (GstBuffer *buf, gpointer priv);
488 This typefind function will inspect a GstBuffer with data and will output
489 a GstCaps structure describing the type. If the typefind function does not
490 understand the buffer contents, it will return NULL.
493 <application>GStreamer</application> has a typefind element in the set
495 that can be used to determine the type of a given pad.
498 The next example will show how a typefind element can be inserted into a pipeline
499 to detect the media type of a file. It will output the capabilities of the pad into
500 an XML representation.
503 #include <gst/gst.h>
505 void type_found (GstElement *typefind, GstCaps* caps);
508 main(int argc, char *argv[])
510 GstElement *bin, *filesrc, *typefind;
512 gst_init (&argc, &argv);
515 g_print ("usage: %s <filename>\n", argv[0]);
519 /* create a new bin to hold the elements */
520 bin = gst_bin_new ("bin");
521 g_assert (bin != NULL);
523 /* create a disk reader */
524 filesrc = gst_element_factory_make ("filesrc", "disk_source");
525 g_assert (filesrc != NULL);
526 g_object_set (G_OBJECT (filesrc), "location", argv[1], NULL);
528 /* create the typefind element */
529 typefind = gst_element_factory_make ("typefind", "typefind");
530 g_assert (typefind != NULL);
532 /* add objects to the main pipeline */
533 gst_bin_add_many (GST_BIN (bin), filesrc, typefind, NULL);
535 g_signal_connect (G_OBJECT (typefind), "have_type",
536 G_CALLBACK (type_found), NULL);
538 gst_element_link (filesrc, typefind);
541 gst_element_set_state (GST_ELEMENT (bin), GST_STATE_PLAYING);
543 gst_bin_iterate (GST_BIN (bin));
545 gst_element_set_state (GST_ELEMENT (bin), GST_STATE_NULL);
551 We create a very simple pipeline with only a filesrc and the typefind
552 element in it. The sinkpad of the typefind element has been linked
553 to the source pad of the filesrc.
556 We attached a signal 'have_type' to the typefind element which will be called
557 when the type of the media stream as been detected.
560 The typefind function will loop over all the registered types and will
561 execute each of the typefind functions. As soon as a function returns
562 a GstCaps pointer, the type_found function will be called:
567 type_found (GstElement *typefind, GstCaps* caps)
572 doc = xmlNewDoc ("1.0");
573 doc->root = xmlNewDocNode (doc, NULL, "Capabilities", NULL);
575 parent = xmlNewChild (doc->root, NULL, "Caps1", NULL);
576 gst_caps_save_thyself (caps, parent);
578 xmlDocDump (stdout, doc);
582 In the type_found function we can print or inspect the type that has been
583 detected using the GstCaps APIs. In this example, we just print out the
584 XML representation of the caps structure to stdout.
587 A more useful option would be to use the registry to look up an element
588 that can handle this particular caps structure, or we can also use the
589 autoplugger to link this caps structure to, for example, a videosink.
594 <sect1 id="section-autoplugging-spider">
595 <title>Another approach to autoplugging</title>
597 The autoplug API is interesting, but often impractical. It is static;
598 it cannot deal with dynamic pipelines. An element that will
599 automatically figure out and decode the type is more useful.
603 <title>The spider element</title>
605 The spider element is a generalized autoplugging element. At this point (April 2002), it's
606 the best we've got; it can be inserted anywhere within a pipeline to perform caps
607 conversion, if possible. Consider the following gst-launch line:
609 $ gst-launch filesrc location=my.mp3 ! spider ! osssink
611 The spider will detect the type of the stream, autoplug it to the osssink's caps, and play
612 the pipeline. It's neat.
616 <title>Spider features</title>
621 Automatically typefinds the incoming stream.
626 Has request pads on the source side. This means that it can
627 autoplug one source stream into many sink streams. For example,
628 an MPEG1 system stream can have audio as well as video; that
629 pipeline would be represented in gst-launch syntax as
632 $ gst-launch filesrc location=my.mpeg1 ! spider ! { queue ! osssink } spider.src_%d!
633 { queue ! xvideosink }