1) The Autoplugger API ---------------------- We'll first describe how to use the autoplugger. We will provide a use case: autoplug an mpeg1 system stream for audio/video playback. a) creating an autoplugger -------------------------- Before any autoplugging can be done, you'll have to create an autoplugger object. Autoplugger objects (autopluggers) are provided by plugins and are created with gst_autoplugfactor_make(). GStreamer has provisions for two types of autopluggers: - regular autopluggers, which act as a complex element construction mechanism. They usually don't create threads and operate solely on GstCaps* for the source and destination. The complex elements created by regular autopluggers have src and sink pad compatible with the requested GstCaps*. - renderer autopluggers, which are designed to create a complex object that can be used to playback media. Renderer autoplugged complex elements have no src pads, only one sink pad. We'll create a renderer autoplugger like this: ! ! GstAutoplug *autoplug; ! ! autoplug = gst_autoplugfactory_make ("staticrender"); ! b) finding out the source media type. ------------------------------------- Before we can start the autoplugger, we have to find out the source media type. This can be done using the typefind functions provided by various plugins. We will create a little pipeline to detect the media type by connecting a disksrc element to a typefind element. The typefind element will repeatedly call all registered typefind functions with the buffer it receives on its sink pad. when a typefind function returns a non NULL GstCaps*, that caps is set to the sink pad of the typefind element and a signal is emitted to notify the app. Due to caps negotiation, the disksrc will have the detected GstCaps* set on its src pad. We typically use a function like below to detect the type of a media stream on an element (typically a disksrc). The function accepts a pipeline and the element inside the pipeline on which the typefind should be performed (passing a GstPad* is probably a better option FIXME). ! ! static GstCaps* ! gst_play_typefind (GstBin *bin, GstElement *element) ! { ! GstElement *typefind; ! GstCaps *caps = NULL; ! ! typefind = gst_elementfactory_make ("typefind", "typefind"); ! g_return_val_if_fail (typefind != NULL, FALSE); ! ! gst_pad_connect (gst_element_get_pad (element, "src"), ! gst_element_get_pad (typefind, "sink")); ! ! gst_bin_add (bin, typefind); ! ! gst_element_set_state (GST_ELEMENT (bin), GST_STATE_PLAYING); ! ! // push a buffer... the have_type signal handler will set the found flag ! gst_bin_iterate (bin); ! ! gst_element_set_state (GST_ELEMENT (bin), GST_STATE_NULL); ! ! caps = gst_pad_get_caps (gst_element_get_pad (element, "src")); ! ! gst_pad_disconnect (gst_element_get_pad (element, "src"), ! gst_element_get_pad (typefind, "sink")); ! gst_bin_remove (bin, typefind); ! gst_object_unref (GST_OBJECT (typefind)); ! ! return caps; ! } ! Also note that the disksrc was added to the pipeline before calling this typefind function. When the function returns a non-NULL pointer, the media type has been determined and autoplugging can commence. Assume that in our mpeg1 use case the above function returns a GstCaps* like: ! ! srccaps = GST_CAPS_NEW ("mpeg1system_typefind", ! "video/mpeg", ! "mpegversion", GST_PROPS_INT (1), ! "systemstream", GST_PROPS_BOOLEAN (TRUE) ! ); ! c) Performing the autoplugging ------------------------------ Since we use the renderer API, we have to create the output elements that are going to be used as the final sink elements. ! ! osssink = gst_elementfactory_make("osssink", "play_audio"); ! videosink = gst_elementfactory_make("xvideosink", "play_video"); ! We then create a complex element using the following code. ! ! new_element = gst_autoplug_to_renderers (autoplug, ! srccaps, ! videosink, ! osssink, ! NULL); ! ! if (!new_element) { ! g_print ("could not autoplug, no suitable codecs found...\n"); ! exit (-1); ! } ! 2) Autoplugging internals ------------------------- We will now describe the internals of the above gst_autoplug_to_renderers() function call. This code is implemented in a plugin found in: gst/autoplug/gststaticautoplugrender.c a) phase1: create lists of factories. --------------------------------------- The autoplugger will start with executing the following piece of code: ! ! i = 0; ! ! for each sink: ! { ! sinkpad = take the first sinkpad of the sink (HACK) ! ! list[i] = gst_autoplug_caps (srccaps, sinkpad->caps); ! ! i++; ! } ! gst_autoplug_caps will figure out (based on the padtemplates) which elementfactories are needed to connect srccaps to sinkpad->caps and will return them in a list. The element list is created by using a modified shortest path algorithm by Dijkstra (http://www.orie.cornell.edu/~or115/handouts/handout3/handout3.html). The nodes of the graph are the elementfactories and the weight of the arcs is based on the pad compatibility of the padtemplates of the elementfactory. For incompatible elementfactories, we use a weight of MAX_COST (999999) and for compatible padtemplates we use 1. ex. we have two sinks with following caps: ! ! video/raw audio/raw ! "...." "...." ! gst_autoplug_caps will figure out that for the first sink the following elements are needed: ! ! mpeg1parse, mp1videoparse, mpeg_play ! for the second sink the following is needed: ! ! mpeg1parse, mad ! Note that for the audio connection the element list "mpeg1parse, mp3parse, mpg123" would also connect the srccaps to the audiosink caps. Since the "mpeg1parse, mad" list is shorter, it it always preferred by the autoplugger. We now have two lists of elementfactories. b) phase2: collect common elements from the lists and add them to a bin. ------------------------------------------------------------------------ The rationale is that from the lists we have created in phase1, there must be some element that is a splitter and that it has to come first (HACK) We try to find that element by comparing the lists until an element differs. We start by creating a toplevel bin that is going to be our complex element. In our use-case we find that mpeg1parse is an element common to both lists, so we add it to the bin. We then try to find a good ghostpad for the resulting complex element. This is done by looping over the sink pads of the first common element and taking the pad that is compatible with the srcaps. We end up with a bin like this: ! ! (----------------------) ! ! autoplug_bin ! ! ! ! ! ! (------------) ! ! ! ! mpeg1parse ! ! ! ! - sink ! ! ! ! / (------------) ! ! sink ! ! (----------------------) ! c) phase3: add remaining elements --------------------------------- now we loop over all the list and try to add the remaining elements (HACK) we always use a new thread for the elements when there is a common element found. if a new thread is needed (either becuase the previous element is a common element or the object flag of the next element is set to GST_SUGGEST_THREAD) we add a queue to the bin and we add a new thread. We add the elements to the bin and connect them using gst_pipeline_pads_autoplug. we finally arrive at the sink element and we're done. ex. we have just found our mpeg1parse common element, so we start a thread. We add a queue to the bin and a new thread, we add the elements mp1videoparse and mpeg_play to the thread. We arrive at the videosink, we see that the SUGGEST_THREAD flag is set, we add a queue and a thread and add the videosink in the thread. the same procedure happens for the audio part. We are now left with the following pipeline: We will also have set a signal "new_pad" on the mpeg1parse element because the element mp1videoparse could not be connected to the element just yet. (---------------------------------------------------------------------------------------------) !autoplug_bin ! ! ! ! (----------------------------------------) (------------) ! ! !thread ! ! thread ! ! ! (-----) ! (-------------) (---------) (-----) ! ! (---------)! ! ! !queue! ! !mp1videoparse! !mpeg_play! !queue! ! ! !videosink!! ! ! sink src-sink src-sink src-sink src-sink !! ! ! (-----------) (-----) ! (-------------) (---------) (-----) ! ! (---------)! ! ! ! mpeg1parse! (----------------------------------------) (------------) ! ! - sink ! ! ! / (-----------) ! sink (----------------------------------------) (------------) ! ! !thread ! ! thread ! ! ! (-----) ! (-------------) (-----) ! ! (---------)! ! ! !queue! ! !mad ! !queue! ! ! !videosink!! ! ! sink src-sink src ------------ sink src-sink !! ! ! (-----) ! (-------------) (-----) ! ! (---------)! ! ! (----------------------------------------) (------------) ! (---------------------------------------------------------------------------------------------) The autoplugger will return the autoplug_bin. the app will then connect the disksrc to the sinkpad of the autoplugged bin. Then we play, create_plan happens, data is flowing and the "new_pad" signal is called from mpeg1parse, gst_pipeline_pad_autoplug is called and the connection between mpeg1parse and the videoqueue is made. same for audio. Et voila. same procedure for mp3/vorbis/avi/qt/mpeg2 etc...