--- /dev/null
+During the course of a discussion on IRC, it turned out
+that there are many possible ways to handle the capabilities.
+
+A capabilitiy is bascially a set of properties attached to a
+mimetype in order to more closely describe the mimetype.
+Capabiltities are supposed to be attached to pads so that the
+autoplugging algorithm has more specific information to connect
+compatible pads.
+
+We present 3 possible implementation for the capabilities. we need
+to pick one of them.
+
+1. static capabilities
+----------------------
+
+When an element is created, it creates its pads like:
+
+ mpg123->sinkpad = gst_pad_new ("sink", GST_PAD_SINK);
+ gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->sinkpad);
+
+ mpg123->srcpad = gst_pad_new ("src", GST_PAD_SRC);
+ gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->srcpad);
+
+In the static capabilities case, it will attach a GstCaps* structure
+to the pad. The GstCaps structure in the above example might look like:
+
+ static GstCapsFactory mpg123_sink_caps = {
+ "audio/mp3",
+ "layer", GST_CAPS_INT_RANGE (1, 3),
+ "bitrate", GST_CAPS_INT_RANGE (8, 320),
+ NULL
+ };
+
+with
+
+mpg123sinkcaps = gst_caps_register (mpg123_sink_caps);
+
+the factory can be converted into a GstCaps* structure. The
+GstCaps* structure is attached to the pad with:
+
+gst_pad_add_caps (mpg123->sinkpad, mpg123sinkcaps);
+
+The GstElement would then have a sinkpad with the given
+mimetype (audio/mp3) and with the capabilitities of accepting
+mpeg layer 1 to 3 and a bitrate from 8 up to 320 Kbps.
+
+Likewise, the src pad could be set up in the same way. An
+example capability factory could look like:
+
+ static GstCapsFactory mpg123_src_caps = {
+ "audio/raw",
+ "format", GST_CAPS_BITFIELD (...),
+ "depth", GST_CAPS_INT (16),
+ "rate", GST_CAPS_INT_RANGE (4000, 96000),
+ "channels", GST_CAPS_INT_RANGE (1, 2),
+ NULL
+ };
+
+All GstElements would present their pads with the appropriate
+capabilities structure.
+
+The autoplugger would then proceed (once the source media type
+is known with a typefind function) in finding all the elements
+with compatible pads and connecting them into a pipeline.
+
+All elements of the complete pipeline could then be constructed
+with one single pass. No new elements should be added to the
+pipeline because we can figure out all the possibilities using the
+pad capabilities.
+
+We call this the static case because the capabilities of the pads
+are supposed to stay the same after creating the element.
+
+While the ability to completly setup the pipeline before actually
+starting playback is an advantage regarding performance, one obvious
+problem with this setup is that the static case may be too static in
+some cases. We can illustrate this with the following setup:
+
+ ----------) (------------
+ mpg123 ! ! audiosink
+ src sink
+ ! !
+ ----------) (------------
+
+The mpg123 element has its src capabilities set up as mpg123_src_caps
+in the above example.
+
+The audio renderer has its capabilities set up with the following
+factory:
+
+ static GstCapsFactory audio_sink_caps = {
+ "audio/raw",
+ "format", GST_CAPS_BITFIELD (...),
+ "depth", GST_CAPS_INT (16),
+ "rate", GST_CAPS_INT_RANGE (22000, 44000),
+ "channels", GST_CAPS_INT_RANGE (1, 2),
+ NULL
+ };
+
+The static autoplugger has to be carefull when connecting the mpg123
+element with the audiosink because it is theoretically possible that
+the mpg123 element outputs raw audio with a rate that cannot be
+handled by the audiosink (ex. 4000KHz). In the absense of another
+audiosink with more capabilities, the autoplugging of this simple
+pipeline will not be possible and would fail.
+
+the autoplugging algorithm would probably select another element to
+insert between the mpg123 element and the audiosink in order to handle
+the (uncommon) case of a rate conversion (audioscaler).
+
+It is clear that this static setup might even fail or work suboptimal
+for even the common case and should therefore be considered as too
+restrictive.
+
+
+2. dynamic capabilities
+-----------------------
+
+The idea of dynamic capabilities is that the capabilities are not set
+at element create time but rather while the pipeline is running.
+
+An element would still list its mime type using:
+
+ gst_pad_add_type_id(mpg123->sinkpad, mp3type);
+
+The idea would then be that a rough draft of the pipeline would be
+built afer the media type of the stream has been detected with the
+typefind functions. The rough draft would consist of laying out a
+global plan to reach the renderer(s). this plan would basically list
+the set of conversions that have to be performed. (mime-type to
+mime-type conversion).
+
+Elements that accept the src mime-type are tried by giving it a buffer.
+If the element accepts the buffer, it will set its capabilities for
+both the sink pad and the src pad. At that time other elements can be
+tried and added to the src pad, until we reach the renderer. As usual
+one has to be carefull to add just the minimum amount of elements to
+reach the renderer. The global plan will help with that.
+
+Since we basically do not use the capabilities of the sink pad one has
+to question the need for sink pad capabilities in the first place.
+
+We might also have a hard time trying different elements until we find
+a compatible one that does not cause a dead end at some point.
+
+
+3. combined setup
+-----------------
+
+This combined setup will minimise the effort needed to try different
+elements encountered by option 2 while still allowing a more dynamic
+setup based on the actual media stream we are handling.
+
+The combined setup will list/add the sink capabilities at create time.
+It will only set the mime-type of its src pads.
+
+As with option2, a global plan will be built. At runtime the src pads
+will actually specify the capabilities they need for any element that
+wants to be connected to its source pads.
+
+In this case we specifiy the capabilities for all the sink pads of an
+element at create time. The capabilities of the src pads would only
+become available when data has been processed by the element.
+
+The autoplugger would then be able to choose an element that can handle
+the capability listed by the src pad.
+
+in our previous example:
+
+ ----------) (------------
+ mpg123 ! ! audiosink
+ src sink
+ ! !
+ ----------) (------------
+
+the audiosink element would specify its sink pad capabilities at create
+time, while the mpg123 elements src pad would not yet have any capabilities
+set.
+
+When data is handled by the mpg123 element, a capability would be added to
+the mpg123 src pad. This capability might be:
+
+ static GstCapsFactory mpg123_src_caps = {
+ "audio/raw",
+ "format", GST_CAPS_INT (S16),
+ "depth", GST_CAPS_INT (16),
+ "rate", GST_CAPS_INT (44000),
+ "channels", GST_CAPS_INT (2),
+ NULL
+ };
+
+This capability would be compatible with the audiosinks sinkpad capabilities
+and the autoplugger would therefore be able to connect the two elements.
+
+While allowing a more flexible setup with option3, compared to option1, we
+introduce a slightly higher overhead when we need to dynamically connect
+elements. This overhead will not be as big as option2 because we don't
+have to 'try' elements.
+
+so:
+
+ src caps: added at runtime to list the caps needed for an element that
+ wants to connect to this pad.
+ sink caps: the (static) capabilities that this sinkpad has.
+
projects / platform / upstream / gstreamer.git / commitdiff