1 SOME OF THE FIRST IDEAS, PRETTY OUTDATED
2 ----------------------------------------
5 During the course of a discussion on IRC, it turned out
6 that there are many possible ways to handle the capabilities.
8 A capability is basically a set of properties attached to a
9 mimetype in order to more closely describe the mimetype.
10 Capabilities are supposed to be attached to pads so that the
11 autoplugging algorithm has more specific information to connect
14 We present 3 possible implementation for the capabilities. we need
17 1. static capabilities
18 ----------------------
20 When an element is created, it creates its pads like:
22 mpg123->sinkpad = gst_pad_new ("sink", GST_PAD_SINK);
23 gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->sinkpad);
25 mpg123->srcpad = gst_pad_new ("src", GST_PAD_SRC);
26 gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->srcpad);
28 In the static capabilities case, it will attach a GstCaps* structure
29 to the pad. The GstCaps structure in the above example might look like:
31 static GstCapsFactory mpg123_sink_caps = {
33 "layer", GST_CAPS_INT_RANGE (1, 3),
34 "bitrate", GST_CAPS_INT_RANGE (8, 320),
40 mpg123sinkcaps = gst_caps_register (mpg123_sink_caps);
42 the factory can be converted into a GstCaps* structure. The
43 GstCaps* structure is attached to the pad with:
45 gst_pad_add_caps (mpg123->sinkpad, mpg123sinkcaps);
47 The GstElement would then have a sinkpad with the given
48 mimetype (audio/mp3) and with the capabilitities of accepting
49 mpeg layer 1 to 3 and a bitrate from 8 up to 320 Kbps.
51 Likewise, the src pad could be set up in the same way. An
52 example capability factory could look like:
54 static GstCapsFactory mpg123_src_caps = {
56 "format", GST_CAPS_BITFIELD (...),
57 "depth", GST_CAPS_INT (16),
58 "rate", GST_CAPS_INT_RANGE (4000, 96000),
59 "channels", GST_CAPS_INT_RANGE (1, 2),
63 All GstElements would present their pads with the appropriate
64 capabilities structure.
66 The autoplugger would then proceed (once the source media type
67 is known with a typefind function) in finding all the elements
68 with compatible pads and connecting them into a pipeline.
70 All elements of the complete pipeline could then be constructed
71 with one single pass. No new elements should be added to the
72 pipeline because we can figure out all the possibilities using the
75 We call this the static case because the capabilities of the pads
76 are supposed to stay the same after creating the element.
78 While the ability to completly setup the pipeline before actually
79 starting playback is an advantage regarding performance, one obvious
80 problem with this setup is that the static case may be too static in
81 some cases. We can illustrate this with the following setup:
83 ----------) (------------
87 ----------) (------------
89 The mpg123 element has its src capabilities set up as mpg123_src_caps
92 The audio renderer has its capabilities set up with the following
95 static GstCapsFactory audio_sink_caps = {
97 "format", GST_CAPS_BITFIELD (...),
98 "depth", GST_CAPS_INT (16),
99 "rate", GST_CAPS_INT_RANGE (22000, 44000),
100 "channels", GST_CAPS_INT_RANGE (1, 2),
104 The static autoplugger has to be carefull when connecting the mpg123
105 element with the audiosink because it is theoretically possible that
106 the mpg123 element outputs raw audio with a rate that cannot be
107 handled by the audiosink (ex. 4000KHz). In the absence of another
108 audiosink with more capabilities, the autoplugging of this simple
109 pipeline will not be possible and would fail.
111 the autoplugging algorithm would probably select another element to
112 insert between the mpg123 element and the audiosink in order to handle
113 the (uncommon) case of a rate conversion (audioscaler).
115 It is clear that this static setup might even fail or work suboptimal
116 for even the common case and should therefore be considered as too
120 2. dynamic capabilities
121 -----------------------
123 The idea of dynamic capabilities is that the capabilities are not set
124 at element create time but rather while the pipeline is running.
126 An element would still list its mime type using:
128 gst_pad_add_type_id(mpg123->sinkpad, mp3type);
130 The idea would then be that a rough draft of the pipeline would be
131 built afer the media type of the stream has been detected with the
132 typefind functions. The rough draft would consist of laying out a
133 global plan to reach the renderer(s). this plan would basically list
134 the set of conversions that have to be performed. (mime-type to
135 mime-type conversion).
137 Elements that accept the src mime-type are tried by giving it a buffer.
138 If the element accepts the buffer, it will set its capabilities for
139 both the sink pad and the src pad. At that time other elements can be
140 tried and added to the src pad, until we reach the renderer. As usual
141 one has to be carefull to add just the minimum amount of elements to
142 reach the renderer. The global plan will help with that.
144 Since we basically do not use the capabilities of the sink pad one has
145 to question the need for sink pad capabilities in the first place.
147 We might also have a hard time trying different elements until we find
148 a compatible one that does not cause a dead end at some point.
154 This combined setup will minimise the effort needed to try different
155 elements encountered by option 2 while still allowing a more dynamic
156 setup based on the actual media stream we are handling.
158 The combined setup will list/add the sink capabilities at create time.
159 It will only set the mime-type of its src pads.
161 As with option2, a global plan will be built. At runtime the src pads
162 will actually specify the capabilities they need for any element that
163 wants to be connected to its source pads.
165 In this case we specify the capabilities for all the sink pads of an
166 element at create time. The capabilities of the src pads would only
167 become available when data has been processed by the element.
169 The autoplugger would then be able to choose an element that can handle
170 the capability listed by the src pad.
172 in our previous example:
174 ----------) (------------
178 ----------) (------------
180 the audiosink element would specify its sink pad capabilities at create
181 time, while the mpg123 elements src pad would not yet have any capabilities
184 When data is handled by the mpg123 element, a capability would be added to
185 the mpg123 src pad. This capability might be:
187 static GstCapsFactory mpg123_src_caps = {
189 "format", GST_CAPS_INT (S16),
190 "depth", GST_CAPS_INT (16),
191 "rate", GST_CAPS_INT (44000),
192 "channels", GST_CAPS_INT (2),
196 This capability would be compatible with the audiosinks sinkpad capabilities
197 and the autoplugger would therefore be able to connect the two elements.
199 While allowing a more flexible setup with option3, compared to option1, we
200 introduce a slightly higher overhead when we need to dynamically connect
201 elements. This overhead will not be as big as option2 because we don't
202 have to 'try' elements.
206 src caps: added at runtime to list the caps needed for an element that
207 wants to connect to this pad.
208 sink caps: the (static) capabilities that this sinkpad has.