11 The pads expose the media types they can handle using a mime
12 type and a set of properties. Before the pad is created or
13 used to pass buffers, we only know the global 'range' of media
14 data this pad can accept. When the element has had a chance to
15 look at the media data, only then it knows the exact values of the
20 ! The mp3 decoder exposes the capabilities of its src pad
21 ! with the following caps:
24 ! MIME type: 'audio/raw':
27 ! rate: Integer range: 11025 - 48000
28 ! channels: Integer range: 1 - 2
30 as you can see in example1, the padtemplate has both a range
31 (for the audio rate) and a list (for the number of channels)
34 only when the mpg123 element has decoded the first mpeg audio
35 header, it knows the exact values of the rate and channels
38 suppose that we want to connect this src pad to the sink pad
39 of an audiosink with the properties given in example2:
44 ! MIME type: 'audio/raw':
49 ! rate: Integer range: 8000 - 44000
50 ! channels: Integer range: 1 - 2
52 we can see that connecting the mpg123 src pad with the
53 audiosinks sink pad can cause a potential problem with the
56 When the mpg123 decoder decides to output raw audio with a
57 48000Hz samplerate, the audiosink will not be able to handle
58 it. The conservative approach would be to disallow the connection
59 between the two incompatible pads. This rules out any potential
60 problems but severely limits the amount of possible connections
63 Another approach would be to allow the connection (and mark it
64 as dangerous) and let the two elements figure out a suitable
65 media type at runtime. This procedure is called caps negotiation.
71 The typing of the data that was attached to a buffer used to be
72 done using GstMeta* (and it still is as of 11 feb 2001). With
73 the new GstCaps and GstProps system this typing is gradually moved
74 to the pads and to the padtemplates. This has several advantages:
76 - the typing of the data tends to be static. The type of media
77 doesn't change for every buffer.
79 - Moving the typing up to the pad(templates) allows us to save
80 them into the registry and allows us to figure out what pads
83 - the current metadata implementation needs header files. this may
84 change when we also use properties for metadata.
88 ! This is the current GstMeta structure that travels with audio buffers
90 ! struct _MetaAudioRaw {
93 ! /* formatting information */
101 The question still remains what purpose the metadata will serve
102 now that we expose the media type in the pads. Some possibilities:
104 - interesting information, not describing the data itself but the
105 context in which the data was generated (suggested buffer size,
110 In this proposal we also assume that the current use of metadata using
111 GstMeta is deprecated and that we move this information to the properties
115 3) the pad/padtemplates caps
116 ----------------------------
118 All elements have to provide a padtemplate for their pads.
120 The padtemplates provide a range of possible media types this pad can
121 src/sink. the main purpose for the padtemplates is to allow a
122 rough guess at which pads are compatible before even a single buffer
123 has been processed by the element.
125 pads are usually created from the templates. When the pad is created
126 it has no GstCaps* attached to it yet. The possible caps this pad
127 can have is exposed in the padtemplate. The caps are filled in by
128 the element when it knows the values for the caps.
131 4) the connect function
132 -----------------------
134 when two pads are connected the following steps will take
135 place (not sure, FIXME):
137 - if both pads have caps, the caps are checked. If the caps
138 are incompatible, the padtemplates are checked, if they
139 are compatible, caps negotiation is performed.
141 - if one of the pads has caps, the caps is checked against
142 the padtemplate of the peer pad. If they are incompatible,
143 the padtemplates are compared, if they are incompatible,
144 caps negotiation is performed.
146 - if none of the pads have caps, the padtemplates are checked,
147 if they are incompatible, a warning is issued.
150 5) when the element knows the media type it is handling
151 -------------------------------------------------------
153 When the element has received its first buffer it will know
154 the media type it is handling by inspecting the buffer.
156 before pushing the data out to its peer element(s), the element
157 will set its src pad with the appropriate caps and properties.
158 These caps must follow the following rules:
160 - the caps must be compatible with the padtemplates of this
163 - the caps cannot contain ranges or lists.
165 when the element wants to change the caps of a pad, it has to
166 perform gst_pad_renegotiate (GstPad *pad). this will trigger
167 the caps negotiation procedure.
169 this will trigger the class method of the pad and calls the pads
170 gst_pad_negotiate function:
172 GstCaps *gst_pad_negotiate (GstPad *pad, GstCaps *caps, guint count);
174 This function takes a GstCaps *structure as an argument (typically the
175 current caps of the pad) and a negotiation counter. this counter can be
176 used to keep track of the negotiation process.
178 The pad then creates a new caps structure with the desired caps.
179 If the caps are accepted, it just returns the provided input caps. the
180 _renegotiate function will set the caps of both pads whenever the
181 input caps are the same (pointer wise) as the input caps.
183 the caps structure is checked against the padtemplate of the peer pad,
184 if it is incompatible the gst_pad_negotiate function is called again
185 and the element is supposed to create another caps structure.
187 the gst_pad_renegotiate function then calls the gst_pad_negotiate
188 function of the peer pad with the new caps as the argument. The peer
189 pad can adjust or create a new caps if it doesn't accept it.
191 the caps structure keeps on bouncing between the two pads until one
192 of the pads negotiation functions returns the caps unmodified.
194 The element can also return a NULL pointer if it has run out of
195 options for the caps structure. When this happens, both pads are set
196 the the NULL caps again and the pad connnection is broken.
198 The negotiation process is stopped after a fixed number of tries,
199 when the counter has reached some limit. This limit is typically
200 checked by the pads negotiate function.
203 6) caps negotiation function
204 ----------------------------
206 the negotiate function of a pad is called whenever the pad or
207 peer pad has performed _renegotiate.
211 ! this is the caps negotiation function implemented by an element on
212 ! one of its sink pads.
215 ! gst_pad_negotiate (GstPad *pad, GstCaps *caps, guint counter)
217 ! /* we don't accept anything else than audio/raw */
218 ! if (strcmp (gst_caps_get_mime (caps), "audio/raw"))
221 ! if (gst_caps_get_int_prop (caps, "format") != AFMT_S16_LE)
224 ! /* we accept everything else */
228 When the negotiate function returns NULL (it does not accept the
229 specified caps of the peer pad), the negotiation process is stopped.
233 APPENDIX A: use cases
234 =====================
236 1) mpg123 src!sink audiosink
237 ----------------------------
239 When the pads are connected the padtemplates are checked and it
240 turns out that the pads might be incompatible (mpg123 can do
241 48000Hz while audiosink can do 44000Hz). Nothing happens at
242 connect time except for the user app that can mark this connection
243 as possibly dangerous and keep some spare elements ready for when
244 the pads turn out to be incompatible.
246 both elements start out with no caps at all (NULL). mpg123 wants
247 to output a buffer with specific properties. It calls
248 gst_pad_renegotiate (mpg123->srcpad).
250 The _renegotiate functions calls the negotiate function of the
251 mpg123->srcpad. the negotiate function would look like this:
255 * The mpg123 element cannot convert the decoded type into something
256 * else so it has to force the caps of the src pad into the specific
257 * type as defined by the mp3.
260 gst_mpeg123_src_negotiate (GstPad *pad, GstCaps *caps, guint counter)
264 mpg123 = GST_MPG123 (gst_pad_get_parent (pad));
266 /* we got caps in, check them */
268 if (!strcmp (gst_caps_get_mime (caps), "audio/raw") &&
269 (gst_caps_get_int_prop (caps, "format") == AFMT_S16_LE) &&
270 (gst_caps_get_int_prop (caps, "depth") == 16) &&
271 (gst_caps_get_int_prop (caps, "rate") == mpg123->rate) &&
272 (gst_caps_get_int_prop (caps, "channels") == mpg123->channels)) {
276 /* we didn't get caps, so we decide */
277 else if (counter != 2) {
280 /* fill in our desired caps */
281 new = gst_caps_new_with_props (
282 "src_caps", /* name */
283 "audio/raw", /* mime */
285 "format", GST_PROPS_INT (AFMT_S16_LE),
286 "depth", GST_PROPS_INT (16),
287 "rate", GST_PROPS_INT (mpg123->rate),
288 "channels", GST_PROPS_INT (mpg123->channels),
294 /* too many attempts at nogotiation, bail out */
299 The audiosink pad negotiate function would look like this:
302 * The audiosink has a wide range of possible parameters for
303 * its sink pad, based on the audio card capabilities and
304 * possibly the element configuration.
305 * we assume the audiosink element can be both the initiator of
306 * the negotiations and the negotiated one.
309 gst_audiosink_sink_negotiate (GstPad *pad, GstCaps *caps, guint counter)
311 GstAudiosink *audiosink;
312 gboolean accepted = TRUE;
314 audiosink = GST_AUDIOSINK (gst_pad_get_parent (pad));
316 /* we got caps in, we know they will match the padtemplate */
320 /* we didn't get caps, so we decide */
321 else if (counter != 2) {
324 /* fill in our desired caps */
325 new = gst_caps_new_with_props (
326 "sink_caps", /* name */
327 "audio/raw", /* mime */
329 "format", GST_PROPS_INT (audiosink->format),
330 "depth", GST_PROPS_INT (audiosink->depth),
331 "rate", GST_PROPS_INT (audiosink->rate),
332 "channels", GST_PROPS_INT (audiosink->channels),
338 /* too many attempts at nogotiation, bail out */