8 The type system is used to attach meaning to the bytes in a GstBuffer.
9 A plugin can decide to add metadata to the GstBuffer, this metadata
10 will carry an associated typeid.
12 Types are also used by the plugins to expose the type of their pads to
15 Types are essential for autoplugging.
17 We will explain the inner workings of the type system in this document, as
18 well as the API used in the plugins.
24 a) identification of the type
26 Types are identified by one or more MIME types.
30 The type of any given GstBuffer can be detected using
32 - a typefind function: a function that will inspect the bytes
33 in the buffer and return a gboolean indicating the
34 buffer is of the given type.
36 - a template for the bytes/bits in the data that must be
37 satisfied in order for the GstBuffer to be of the given
40 - other properties that act more like a hint like:
41 the extension of the source filename.
42 the URI of the source.
48 a) the core libraries will create to types:
53 b) all other types will be provided by the plugins
55 before a type can become available to other plugins, the plugin
56 has to register the type.
58 The system will keep a directed graph of the types and the plugins
67 video/mpeg1 audio/mp3 -----\
69 mpeg_play mpeg2dec mpg123 xing ...
71 video/raw audio/raw-----\
73 videosink SDLsink audiosink alsasink ...
76 The system can find the needed plugins to convert video/mpeg to
77 audio/raw using this graph.
83 some types can have the same meaning for example:
85 audio/mp3 and audio/mpeg
89 video/raw and image/raw
92 a plugin that exposes its output type as audio/mpeg and another plugins
93 with input type audio/mp3 can be connected. The system has to know about
94 the equivalence of both types, even it they have a different mime type.
100 some plugins can ouput a specific subset of an already existing type.
104 mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames
105 with mime type: audio/mp3-frame
107 mpg123 only accepts audio/mp3-frame but not audio/mp3.
109 another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also
112 it must be possible to connect both libmpg123 and mpg123 to the mp3parse
114 it must not be possible to connect mpg123 to an element that outputs only
115 audio/mp3 but not audio/mp3-frame.
118 We say that audio/mp3-frame is a more specific subset of type audio/mp3.
120 we can create a type hierarchy:
124 audio/mp3-frame audio/mp3-layer12
126 audio/mp3-layer1 audio/mp3-layer2
132 certain plugins might accept multiple non equivalent types in one of their
133 input pads. Consequently a plugin might output non equivalent types in
136 It must therefore be possible to specify multiple types for a pad.
140 mpegdemux may be able to demux both MPEG1 and MPEG2 system streams.
141 we show the type hierarchy of the video/mpeg as follows:
145 video/mpeg1 video/mpeg2 ---------------\
147 mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video
150 the mpegdemux element might specify the type of the input pad as
151 one of video/mpeg1-system and video/mpeg2-ts
155 7. definition of types
156 ----------------------
158 A plugin will provide the following information to the type system:
160 - a mime type string describing the hierarchy and where the types
161 they provide are located in that hierarchy.
163 - typefind functions for some of the types.
165 - extensions for some of the types
167 We will propose a syntax to define the type hierarchy
169 a) equivalent types :
171 separated with a | sign
173 ( audio/mp3 | audio/mpeg )
179 ( audio/mp3 ( audio/mp3-frame))
183 ( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) )
185 the pad will have type mpegdemux which is the parent for
186 type video/mpeg1-system or video/mpeg2-ps
190 video/mpeg1-system video/mpeg2-ps
194 once the type hierarchy has been registered, the typeid of each
195 element can be obtained with:
197 guint16 gst_type_find_by_mime (gchar *mime)
199 extra typefind functions and/or extensions can be added to the
206 The more specific typefind functions, the functions associated with
207 types in the leaf nodes, are tried first.
209 when a specific type has been found ex. video/mpeg1-system elements
210 that can handle this type or one of its parents are selected:
212 mpegdemux: mpeg1parse: supermpegdecoder:
214 video/mpeg video/mpeg video/mpeg
216 mpegdemux video/mpeg1-system
220 In the above case, also the super mpeg element is selected because it stated
221 that it can handle all sorts of video/mpeg data.
226 suppose the typefind functions finds an mp3 stream, fillowing elments
233 mpg123 is not selected because its pad type is too specific (audio/mp3-frame):
241 if the typefind would find a mp3-frame type, all three objects would be selected.
247 It is clear that clear indications have to be given to the type hierarchy,
248 especially for the top nodes.
250 The more specific an element is in its mime type specification, the more flexible
251 and extendible the plugin system can be.