2 title: Types and Properties
7 There is a very large set of possible types that may be used to pass
8 data between elements. Indeed, each new element that is defined may use
9 a new data format (though unless at least one other element recognises
10 that format, it will be most likely be useless since nothing will be
11 able to link with it).
13 In order for types to be useful, and for systems like autopluggers to
14 work, it is necessary that all elements agree on the type definitions,
15 and which properties are required for each type. The GStreamer framework
16 itself simply provides the ability to define types and parameters, but
17 does not fix the meaning of types and parameters, and does not enforce
18 standards on the creation of new types. This is a matter for a policy to
19 decide, not technical systems to enforce.
21 For now, the policy is simple:
23 - Do not create a new type if you could use one which already exists.
25 - If creating a new type, discuss it first with the other GStreamer
26 developers, on at least one of: IRC, mailing lists.
28 - Try to ensure that the name for a new format is as unlikely to
29 conflict with anything else created already, and is not a more
30 generalised name than it should be. For example: "audio/compressed"
31 would be too generalised a name to represent audio data compressed
32 with an mp3 codec. Instead "audio/mp3" might be an appropriate name,
33 or "audio/compressed" could exist and have a property indicating the
34 type of compression used.
36 - Ensure that, when you do create a new type, you specify it clearly,
37 and get it added to the list of known types so that other developers
38 can use the type correctly when writing their elements.
40 ## Building a Simple Format for Testing
42 If you need a new format that has not yet been defined in our [List of
43 Defined Types](#list-of-defined-types), you will want to have some
44 general guidelines on media type naming, properties and such. A media
45 type would ideally be equivalent to the Mime-type defined by IANA; else,
46 it should be in the form type/x-name, where type is the sort of data
47 this media type handles (audio, video, ...) and name should be something
48 specific for this specific type. Audio and video media types should try
49 to support the general audio/video properties (see the list), and can
50 use their own properties, too. To get an idea of what properties we
51 think are useful, see (again) the list.
53 Take your time to find the right set of properties for your type. There
54 is no reason to hurry. Also, experimenting with this is generally a good
55 idea. Experience learns that theoretically thought-out types are good,
56 but they still need practical use to assure that they serve their needs.
57 Make sure that your property names do not clash with similar properties
58 used in other types. If they match, make sure they mean the same thing;
59 properties with different types but the same names are *not* allowed.
61 ## Typefind Functions and Autoplugging
63 With only *defining* the types, we're not yet there. In order for a
64 random data file to be recognized and played back as such, we need a way
65 of recognizing their type out of the blue. For this purpose,
66 “typefinding” was introduced. Typefinding is the process of detecting
67 the type of a data stream. Typefinding consists of two separate parts:
68 first, there's an unlimited number of functions that we call *typefind
69 functions*, which are each able to recognize one or more types from an
70 input stream. Then, secondly, there's a small engine which registers and
71 calls each of those functions. This is the typefind core. On top of this
72 typefind core, you would normally write an autoplugger, which is able to
73 use this type detection system to dynamically build a pipeline around an
74 input stream. Here, we will focus only on typefind functions.
76 A typefind function usually lives in
77 `gst-plugins-base/gst/typefind/gsttypefindfunctions.c`, unless there's a
78 good reason (like library dependencies) to put it elsewhere. The reason
79 for this centralization is to reduce the number of plugins that need to
80 be loaded in order to detect a stream's type. Below is an example that
81 will recognize AVI files, which start with a “RIFF” tag, then the size
82 of the file and then an “AVI” tag:
86 gst_my_typefind_function (GstTypeFind *tf,
89 guint8 *data = gst_type_find_peek (tf, 0, 12);
92 GUINT32_FROM_LE (&((guint32 *) data)[0]) == GST_MAKE_FOURCC ('R','I','F','F') &&
93 GUINT32_FROM_LE (&((guint32 *) data)[2]) == GST_MAKE_FOURCC ('A','V','I',' ')) {
94 gst_type_find_suggest (tf, GST_TYPE_FIND_MAXIMUM,
95 gst_caps_new_simple ("video/x-msvideo", NULL));
100 plugin_init (GstPlugin *plugin)
102 if (!gst_type_find_register (plugin, "", GST_RANK_PRIMARY,
103 gst_my_typefind_function, "avi",
104 gst_caps_new_simple ("video/x-msvideo",
111 Note that `gst-plugins/gst/typefind/gsttypefindfunctions.c` has some
112 simplification macros to decrease the amount of code. Make good use of
113 those if you want to submit typefinding patches with new typefind
116 Autoplugging has been discussed in great detail in the Application
119 ## List of Defined Types
121 Below is a list of all the defined types in GStreamer. They are split up
122 in separate tables for audio, video, container, subtitle and other
123 types, for the sake of readability. Below each table might follow a list
124 of notes that apply to that table. In the definition of each type, we
125 try to follow the types and rules as defined by
126 [IANA](http://www.iana.org/assignments/media-types) for as far as
129 Jump directly to a specific table:
131 - [Table of Audio Types](#table-of-audio-types)
133 - [Table of Video Types](#table-of-video-types)
135 - [Table of Container Types](#table-of-container-types)
137 - [Table of Subtitle Types](#table-of-subtitle-types)
139 - [Table of Other Types](#table-of-other-types)
141 Note that many of the properties are not *required*, but rather
142 *optional* properties. This means that most of these properties can be
143 extracted from the container header, but that - in case the container
144 header does not provide these - they can also be extracted by parsing
145 the stream header or the stream content. The policy is that your element
146 should provide the data that it knows about by only parsing its own
147 content, not another element's content. Example: the AVI header provides
148 samplerate of the contained audio stream in the header. MPEG system
149 streams don't. This means that an AVI stream demuxer would provide
150 samplerate as a property for MPEG audio streams, whereas an MPEG demuxer
151 would not. A decoder needing this data would require a stream parser in
152 between two extract this from the header or calculate it from the
156 <caption>Table of Audio Types</caption>
169 <td><em>All audio types.</em></td>
173 <td><em>All audio types</em></td>
180 <td>channel-mask</td>
192 <td><em>All raw audio types.</em></td>
196 <td>Unstructured and uncompressed raw audio data.</td>
199 <td><em>All encoded audio types.</em></td>
203 <td>AC-3 or A52 audio streams.</td>
206 <td>audio/x-adpcm</td>
207 <td>ADPCM Audio streams.</td>
214 <td>audio/x-cinepak</td>
215 <td>Audio as provided in a Cinepak (Quicktime) stream.</td>
219 <td>Audio as provided in a Digital Video stream.</td>
222 <td>audio/x-flac</td>
223 <td>Free Lossless Audio codec (FLAC).</td>
227 <td>Data encoded by the GSM codec.</td>
230 <td>audio/x-alaw</td>
231 <td>A-Law Audio.</td>
234 <td>audio/x-mulaw</td>
235 <td>Mu-Law Audio.</td>
238 <td>audio/x-mace</td>
239 <td>MACE Audio (used in Quicktime).</td>
243 <td>Audio data compressed using the MPEG audio encoding scheme.</td>
258 <td>audio/x-qdm2</td>
259 <td>Data encoded by the QDM version 2 codec.</td>
262 <td>audio/x-pn-realaudio</td>
263 <td>Realmedia Audio data.</td>
266 <td>audio/x-speex</td>
267 <td>Data encoded by the Speex audio codec</td>
270 <td>audio/x-vorbis</td>
271 <td>Vorbis audio data</td>
275 <td>Windows Media Audio</td>
278 <td>audio/x-paris</td>
279 <td>Ensoniq PARIS audio</td>
283 <td>Amiga IFF / SVX8 / SV16 audio</td>
286 <td>audio/x-nist</td>
287 <td>Sphere NIST audio</td>
291 <td>Sound Blaster VOC audio</td>
294 <td>audio/x-ircam</td>
295 <td>Berkeley/IRCAM/CARL audio</td>
299 <td>Sonic Foundry's 64 bit RIFF/WAV</td>
305 <caption>Table of Video Types</caption>
318 <td><em>All video types.</em></td>
322 <td><em>All video types</em></td>
333 <td>max-framerate</td>
341 <td>interlace-mode</td>
353 <td>pixel-aspect-ratio</td>
361 <td><em>All raw video types.</em></td>
365 <td>Unstructured and uncompressed raw video data.</td>
368 <td><em>All encoded video types.</em></td>
371 <td>video/x-3ivx</td>
375 <td>video/x-divx</td>
380 <td>Digital Video.</td>
384 <td>FFMpeg video.</td>
387 <td>video/x-h263</td>
388 <td>H-263 video.</td>
395 <td>video/x-h264</td>
396 <td>H-264 video.</td>
399 <td>video/x-huffyuv</td>
400 <td>Huffyuv video.</td>
403 <td>video/x-indeo</td>
404 <td>Indeo video.</td>
407 <td>video/x-intel-h263</td>
408 <td>H-263 video.</td>
411 <td>video/x-jpeg</td>
412 <td>Motion-JPEG video.</td>
419 <td>systemstream</td>
423 <td>video/x-msmpeg</td>
424 <td>Microsoft MPEG-4 video deviations.</td>
427 <td>video/x-msvideocodec</td>
428 <td>Microsoft Video 1 (oldish codec).</td>
431 <td>video/x-pn-realvideo</td>
432 <td>Realmedia video.</td>
436 <td>RLE animation format.</td>
443 <td>palette_data</td>
448 <td>Sorensen Video.</td>
451 <td>video/x-tarkin</td>
452 <td>Tarkin video.</td>
455 <td>video/x-theora</td>
456 <td>Theora video.</td>
464 <td>Windows Media Video</td>
467 <td>video/x-xvid</td>
471 <td><em>All image types.</em></td>
475 <td>Graphics Interchange Format.</td>
479 <td>Joint Picture Expert Group Image.</td>
483 <td>Portable Network Graphics Image.</td>
487 <td>Tagged Image File Format.</td>
493 <caption>Table of Container Types</caption>
506 <td>video/x-ms-asf</td>
507 <td>Advanced Streaming Format (ASF).</td>
510 <td>video/x-msvideo</td>
515 <td>Digital Video.</td>
518 <td>video/x-matroska</td>
523 <td>Motion Pictures Expert Group System Stream.</td>
526 <td>application/ogg</td>
530 <td>video/quicktime</td>
534 <td>application/vnd.rn-realmedia</td>
545 <caption>Table of Subtitle Types</caption>
565 <caption>Table of Other Types</caption>