1 A lot of data streams need a few initial buffers to make sense of the
2 data stream. With these initial buffers, it can pick up at any point in
3 the rest of the data stream.
6 - Vorbis and Theora have three initial Ogg packets
8 - GDP protocol serializes the initial new_segment event and the initial caps
10 It is important that elements that get connected after the stream starts,
11 receive these initial buffers. Also, sink elements should know that these
12 initial buffers are necessary for every new "client" of the sink element;
13 for example, multifdsink needs to send these initial buffers before the
16 Currently, this is done by putting a 'streamheader' value on the caps.
17 'streamheader' is a GST_TYPE_ARRAY of GstBuffer, ie. an ordered list of
20 The buffers themselves still get pushed as normal buffers, but with the
21 IN_CAPS flag set. This allows elements that do not know about streamheader
22 to still function correctly (ie. filesink and others)
24 It is of course important that these streamheader buffers are not sent twice
25 (once because they're in the caps, and once because they're received as
28 So, an element that is aware of streamheader in the caps should probably best
29 drop all incoming IN_CAPS buffers.
31 Rules for sending streamheaders:
32 - all streamheader buffers should have IN_CAPS set
33 - from this list of streamheader buffers, a "streamheader" caps field should
34 be created as a GST_TYPE_ARRAY of GstBuffer. This should be done by
36 (the only important thing about the buffers in this list is the data;
37 other attributes of the buffers get ignored)
38 - each of these buffers should then have these caps set on them.
39 - when all streamheader buffers are collected in the element, pad caps should
40 be set, including this streamheader
41 - streamheader buffers should be sent consecutively, and before any of the
42 data (non-IN_CAPS) buffers they apply to. If necessary, the element
43 should internally queue non-IN_CAPS buffers until the streamheaders
44 are completely assembled.
45 - when new streamheader buffers need to be pushed out, this process is
46 repeated. Receiving a new IN_CAPS buffer after a non-IN_CAPS buffer
47 signifies resetting streamheader, as does the new set_caps with different
48 streamheader right before. (FIXME: it's probably better to explicitly
49 have an event/buffer that clears streamheaders; consider the case of
50 an element like GDP that has created streamheader from the first newsegment
51 and the first caps, and then receives a new tag event that it also wants
52 to put on the streamheader - it should be able to invalidate the previous
55 Elements that can send streamheader caps:
60 Elements that can receive streamheader caps:
63 Elements that can receive and send streamheader caps:
66 Elements that could/should use this:
67 - ffenc_mpeg4 (currently uses/sets codec_data)
68 - theoradec, vorbisdec (currently only look at incoming buffers)
73 The concept of streamheader is more generally applicable. We may want to
74 find a better way of implementing this than having two ways of telling
75 downstream elements about them. Currently an element that is
76 streamheader-aware needs to look both at caps and incoming buffers.
78 One option would be to only rely on the buffer flow, and make all elements
79 aware of a HEADER flag, causing them to keep these buffers around and push
80 them for each new pad link. This needs to be done in every element; a
82 videotestsrc ! theoraenc
83 where later on an oggmux gets connected, should still work without dropping
84 these header buffers. But similarly,
85 videotestsrc ! theoraenc ! identity
86 with a later connection of oggmux to identity should work.
88 This could tie in with an idea to have pads store some initial data (the
89 first new_segment event, tag events, ...), which at the moment also get
90 dropped on dynamic in-stream links.