4 This document outlines the buffering policy used in the GStreamer
5 core that can be used by plugins and applications.
7 The purpose of buffering is to accumulate enough data in a pipeline so that
8 playback can occur smoothly and without interruptions. It is typically done
9 when reading from a (slow) non-live network source but can also be used for
12 We want to be able to implement the following features:
14 - buffering up to a specific amount of data, in memory, before starting playback
15 so that network fluctuations are minimized.
16 - download of the network file to a local disk with fast seeking in the
17 downloaded data. This is similar to the quicktime/youtube players.
18 - caching of semi-live streams to a local, on disk, ringbuffer with seeking in
19 the cached area. This is similar to tivo-like timeshifting.
20 - progress report about the buffering operations
21 - the possibility for the application to do more complex buffering
27 +---------+ +---------+ +-------+
28 | httpsrc | | buffer | | demux |
29 | src - sink src - sink ....
30 +---------+ +---------+ +-------+
32 In this case we are reading from a slow network source into a buffer element
35 The buffer element has a low and high watermark expressed in bytes. The
36 buffer uses the watermarks as follows:
38 - The buffer element will post BUFFERING messages until the high watermark
39 is hit. This instructs the application to keep the pipeline PAUSED, which
40 will eventually block the srcpad from pushing while data is prerolled in
42 - When the high watermark is hit, a BUFFERING message with 100% will be
43 posted, which instructs the application to continue playback.
44 - When the low watermark is hit during playback, the queue will start posting
45 BUFFERING messages again, making the application PAUSE the pipeline again
46 until the high watermark is hit again. This is called the rebuffering
48 - During playback, the queue level will fluctuate between the high and
49 low watermarks as a way to compensate for network irregularities.
51 This buffering method is usable when the demuxer operates in push mode.
52 Seeking in the stream requires the seek to happen in the network source.
53 It is mostly desirable when the total duration of the file is not known, such
54 as in live streaming or when efficient seeking is not possible/required.
56 * Incremental download
58 +---------+ +---------+ +-------+
59 | httpsrc | | buffer | | demux |
60 | src - sink src - sink ....
61 +---------+ +----|----+ +-------+
65 In this case, we know the server is streaming a fixed length file to the
66 client. The application can choose to download the file to disk. The buffer
67 element will provide a push or pull based srcpad to the demuxer to navigate in
70 This mode is only suitable when the client can determine the length of the
73 In this case, buffering messages will be emitted as usual when the requested
74 range is not within the downloaded area + buffersize. The buffering message
75 will also contain an indication that incremental download is being performed.
76 This flag can be used to let the application control the buffering in a more
77 intelligent way, using the BUFFERING query, for example.
79 The application can use the BUFFERING query to get the estimated download time
80 and match this time to the current/remaining playback time to control when
81 playback should start to have a non-interrupted playback experience.
86 +---------+ +---------+ +-------+
87 | httpsrc | | buffer | | demux |
88 | src - sink src - sink ....
89 +---------+ +----|----+ +-------+
93 In this mode, a fixed size ringbuffer is kept to download the server content.
94 This allows for seeking in the buffered data. Depending on the size of the
95 buffer one can seek further back in time.
97 This mode is suitable for all live streams.
99 As with the incremental download mode, buffering messages are emitted along
100 with an indication that timeshifting download is in progress.
105 In live pipelines we usually introduce some latency between the capture and
106 the playback elements. This latency can be introduced by a queue (such as a
107 jitterbuffer) or by other means (in the audiosink).
109 Buffering messages can be emitted in those live pipelines as well and serve as
110 an indication to the user of the latency buffering. The application usually
111 does not react to these buffering messages with a state change.
117 A GST_MESSAGE_BUFFERING must be posted on the bus when playback temporarily
118 stops to buffer and when buffering finishes. When the percentage field in the
119 BUFFERING message is 100, buffering is done. Values less than 100 mean that
120 buffering is in progress.
122 The BUFFERING message should be intercepted and acted upon by the application.
123 The message contains at least one field that is sufficient for basic
126 "buffer-percent", G_TYPE_INT, between 0 and 100
128 Several more clever ways of dealing with the buffering messages can be used when
129 in incremental or timeshifting download mode. For this purpose additional fields
130 are added to the buffering message:
132 "buffering-mode", GST_TYPE_BUFFERING_MODE,
133 enum { "stream", "download", "timeshift", "live" }
134 - Buffering mode in use. See above for an explanation of the
135 different alternatives. This field can be used to let the
136 application have more control over the buffering process.
138 "avg-in-rate", G_TYPE_INT
139 - Average input buffering speed in bytes/second. -1 is unknown.
140 This is the average number of bytes per second that is received on the
141 buffering element input (sink) pads. It is a measurement of the network
144 "avg-out-rate", G_TYPE_INT
145 - Average consumption speed in bytes/second. -1 is unknown.
146 This is the average number of bytes per second that is consumed by the
147 downstream element of the buffering element.
149 "buffering-left", G_TYPE_INT64
150 - Estimated time that buffering will take in milliseconds. -1 is unknown.
151 This is measured based on the avg-in-rate and the filled level of the
152 queue. The application can use this hint to update the GUI about the
153 estimated remaining time that buffering will take.
158 While data is buffered the pipeline should remain in the PAUSED state. It is
159 also possible that more data should be buffered while the pipeline is PLAYING,
160 in which case the pipeline should be PAUSED until the buffering finishes.
162 BUFFERING messages can be posted while the pipeline is prerolling. The
163 application should not set the pipeline to PLAYING before a BUFFERING message
164 with a 100 percent value is received, which might only happen after the pipeline
167 An exception is made for live pipelines. The application may not change
168 the state of a live pipeline when a buffering message is received. Usually these
169 buffering messages contain the "buffering-mode" = "live".
171 The buffering message can also instruct the application to switch to a
172 periodical BUFFERING query instead, so it can more precisely control the
173 buffering process. The application can, for example, choose not to act on the
174 BUFFERING complete message (buffer-percent = 100) to resume playback but use
175 the estimated download time instead, resuming playback when it has determined
176 that it should be able to provide uninterrupted playback.
182 In addition to the BUFFERING messages posted by the buffering elements, we want
183 to be able to query the same information from the application. We also want to
184 be able to present the user with information about the downloaded range in the
185 file so that the GUI can react on it.
187 In addition to all the fields present in the buffering message, the BUFFERING
188 query contains the following field, which indicates the available downloaded
189 range in a specific format and the estimated time to complete:
191 "busy", G_TYPE_BOOLEAN
192 - if buffering was busy. This flag allows the application to pause the
193 pipeline by using the query only.
195 "format", GST_TYPE_FORMAT
196 - the format of the "start" and "stop" values below
198 "start", G_TYPE_INT64, -1 unknown
199 - the start position of the available data. If there are multiple ranges,
200 this field contains the start position of the currently downloading
203 "stop", G_TYPE_INT64, -1 unknown
204 - the stop position of the available data. If there are multiple ranges,
205 this field contains the stop position of the currently downloading
208 "estimated-total", G_TYPE_INT64
209 - gives the estimated download time in milliseconds. -1 unknown.
211 When the size of the downloaded file is known, this value will contain
212 the latest estimate of the remaining download time of the currently
213 downloading range. This value is usually only filled for the "download"
214 buffering mode. The application can use this information to estimate the
215 amount of remaining time to download till the end of the file.
217 "buffering-ranges", G_TYPE_ARRAY of GstQueryBufferingRange
218 - contains optionally the downloaded areas in the format given above. One
219 of the ranges contains the same start/stop position as above.
225 } GstQueryBufferingRange;
228 For the "download" and "timeshift" buffering-modes, the start and stop positions
229 specify the ranges where efficient seeking in the downloaded media is possible.
230 Seeking outside of these ranges might be slow or not at all possible.
232 For the "stream" and "live" mode the start and stop values describe the oldest
233 and newest item (expressed in "format") in the buffer.
239 Some defaults for common elements:
241 A GstBaseSrc with random access replies to the BUFFERING query with:
243 "buffer-percent" = 100
244 "buffering-mode" = "stream"
248 "format" = GST_FORMAT_BYTES
250 "stop" = the total filesize
251 "estimated-total" = 0
252 "buffering-ranges" = NULL
254 A GstBaseSrc in push mode replies to the BUFFERING query with:
256 "buffer-percent" = 100
257 "buffering-mode" = "stream"
261 "format" = a valid GST_TYPE_FORMAT
262 "start" = current position
263 "stop" = current position
264 "estimated-total" = -1
265 "buffering-ranges" = NULL
271 Buffering strategies are specific implementations based on the buffering
272 message and query described above.
274 Most strategies have to balance buffering time versus maximal playback
279 NON-live pipelines are kept in the paused state while buffering messages with
280 a percent < 100% are received.
282 This buffering strategy relies on the buffer size and low/high watermarks of
283 the element. It can work with a fixed size buffer in memory or on disk.
285 The size of the buffer is usually expressed in a fixed amount of time units
286 and the estimated bitrate of the upstream source is used to convert this time
289 All GStreamer applications must implement this strategy. Failure to do so
290 will result in starvation at the sink.
292 * no-rebuffer strategy
294 This strategy tries to buffer as much data as possible so that playback can
295 continue without any further rebuffering.
297 This strategy is initially similar to simple buffering, the difference is in
298 deciding on the condition to continue playback. When a 100% buffering message
299 has been received, the application will not yet start the playback but it will
300 start a periodic buffering query, which will return the estimated amount of
301 buffering time left. When the estimated time left is less than the remaining
302 playback time, playback can continue.
304 This strategy requires a unlimited buffer size in memory or on disk, such as
305 provided by elements that implement the incremental download buffering mode.
307 Usually, the application can choose to start playback even before the
308 remaining buffer time elapsed in order to more quickly start the playback at
309 the expense of a possible rebuffering phase.
311 * Incremental rebuffering
313 The application implements the simple buffering strategy but with each
314 rebuffering phase, it increases the size of the buffer.
316 This strategy has quick, fixed time startup times but incrementally longer
317 rebuffering times if the network is slower than the media bitrate.