7 The purpose of buffering is to accumulate enough data in a pipeline so
8 that playback can occur smoothly and without interruptions. It is
9 typically done when reading from a (slow) and non-live network source
10 but can also be used for live sources.
12 GStreamer provides support for the following use cases:
14 - Buffering up to a specific amount of data, in memory, before
15 starting playback so that network fluctuations are minimized. See
16 [Stream buffering](#stream-buffering).
18 - Download of the network file to a local disk with fast seeking in
19 the downloaded data. This is similar to the quicktime/youtube
20 players. See [Download buffering](#download-buffering).
22 - Caching of (semi)-live streams to a local, on disk, ringbuffer with
23 seeking in the cached area. This is similar to tivo-like
24 timeshifting. See [Timeshift buffering](#timeshift-buffering).
26 GStreamer can provide the application with progress reports about the
27 current buffering state as well as let the application decide on how to
28 buffer and when the buffering stops.
30 In the most simple case, the application has to listen for BUFFERING
31 messages on the bus. If the percent indicator inside the BUFFERING
32 message is smaller than 100, the pipeline is buffering. When a message
33 is received with 100 percent, buffering is complete. In the buffering
34 state, the application should keep the pipeline in the PAUSED state.
35 When buffering completes, it can put the pipeline (back) in the PLAYING
38 What follows is an example of how the message handler could deal with
39 the BUFFERING messages. We will see more advanced methods in [Buffering
40 strategies](#buffering-strategies).
46 switch (GST_MESSAGE_TYPE (message)) {
47 case GST_MESSAGE_BUFFERING:{
50 /* no state management needed for live pipelines */
54 gst_message_parse_buffering (message, &percent);
57 /* a 100% message means buffering is done */
59 /* if the desired state is playing, go back */
60 if (target_state == GST_STATE_PLAYING) {
61 gst_element_set_state (pipeline, GST_STATE_PLAYING);
65 if (!buffering && target_state == GST_STATE_PLAYING) {
66 /* we were not buffering but PLAYING, PAUSE the pipeline. */
67 gst_element_set_state (pipeline, GST_STATE_PAUSED);
82 +---------+ +---------+ +-------+
83 | httpsrc | | buffer | | demux |
84 | src - sink src - sink ....
85 +---------+ +---------+ +-------+
89 In this case we are reading from a slow network source into a buffer
90 element (such as queue2).
92 The buffer element has a low and high watermark expressed in bytes. The
93 buffer uses the watermarks as follows:
95 - The buffer element will post BUFFERING messages until the high
96 watermark is hit. This instructs the application to keep the
97 pipeline PAUSED, which will eventually block the srcpad from pushing
98 while data is prerolled in the sinks.
100 - When the high watermark is hit, a BUFFERING message with 100% will
101 be posted, which instructs the application to continue playback.
103 - When during playback, the low watermark is hit, the queue will start
104 posting BUFFERING messages again, making the application PAUSE the
105 pipeline again until the high watermark is hit again. This is called
106 the rebuffering stage.
108 - During playback, the queue level will fluctuate between the high and
109 the low watermark as a way to compensate for network irregularities.
111 This buffering method is usable when the demuxer operates in push mode.
112 Seeking in the stream requires the seek to happen in the network source.
113 It is mostly desirable when the total duration of the file is not known,
114 such as in live streaming or when efficient seeking is not
117 The problem is configuring a good low and high watermark. Here are some
120 - It is possible to measure the network bandwidth and configure the
121 low/high watermarks in such a way that buffering takes a fixed
124 The queue2 element in GStreamer core has the max-size-time property
125 that, together with the use-rate-estimate property, does exactly
126 that. Also the playbin buffer-duration property uses the rate
127 estimate to scale the amount of data that is buffered.
129 - Based on the codec bitrate, it is also possible to set the
130 watermarks in such a way that a fixed amount of data is buffered
131 before playback starts. Normally, the buffering element doesn't know
132 about the bitrate of the stream but it can get this with a query.
134 - Start with a fixed amount of bytes, measure the time between
135 rebuffering and increase the queue size until the time between
136 rebuffering is within the application's chosen limits.
138 The buffering element can be inserted anywhere in the pipeline. You
139 could, for example, insert the buffering element before a decoder. This
140 would make it possible to set the low/high watermarks based on time.
142 The buffering flag on playbin, performs buffering on the parsed data.
143 Another advantage of doing the buffering at a later stage is that you
144 can let the demuxer operate in pull mode. When reading data from a slow
145 network drive (with filesrc) this can be an interesting way to buffer.
147 ## Download buffering
150 +---------+ +---------+ +-------+
151 | httpsrc | | buffer | | demux |
152 | src - sink src - sink ....
153 +---------+ +----|----+ +-------+
159 If we know the server is streaming a fixed length file to the client,
160 the application can choose to download the entire file on disk. The
161 buffer element will provide a push or pull based srcpad to the demuxer
162 to navigate in the downloaded file.
164 This mode is only suitable when the client can determine the length of
165 the file on the server.
167 In this case, buffering messages will be emitted as usual when the
168 requested range is not within the downloaded area + buffersize. The
169 buffering message will also contain an indication that incremental
170 download is being performed. This flag can be used to let the
171 application control the buffering in a more intelligent way, using the
172 BUFFERING query, for example. See [Buffering
173 strategies](#buffering-strategies).
175 ## Timeshift buffering
178 +---------+ +---------+ +-------+
179 | httpsrc | | buffer | | demux |
180 | src - sink src - sink ....
181 +---------+ +----|----+ +-------+
187 In this mode, a fixed size ringbuffer is kept to download the server
188 content. This allows for seeking in the buffered data. Depending on the
189 size of the ringbuffer one can seek further back in time.
191 This mode is suitable for all live streams. As with the incremental
192 download mode, buffering messages are emitted along with an indication
193 that timeshifting download is in progress.
197 In live pipelines we usually introduce some fixed latency between the
198 capture and the playback elements. This latency can be introduced by a
199 queue (such as a jitterbuffer) or by other means (in the audiosink).
201 Buffering messages can be emitted in those live pipelines as well and
202 serve as an indication to the user of the latency buffering. The
203 application usually does not react to these buffering messages with a
206 ## Buffering strategies
208 What follows are some ideas for implementing different buffering
209 strategies based on the buffering messages and buffering query.
211 ### No-rebuffer strategy
213 We would like to buffer enough data in the pipeline so that playback
214 continues without interruptions. What we need to know to implement this
215 is know the total remaining playback time in the file and the total
216 remaining download time. If the buffering time is less than the playback
217 time, we can start playback without interruptions.
219 We have all this information available with the DURATION, POSITION and
220 BUFFERING queries. We need to periodically execute the buffering query
221 to get the current buffering status. We also need to have a large enough
222 buffer to hold the complete file, worst case. It is best to use this
223 buffering strategy with download buffering (see [Download
224 buffering](#download-buffering)).
226 This is what the code would look like:
233 GstState target_state;
234 static gboolean is_live;
235 static gboolean is_buffering;
238 buffer_timeout (gpointer data)
240 GstElement *pipeline = data;
244 gint64 estimated_total;
245 gint64 position, duration;
248 query = gst_query_new_buffering (GST_FORMAT_TIME);
250 if (!gst_element_query (pipeline, query))
253 gst_query_parse_buffering_percent (query, &busy, &percent);
254 gst_query_parse_buffering_range (query, NULL, NULL, NULL, &estimated_total);
256 if (estimated_total == -1)
259 /* calculate the remaining playback time */
260 if (!gst_element_query_position (pipeline, GST_FORMAT_TIME, &position))
262 if (!gst_element_query_duration (pipeline, GST_FORMAT_TIME, &duration))
265 if (duration != -1 && position != -1)
266 play_left = GST_TIME_AS_MSECONDS (duration - position);
270 g_message ("play_left %" G_GUINT64_FORMAT", estimated_total %" G_GUINT64_FORMAT
271 ", percent %d", play_left, estimated_total, percent);
273 /* we are buffering or the estimated download time is bigger than the
274 * remaining playback time. We keep buffering. */
275 is_buffering = (busy || estimated_total * 1.1 > play_left);
278 gst_element_set_state (pipeline, target_state);
284 on_message_buffering (GstBus *bus, GstMessage *message, gpointer user_data)
286 GstElement *pipeline = user_data;
289 /* no state management needed for live pipelines */
293 gst_message_parse_buffering (message, &percent);
299 if (target_state == GST_STATE_PLAYING) {
300 /* we were not buffering but PLAYING, PAUSE the pipeline. */
301 gst_element_set_state (pipeline, GST_STATE_PAUSED);
308 on_message_async_done (GstBus *bus, GstMessage *message, gpointer user_data)
310 GstElement *pipeline = user_data;
313 gst_element_set_state (pipeline, target_state);
315 g_timeout_add (500, buffer_timeout, pipeline);
322 GstElement *pipeline;
325 GstStateChangeReturn ret;
328 gst_init (&argc, &argv);
329 loop = g_main_loop_new (NULL, FALSE);
331 /* make sure we have a URI */
333 g_print ("Usage: %s <URI>\n", argv[0]);
338 pipeline = gst_element_factory_make ("playbin", "pipeline");
339 g_object_set (G_OBJECT (pipeline), "uri", argv[1], NULL);
340 g_object_set (G_OBJECT (pipeline), "flags", 0x697 , NULL);
342 bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
343 gst_bus_add_signal_watch (bus);
345 g_signal_connect (bus, "message::buffering",
346 (GCallback) on_message_buffering, pipeline);
347 g_signal_connect (bus, "message::async-done",
348 (GCallback) on_message_async_done, pipeline);
349 gst_object_unref (bus);
351 is_buffering = FALSE;
352 target_state = GST_STATE_PLAYING;
353 ret = gst_element_set_state (pipeline, GST_STATE_PAUSED);
356 case GST_STATE_CHANGE_SUCCESS:
360 case GST_STATE_CHANGE_FAILURE:
361 g_warning ("failed to PAUSE");
364 case GST_STATE_CHANGE_NO_PREROLL:
373 g_main_loop_run (loop);
376 gst_element_set_state (pipeline, GST_STATE_NULL);
377 gst_object_unref (GST_OBJECT (pipeline));
378 g_main_loop_unref (loop);
387 See how we set the pipeline to the PAUSED state first. We will receive
388 buffering messages during the preroll state when buffering is needed.
389 When we are prerolled (on\_message\_async\_done) we see if buffering is
390 going on, if not, we start playback. If buffering was going on, we start
391 a timeout to poll the buffering state. If the estimated time to download
392 is less than the remaining playback time, we start playback.