core that can be used by plugins and applications.
The purpose of buffering is to accumulate enough data in a pipeline so that
-playback can occur smoothly and without interruptions. It is typically done when
-reading from a (slow) and non-live network source.
+playback can occur smoothly and without interruptions. It is typically done
+when reading from a (slow) non-live network source but can also be used for
+live sources.
-While data is buffered, the pipeline should remain in the PAUSED state. It is
-also possible that more data should be buffered while the pipeline is PLAYING,
-in which case the pipeline should be PAUSED until the buffering finished.
+We want to be able to implement the following features:
+
+ - buffering up to a specific amount of data, in memory, before starting playback
+ so that network fluctuations are minimized.
+ - download of the network file to a local disk with fast seeking in the
+ downloaded data. This is similar to the quicktime/youtube players.
+ - caching of semi-live streams to a local, on disk, ringbuffer with seeking in
+ the cached area. This is similar to tivo-like timeshifting.
+ - progress report about the buffering operations
+ - the possibility for the application to do more complex buffering
+
+Some use cases:
+
+ * Stream buffering:
+
+ +---------+ +---------+ +-------+
+ | httpsrc | | buffer | | demux |
+ | src - sink src - sink ....
+ +---------+ +---------+ +-------+
+
+ In this case we are reading from a slow network source into a buffer element
+ (such as queue2).
+
+ The buffer element has a low and high watermark expressed in bytes. The
+ buffer uses the watermarks as follows:
+
+ - The buffer element will post BUFFERING messages until the high watermark
+ is hit. This instructs the application to keep the pipeline PAUSED, which
+ will eventually block the srcpad from pushing while data is prerolled in
+ the sinks.
+ - When the high watermark is hit, a BUFFERING message with 100% will be
+ posted, which instructs the application to continue playback.
+ - When the low watermark is hit during playback, the queue will start posting
+ BUFFERING messages again, making the application PAUSE the pipeline again
+ until the high watermark is hit again. This is called the rebuffering
+ stage.
+ - During playback, the queue level will fluctuate between the high and
+ low watermarks as a way to compensate for network irregularities.
+
+ This buffering method is usable when the demuxer operates in push mode.
+ Seeking in the stream requires the seek to happen in the network source.
+ It is mostly desirable when the total duration of the file is not known, such
+ as in live streaming or when efficient seeking is not possible/required.
+
+ * Incremental download
+
+ +---------+ +---------+ +-------+
+ | httpsrc | | buffer | | demux |
+ | src - sink src - sink ....
+ +---------+ +----|----+ +-------+
+ V
+ file
+
+ In this case, we know the server is streaming a fixed length file to the
+ client. The application can choose to download the file to disk. The buffer
+ element will provide a push or pull based srcpad to the demuxer to navigate in
+ the downloaded file.
+
+ This mode is only suitable when the client can determine the length of the
+ file on the server.
+
+ In this case, buffering messages will be emitted as usual when the requested
+ range is not within the downloaded area + buffersize. The buffering message
+ will also contain an indication that incremental download is being performed.
+ This flag can be used to let the application control the buffering in a more
+ intelligent way, using the BUFFERING query, for example.
+
+ The application can use the BUFFERING query to get the estimated download time
+ and match this time to the current/remaining playback time to control when
+ playback should start to have a non-interrupted playback experience.
+
+
+ * Timeshifting
+
+ +---------+ +---------+ +-------+
+ | httpsrc | | buffer | | demux |
+ | src - sink src - sink ....
+ +---------+ +----|----+ +-------+
+ V
+ file-ringbuffer
+
+ In this mode, a fixed size ringbuffer is kept to download the server content.
+ This allows for seeking in the buffered data. Depending on the size of the
+ buffer one can seek further back in time.
+
+ This mode is suitable for all live streams.
+
+ As with the incremental download mode, buffering messages are emitted along
+ with an indication that timeshifting download is in progress.
+
+
+ * Live buffering
+
+ In live pipelines we usually introduce some latency between the capture and
+ the playback elements. This latency can be introduced by a queue (such as a
+ jitterbuffer) or by other means (in the audiosink).
+
+ Buffering messages can be emitted in those live pipelines as well and serve as
+ an indication to the user of the latency buffering. The application usually
+ does not react to these buffering messages with a state change.
Messages
---------
+~~~~~~~~
+
+A GST_MESSAGE_BUFFERING must be posted on the bus when playback temporarily
+stops to buffer and when buffering finishes. When the percentage field in the
+BUFFERING message is 100, buffering is done. Values less than 100 mean that
+buffering is in progress.
+
+The BUFFERING message should be intercepted and acted upon by the application.
+The message contains at least one field that is sufficient for basic
+functionality:
+
+ "buffer-percent", G_TYPE_INT, between 0 and 100
+
+Several more clever ways of dealing with the buffering messages can be used when
+in incremental or timeshifting download mode. For this purpose additional fields
+are added to the buffering message:
+
+ "buffering-mode", GST_TYPE_BUFFERING_MODE,
+ enum { "stream", "download", "timeshift", "live" }
+ - Buffering mode in use. See above for an explanation of the
+ different alternatives. This field can be used to let the
+ application have more control over the buffering process.
+
+ "avg-in-rate", G_TYPE_INT
+ - Average input buffering speed in bytes/second. -1 is unknown.
+ This is the average number of bytes per second that is received on the
+ buffering element input (sink) pads. It is a measurement of the network
+ speed in most cases.
+
+ "avg-out-rate", G_TYPE_INT
+ - Average consumption speed in bytes/second. -1 is unknown.
+ This is the average number of bytes per second that is consumed by the
+ downstream element of the buffering element.
+
+ "buffering-left", G_TYPE_INT64
+ - Estimated time that buffering will take in milliseconds. -1 is unknown.
+ This is measured based on the avg-in-rate and the filled level of the
+ queue. The application can use this hint to update the GUI about the
+ estimated remaining time that buffering will take.
+
+Application
+~~~~~~~~~~~
+
+While data is buffered the pipeline should remain in the PAUSED state. It is
+also possible that more data should be buffered while the pipeline is PLAYING,
+in which case the pipeline should be PAUSED until the buffering finishes.
+
+BUFFERING messages can be posted while the pipeline is prerolling. The
+application should not set the pipeline to PLAYING before a BUFFERING message
+with a 100 percent value is received, which might only happen after the pipeline
+prerolls.
+
+An exception is made for live pipelines. The application may not change
+the state of a live pipeline when a buffering message is received. Usually these
+buffering messages contain the "buffering-mode" = "live".
+
+The buffering message can also instruct the application to switch to a
+periodical BUFFERING query instead, so it can more precisely control the
+buffering process. The application can, for example, choose not to act on the
+BUFFERING complete message (buffer-percent = 100) to resume playback but use
+the estimated download time instead, resuming playback when it has determined
+that it should be able to provide uninterrupted playback.
+
+
+Buffering Query
+~~~~~~~~~~~~~~~
+
+In addition to the BUFFERING messages posted by the buffering elements, we want
+to be able to query the same information from the application. We also want to
+be able to present the user with information about the downloaded range in the
+file so that the GUI can react on it.
+
+In addition to all the fields present in the buffering message, the BUFFERING
+query contains the following field, which indicates the available downloaded
+range in a specific format and the estimated time to complete:
+
+ "busy", G_TYPE_BOOLEAN
+ - if buffering was busy. This flag allows the application to pause the
+ pipeline by using the query only.
+
+ "format", GST_TYPE_FORMAT
+ - the format of the "start" and "stop" values below
+
+ "start", G_TYPE_INT64, -1 unknown
+ - the start position of the available data. If there are multiple ranges,
+ this field contains the start position of the currently downloading
+ range.
+
+ "stop", G_TYPE_INT64, -1 unknown
+ - the stop position of the available data. If there are multiple ranges,
+ this field contains the stop position of the currently downloading
+ range.
+
+ "estimated-total", G_TYPE_INT64
+ - gives the estimated download time in milliseconds. -1 unknown.
+
+ When the size of the downloaded file is known, this value will contain
+ the latest estimate of the remaining download time of the currently
+ downloading range. This value is usually only filled for the "download"
+ buffering mode. The application can use this information to estimate the
+ amount of remaining time to download till the end of the file.
+
+ "buffering-ranges", G_TYPE_ARRAY of GstQueryBufferingRange
+ - contains optionally the downloaded areas in the format given above. One
+ of the ranges contains the same start/stop position as above.
+
+ typedef struct
+ {
+ gint64 start;
+ gint64 stop;
+ } GstQueryBufferingRange;
+
+
+For the "download" and "timeshift" buffering-modes, the start and stop positions
+specify the ranges where efficient seeking in the downloaded media is possible.
+Seeking outside of these ranges might be slow or not at all possible.
+
+For the "stream" and "live" mode the start and stop values describe the oldest
+and newest item (expressed in "format") in the buffer.
+
+
+Defaults
+~~~~~~~~
+
+Some defaults for common elements:
+
+A GstBaseSrc with random access replies to the BUFFERING query with:
+
+ "buffer-percent" = 100
+ "buffering-mode" = "stream"
+ "avg-in-rate" = -1
+ "avg-out-rate" = -1
+ "buffering-left" = 0
+ "format" = GST_FORMAT_BYTES
+ "start" = 0
+ "stop" = the total filesize
+ "estimated-total" = 0
+ "buffering-ranges" = NULL
+
+A GstBaseSrc in push mode replies to the BUFFERING query with:
+
+ "buffer-percent" = 100
+ "buffering-mode" = "stream"
+ "avg-in-rate" = -1
+ "avg-out-rate" = -1
+ "buffering-left" = 0
+ "format" = a valid GST_TYPE_FORMAT
+ "start" = current position
+ "stop" = current position
+ "estimated-total" = -1
+ "buffering-ranges" = NULL
+
+
+Buffering strategies
+~~~~~~~~~~~~~~~~~~~~
+
+ Buffering strategies are specific implementations based on the buffering
+ message and query described above.
+
+ Most strategies have to balance buffering time versus maximal playback
+ experience.
+
+ * simple buffering
+
+ NON-live pipelines are kept in the paused state while buffering messages with
+ a percent < 100% are received.
+
+ This buffering strategy relies on the buffer size and low/high watermarks of
+ the element. It can work with a fixed size buffer in memory or on disk.
+
+ The size of the buffer is usually expressed in a fixed amount of time units
+ and the estimated bitrate of the upstream source is used to convert this time
+ to bytes.
+
+ All GStreamer applications must implement this strategy. Failure to do so
+ will result in starvation at the sink.
+
+ * no-rebuffer strategy
+
+ This strategy tries to buffer as much data as possible so that playback can
+ continue without any further rebuffering.
+
+ This strategy is initially similar to simple buffering, the difference is in
+ deciding on the condition to continue playback. When a 100% buffering message
+ has been received, the application will not yet start the playback but it will
+ start a periodic buffering query, which will return the estimated amount of
+ buffering time left. When the estimated time left is less than the remaining
+ playback time, playback can continue.
+
+ This strategy requires a unlimited buffer size in memory or on disk, such as
+ provided by elements that implement the incremental download buffering mode.
+
+ Usually, the application can choose to start playback even before the
+ remaining buffer time elapsed in order to more quickly start the playback at
+ the expense of a possible rebuffering phase.
+
+ * Incremental rebuffering
+
+ The application implements the simple buffering strategy but with each
+ rebuffering phase, it increases the size of the buffer.
+
+ This strategy has quick, fixed time startup times but incrementally longer
+ rebuffering times if the network is slower than the media bitrate.
+
+
+
+
projects / platform / upstream / gstreamer.git / blobdiff