4 This document describes the design for buffers.
6 A GstBuffer is the object that is passed from an upstream element to a
7 downstream element and contains memory and metadata information.
13 * allocation, free, low fragmentation
14 - Must be able to attach multiple memory blocks to the buffer
15 - Must be able to attach artbitrary metadata to buffers
16 - efficient handling of subbuffer, copy, span, trim
21 GstMemory extends from GstMiniObject and therefore uses its lifecycle
22 management (See part-miniobject.txt).
27 When a Buffers is writable as returned from gst_buffer_is_writable():
29 - metadata can be added/removed and the metadata can be changed
30 - GstMemory blocks can be added/removed
32 The individual memory blocks have their own locking and READONLY flags
33 that might influence their writability.
35 Buffers can be made writable with gst_buffer_make_writable(). This will copy the
36 buffer with the metadata and will ref the memory in the buffer. This means that
37 the memory is not automatically copied when copying buffers.
43 A GstBuffer contains an array of pointers to GstMemory objects.
45 When the buffer is writable, gst_buffer_insert_memory() can be used to add a
46 new GstMemory object to the buffer. When the array of memory is full, memory
47 will be merged to make room for the new memory object.
49 gst_buffer_n_memory() is used to get the amount of memory blocks on the
52 With gst_buffer_peek_memory(), memory can be retrieved from the memory array.
53 The desired access pattern for the memory block should be specified so that
54 appropriate checks can be made and, in case of GST_MAP_WRITE, a writable copy
55 can be constructed when needed.
57 gst_buffer_remove_memory_range() and gst_buffer_remove_memory() can be used to
58 remove memory from the GstBuffer.
64 Subbuffers are made by copying only a region of the memory blocks and copying
71 Spanning will merge together the data of 2 buffers into a new buffer
77 Accessing the data of the buffer can happen by retrieving the individual
78 GstMemory objects in the GstBuffer or by using the gst_buffer_map() and
79 gst_buffer_unmap() functions.
81 The _map and _unmap functions will always return the memory of all blocks as
82 one large contiguous region of memory. Using the _map and _unmap functions
83 might be more convenient than accessing the individual memory blocks at the
84 expense of being more expensive because it might perform memcpy operations.
86 For buffers with only one GstMemory object (the most common case), _map and
87 _unmap have no performance penalty at all.
90 * Read access with 1 memory block
92 The memory block is accessed and mapped for read access.
93 The memory block is unmapped after usage
95 * write access with 1 memory block
97 The buffer should be writable or this operation will fail.
98 The memory block is accessed. If the memory block is readonly, a copy is made
99 and the original memory block is replaced with this copy. Then the memory
100 block is mapped in write mode and unmapped after usage.
102 * Read access with multiple memory blocks
104 The memory blocks are combined into one large memory block. If the buffer is
105 writable, the memory blocks are replaced with this new combined block. If the
106 buffer is not writable, the memory is returned as is. The memory block is
107 then mapped in read mode.
109 When the memory is unmapped after usage and the buffer has multiple memory
110 blocks, this means that the map operation was not able to store the combined
111 buffer and it thus returned memory that should be freed. Otherwise, the memory
114 * Write access with multiple memory blocks
116 The buffer should be writable or the operation fails. The memory blocks are
117 combined into one large memory block and the existing blocks are replaced with
118 this new block. The memory is then mapped in write mode and unmapped after
125 Generating RTP packets from h264 video
126 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
128 We receive a GstBuffer as input with an encoded h264 image and we need to
129 create RTP packets containing this h264 data as the payload. We typically need
130 to fragment the h264 data into multiple packets, each with their own RTP and
131 payload specific header.
133 +-------+-------+---------------------------+--------+
134 input H264 buffer: | NALU1 | NALU2 | ..... | NALUx |
135 +-------+-------+---------------------------+--------+
138 array of +-+ +-------+ +-+ +-------+ +-+ +-------+
139 output buffers: | | | NALU1 | | | | NALU2 | .... | | | NALUx |
140 +-+ +-------+ +-+ +-------+ +-+ +-------+
142 \-----------/ \-----------/
145 The output buffer array consists of x buffers consisting of an RTP payload header
146 and a subbuffer of the original input H264 buffer. Since the rtp headers and
147 the h264 data don't need to be contiguous in memory, they are added to the buffer
148 as separate GstMemory blocks and we can avoid to memcpy the h264 data into
151 A typical udpsink will then use something like sendmsg to send the memory regions
152 on the network inside one UDP packet. This will further avoid having to memcpy
153 data into contiguous memory.
155 Using bufferlists, the complete array of output buffers can be pushed in one
156 operation to the peer element.