1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
\r
5 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15"/>
\r
6 <title>Ogg Vorbis Documentation</title>
\r
8 <style type="text/css">
\r
10 margin: 0 18px 0 18px;
\r
11 padding-bottom: 30px;
\r
12 font-family: Verdana, Arial, Helvetica, sans-serif;
\r
26 margin: 30px 0 16px 0;
\r
33 h1, h1 a, h2, h2 a, h3, h3 a {
\r
36 margin: 1.3em 0 8px 0;
\r
70 <a href="http://www.xiph.org/"><img src="fish_xiph_org.png" alt="Fish Logo and Xiph.org"/></a>
\r
73 <h1>Ogg logical bitstream framing</h1>
\r
75 <h2>Ogg bitstreams</h2>
\r
77 <p>The Ogg transport bitstream is designed to provide framing, error
\r
78 protection and seeking structure for higher-level codec streams that
\r
79 consist of raw, unencapsulated data packets, such as the Vorbis audio
\r
80 codec or Tarkin video codec.</p>
\r
82 <h2>Application example: Vorbis</h2>
\r
84 <p>Vorbis encodes short-time blocks of PCM data into raw packets of
\r
85 bit-packed data. These raw packets may be used directly by transport
\r
86 mechanisms that provide their own framing and packet-separation
\r
87 mechanisms (such as UDP datagrams). For stream based storage (such as
\r
88 files) and transport (such as TCP streams or pipes), Vorbis uses the
\r
89 Ogg bitstream format to provide framing/sync, sync recapture
\r
90 after error, landmarks during seeking, and enough information to
\r
91 properly separate data back into packets at the original packet
\r
92 boundaries without relying on decoding to find packet boundaries.</p>
\r
94 <h2>Design constraints for Ogg bitstreams</h2>
\r
97 <li>True streaming; we must not need to seek to build a 100%
\r
98 complete bitstream.</li>
\r
99 <li>Use no more than approximately 1-2% of bitstream bandwidth for
\r
100 packet boundary marking, high-level framing, sync and seeking.</li>
\r
101 <li>Specification of absolute position within the original sample
\r
103 <li>Simple mechanism to ease limited editing, such as a simplified
\r
104 concatenation mechanism.</li>
\r
105 <li>Detection of corruption, recapture after error and direct, random
\r
106 access to data at arbitrary positions in the bitstream.</li>
\r
109 <h2>Logical and Physical Bitstreams</h2>
\r
111 <p>A <em>logical</em> Ogg bitstream is a contiguous stream of
\r
112 sequential pages belonging only to the logical bitstream. A
\r
113 <em>physical</em> Ogg bitstream is constructed from one or more
\r
114 than one logical Ogg bitstream (the simplest physical bitstream
\r
115 is simply a single logical bitstream). We describe below the exact
\r
116 formatting of an Ogg logical bitstream. Combining logical
\r
117 bitstreams into more complex physical bitstreams is described in the
\r
118 <a href="oggstream.html">Ogg bitstream overview</a>. The exact
\r
119 mapping of raw Vorbis packets into a valid Ogg Vorbis physical
\r
120 bitstream is described in <a href="vorbis-stream.html">Vorbis
\r
121 bitstream mapping</a>.</p>
\r
123 <h2>Bitstream structure</h2>
\r
125 <p>An Ogg stream is structured by dividing incoming packets into
\r
126 segments of up to 255 bytes and then wrapping a group of contiguous
\r
127 packet segments into a variable length page preceded by a page
\r
128 header. Both the header size and page size are variable; the page
\r
129 header contains sizing information and checksum data to determine
\r
130 header/page size and data integrity.</p>
\r
132 <p>The bitstream is captured (or recaptured) by looking for the beginning
\r
133 of a page, specifically the capture pattern. Once the capture pattern
\r
134 is found, the decoder verifies page sync and integrity by computing
\r
135 and comparing the checksum. At that point, the decoder can extract the
\r
136 packets themselves.</p>
\r
138 <h3>Packet segmentation</h3>
\r
140 <p>Packets are logically divided into multiple segments before encoding
\r
141 into a page. Note that the segmentation and fragmentation process is a
\r
142 logical one; it's used to compute page header values and the original
\r
143 page data need not be disturbed, even when a packet spans page
\r
146 <p>The raw packet is logically divided into [n] 255 byte segments and a
\r
147 last fractional segment of < 255 bytes. A packet size may well
\r
148 consist only of the trailing fractional segment, and a fractional
\r
149 segment may be zero length. These values, called "lacing values" are
\r
150 then saved and placed into the header segment table.</p>
\r
152 <p>An example should make the basic concept clear:</p>
\r
157 ___________________________________________
\r
158 |______________packet data__________________| 753 bytes
\r
160 lacing values for page header segment table: 255,255,243
\r
164 <p>We simply add the lacing values for the total size; the last lacing
\r
165 value for a packet is always the value that is less than 255. Note
\r
166 that this encoding both avoids imposing a maximum packet size as well
\r
167 as imposing minimum overhead on small packets (as opposed to, eg,
\r
168 simply using two bytes at the head of every packet and having a max
\r
169 packet size of 32k. Small packets (<255, the typical case) are
\r
170 penalized with twice the segmentation overhead). Using the lacing
\r
171 values as suggested, small packets see the minimum possible
\r
172 byte-aligned overheade (1 byte) and large packets, over 512 bytes or
\r
173 so, see a fairly constant ~.5% overhead on encoding space.</p>
\r
175 <p>Note that a lacing value of 255 implies that a second lacing value
\r
176 follows in the packet, and a value of < 255 marks the end of the
\r
177 packet after that many additional bytes. A packet of 255 bytes (or a
\r
178 multiple of 255 bytes) is terminated by a lacing value of 0:</p>
\r
182 _______________________________
\r
183 |________packet data____________| 255 bytes
\r
185 lacing values: 255, 0
\r
188 <p>Note also that a 'nil' (zero length) packet is not an error; it
\r
189 consists of nothing more than a lacing value of zero in the header.</p>
\r
191 <h3>Packets spanning pages</h3>
\r
193 <p>Packets are not restricted to beginning and ending within a page,
\r
194 although individual segments are, by definition, required to do so.
\r
195 Packets are not restricted to a maximum size, although excessively
\r
196 large packets in the data stream are discouraged; the Ogg
\r
197 bitstream specification strongly recommends nominal page size of
\r
198 approximately 4-8kB (large packets are foreseen as being useful for
\r
199 initialization data at the beginning of a logical bitstream).</p>
\r
201 <p>After segmenting a packet, the encoder may decide not to place all the
\r
202 resulting segments into the current page; to do so, the encoder places
\r
203 the lacing values of the segments it wishes to belong to the current
\r
204 page into the current segment table, then finishes the page. The next
\r
205 page is begun with the first value in the segment table belonging to
\r
206 the next packet segment, thus continuing the packet (data in the
\r
207 packet body must also correspond properly to the lacing values in the
\r
208 spanned pages. The segment data in the first packet corresponding to
\r
209 the lacing values of the first page belong in that page; packet
\r
210 segments listed in the segment table of the following page must begin
\r
211 the page body of the subsequent page).</p>
\r
213 <p>The last mechanic to spanning a page boundary is to set the header
\r
214 flag in the new page to indicate that the first lacing value in the
\r
215 segment table continues rather than begins a packet; a header flag of
\r
216 0x01 is set to indicate a continued packet. Although mandatory, it
\r
217 is not actually algorithmically necessary; one could inspect the
\r
218 preceding segment table to determine if the packet is new or
\r
219 continued. Adding the information to the packet_header flag allows a
\r
220 simpler design (with no overhead) that needs only inspect the current
\r
221 page header after frame capture. This also allows faster error
\r
222 recovery in the event that the packet originates in a corrupt
\r
223 preceding page, implying that the previous page's segment table
\r
224 cannot be trusted.</p>
\r
226 <p>Note that a packet can span an arbitrary number of pages; the above
\r
227 spanning process is repeated for each spanned page boundary. Also a
\r
228 'zero termination' on a packet size that is an even multiple of 255
\r
229 must appear even if the lacing value appears in the next page as a
\r
230 zero-length continuation of the current packet. The header flag
\r
231 should be set to 0x01 to indicate that the packet spanned, even though
\r
232 the span is a nil case as far as data is concerned.</p>
\r
234 <p>The encoding looks odd, but is properly optimized for speed and the
\r
235 expected case of the majority of packets being between 50 and 200
\r
236 bytes (note that it is designed such that packets of wildly different
\r
237 sizes can be handled within the model; placing packet size
\r
238 restrictions on the encoder would have only slightly simplified design
\r
239 in page generation and increased overall encoder complexity).</p>
\r
241 <p>The main point behind tracking individual packets (and packet
\r
242 segments) is to allow more flexible encoding tricks that requiring
\r
243 explicit knowledge of packet size. An example is simple bandwidth
\r
244 limiting, implemented by simply truncating packets in the nominal case
\r
245 if the packet is arranged so that the least sensitive portion of the
\r
246 data comes last.</p>
\r
248 <h3>Page header</h3>
\r
250 <p>The headering mechanism is designed to avoid copying and re-assembly
\r
251 of the packet data (ie, making the packet segmentation process a
\r
252 logical one); the header can be generated directly from incoming
\r
253 packet data. The encoder buffers packet data until it finishes a
\r
254 complete page at which point it writes the header followed by the
\r
255 buffered packet segments.</p>
\r
257 <h4>capture_pattern</h4>
\r
259 <p>A header begins with a capture pattern that simplifies identifying
\r
260 pages; once the decoder has found the capture pattern it can do a more
\r
261 intensive job of verifying that it has in fact found a page boundary
\r
262 (as opposed to an inadvertent coincidence in the byte stream).</p>
\r
273 <h4>stream_structure_version</h4>
\r
275 <p>The capture pattern is followed by the stream structure revision:</p>
\r
283 <h4>header_type_flag</h4>
\r
285 <p>The header type flag identifies this page's context in the bitstream:</p>
\r
290 5 bitflags: 0x01: unset = fresh packet
\r
291 set = continued packet
\r
292 0x02: unset = not first page of logical bitstream
\r
293 set = first page of logical bitstream (bos)
\r
294 0x04: unset = not last page of logical bitstream
\r
295 set = last page of logical bitstream (eos)
\r
298 <h4>absolute granule position</h4>
\r
300 <p>(This is packed in the same way the rest of Ogg data is packed; LSb
\r
301 of LSB first. Note that the 'position' data specifies a 'sample'
\r
302 number (eg, in a CD quality sample is four octets, 16 bits for left
\r
303 and 16 bits for right; in video it would likely be the frame number.
\r
304 It is up to the specific codec in use to define the semantic meaning
\r
305 of the granule position value). The position specified is the total
\r
306 samples encoded after including all packets finished on this page
\r
307 (packets begun on this page but continuing on to the next page do not
\r
308 count). The rationale here is that the position specified in the
\r
309 frame header of the last page tells how long the data coded by the
\r
310 bitstream is. A truncated stream will still return the proper number
\r
311 of samples that can be decoded fully.</p>
\r
313 <p>A special value of '-1' (in two's complement) indicates that no packets
\r
314 finish on this page.</p>
\r
329 <h4>stream serial number</h4>
\r
331 <p>Ogg allows for separate logical bitstreams to be mixed at page
\r
332 granularity in a physical bitstream. The most common case would be
\r
333 sequential arrangement, but it is possible to interleave pages for
\r
334 two separate bitstreams to be decoded concurrently. The serial
\r
335 number is the means by which pages physical pages are associated with
\r
336 a particular logical stream. Each logical stream must have a unique
\r
337 serial number within a physical stream:</p>
\r
348 <h4>page sequence no</h4>
\r
350 <p>Page counter; lets us know if a page is lost (useful where packets
\r
351 span page boundaries).</p>
\r
362 <h4>page checksum</h4>
\r
364 <p>32 bit CRC value (direct algorithm, initial val and final XOR = 0,
\r
365 generator polynomial=0x04c11db7). The value is computed over the
\r
366 entire header (with the CRC field in the header set to zero) and then
\r
367 continued over the page. The CRC field is then filled with the
\r
368 computed value.</p>
\r
370 <p>(A thorough discussion of CRC algorithms can be found in <a
\r
371 href="ftp://ftp.rocksoft.com/papers/crc_v3.txt">"A
\r
372 Painless Guide to CRC Error Detection Algorithms"</a> by Ross
\r
374 href="mailto:ross@guest.adelaide.edu.au">ross@guest.adelaide.edu.au</a>.)</p>
\r
385 <h4>page_segments</h4>
\r
387 <p>The number of segment entries to appear in the segment table. The
\r
388 maximum number of 255 segments (255 bytes each) sets the maximum
\r
389 possible physical page size at 65307 bytes or just under 64kB (thus
\r
390 we know that a header corrupted so as destroy sizing/alignment
\r
391 information will not cause a runaway bitstream. We'll read in the
\r
392 page according to the corrupted size information that's guaranteed to
\r
393 be a reasonable size regardless, notice the checksum mismatch, drop
\r
394 sync and then look for recapture).</p>
\r
399 26 0x00-0xff (0-255)
\r
402 <h4>segment_table (containing packet lacing values)</h4>
\r
404 <p>The lacing values for each packet segment physically appearing in
\r
405 this page are listed in contiguous order.</p>
\r
410 27 0x00-0xff (0-255)
\r
412 n 0x00-0xff (0-255, n=page_segments+26)
\r
415 <p>Total page size is calculated directly from the known header size and
\r
416 lacing values in the segment table. Packet data segments follow
\r
417 immediately after the header.</p>
\r
419 <p>Page headers typically impose a flat .25-.5% space overhead assuming
\r
420 nominal ~8k page sizes. The segmentation table needed for exact
\r
421 packet recovery in the streaming layer adds approximately .5-1%
\r
422 nominal assuming expected encoder behavior in the 44.1kHz, 128kbps
\r
423 stereo encodings.</p>
\r
425 <div id="copyright">
\r
426 The Xiph Fish Logo is a
\r
427 trademark (™) of Xiph.Org.<br/>
\r
429 These pages © 1994 - 2005 Xiph.Org. All rights reserved.
\r
projects / platform / upstream / libvorbis.git / blob