5 Pure JavaScript is Unicode friendly but not nice to binary data. When
6 dealing with TCP streams or the file system, it's necessary to handle octet
7 streams. io.js has several strategies for manipulating, creating, and
8 consuming octet streams.
10 Raw data is stored in instances of the `Buffer` class. A `Buffer` is similar
11 to an array of integers but corresponds to a raw memory allocation outside
12 the V8 heap. A `Buffer` cannot be resized.
14 The `Buffer` class is a global, making it very rare that one would need
15 to ever `require('buffer')`.
17 Converting between Buffers and JavaScript string objects requires an explicit
18 encoding method. Here are the different string encodings.
20 * `'ascii'` - for 7 bit ASCII data only. This encoding method is very fast, and
21 will strip the high bit if set.
23 * `'utf8'` - Multibyte encoded Unicode characters. Many web pages and other
24 document formats use UTF-8.
26 * `'utf16le'` - 2 or 4 bytes, little endian encoded Unicode characters.
27 Surrogate pairs (U+10000 to U+10FFFF) are supported.
29 * `'ucs2'` - Alias of `'utf16le'`.
31 * `'base64'` - Base64 string encoding.
33 * `'binary'` - A way of encoding raw binary data into strings by using only
34 the first 8 bits of each character. This encoding method is deprecated and
35 should be avoided in favor of `Buffer` objects where possible. This encoding
36 will be removed in future versions of io.js.
38 * `'hex'` - Encode each byte as two hexadecimal characters.
40 Creating a typed array from a `Buffer` works with the following caveats:
42 1. The buffer's memory is copied, not shared.
44 2. The buffer's memory is interpreted as an array, not a byte array. That is,
45 `new Uint32Array(new Buffer([1,2,3,4]))` creates a 4-element `Uint32Array`
46 with elements `[1,2,3,4]`, not an `Uint32Array` with a single element
47 `[0x1020304]` or `[0x4030201]`.
49 NOTE: Node.js v0.8 simply retained a reference to the buffer in `array.buffer`
50 instead of cloning it.
52 While more efficient, it introduces subtle incompatibilities with the typed
53 arrays specification. `ArrayBuffer#slice()` makes a copy of the slice while
54 `Buffer#slice()` creates a view.
58 The Buffer class is a global type for dealing with binary data directly.
59 It can be constructed in a variety of ways.
65 Allocates a new buffer of `size` octets. Note, `size` must be no more than
66 [kMaxLength](smalloc.html#smalloc_smalloc_kmaxlength). Otherwise, a `RangeError`
73 Allocates a new buffer using an `array` of octets.
75 ### new Buffer(buffer)
79 Copies the passed `buffer` data onto a new `Buffer` instance.
81 ### new Buffer(str[, encoding])
83 * `str` String - string to encode.
84 * `encoding` String - encoding to use, Optional.
86 Allocates a new buffer containing the given `str`.
87 `encoding` defaults to `'utf8'`.
89 ### Class Method: Buffer.isEncoding(encoding)
91 * `encoding` {String} The encoding string to test
93 Returns true if the `encoding` is a valid encoding argument, or false
96 ### Class Method: Buffer.isBuffer(obj)
101 Tests if `obj` is a `Buffer`.
103 ### Class Method: Buffer.byteLength(string[, encoding])
106 * `encoding` String, Optional, Default: 'utf8'
109 Gives the actual byte length of a string. `encoding` defaults to `'utf8'`.
110 This is not the same as `String.prototype.length` since that returns the
111 number of *characters* in a string.
115 str = '\u00bd + \u00bc = \u00be';
117 console.log(str + ": " + str.length + " characters, " +
118 Buffer.byteLength(str, 'utf8') + " bytes");
120 // ½ + ¼ = ¾: 9 characters, 12 bytes
122 ### Class Method: Buffer.concat(list[, totalLength])
124 * `list` {Array} List of Buffer objects to concat
125 * `totalLength` {Number} Total length of the buffers when concatenated
127 Returns a buffer which is the result of concatenating all the buffers in
130 If the list has no items, or if the totalLength is 0, then it returns a
133 If the list has exactly one item, then the first item of the list is
136 If the list has more than one item, then a new Buffer is created.
138 If totalLength is not provided, it is read from the buffers in the list.
139 However, this adds an additional loop to the function, so it is faster
140 to provide the length explicitly.
142 ### Class Method: Buffer.compare(buf1, buf2)
147 The same as [`buf1.compare(buf2)`](#buffer_buf_compare_otherbuffer). Useful
148 for sorting an Array of Buffers:
150 var arr = [Buffer('1234'), Buffer('0123')];
151 arr.sort(Buffer.compare);
158 The size of the buffer in bytes. Note that this is not necessarily the size
159 of the contents. `length` refers to the amount of memory allocated for the
160 buffer object. It does not change when the contents of the buffer are changed.
162 buf = new Buffer(1234);
164 console.log(buf.length);
165 buf.write("some string", 0, "ascii");
166 console.log(buf.length);
171 While the `length` property is not immutable, changing the value of `length`
172 can result in undefined and inconsistent behavior. Applications that wish to
173 modify the length of a buffer should therefore treat `length` as read-only and
174 use `buf.slice` to create a new buffer.
176 buf = new Buffer(10);
177 buf.write("abcdefghj", 0, "ascii");
178 console.log(buf.length); // 10
179 buf = buf.slice(0,5);
180 console.log(buf.length); // 5
182 ### buf.write(string[, offset][, length][, encoding])
184 * `string` String - data to be written to buffer
185 * `offset` Number, Optional, Default: 0
186 * `length` Number, Optional, Default: `buffer.length - offset`
187 * `encoding` String, Optional, Default: 'utf8'
189 Writes `string` to the buffer at `offset` using the given encoding.
190 `offset` defaults to `0`, `encoding` defaults to `'utf8'`. `length` is
191 the number of bytes to write. Returns number of octets written. If `buffer` did
192 not contain enough space to fit the entire string, it will write a partial
193 amount of the string. `length` defaults to `buffer.length - offset`.
194 The method will not write partial characters.
196 buf = new Buffer(256);
197 len = buf.write('\u00bd + \u00bc = \u00be', 0);
198 console.log(len + " bytes: " + buf.toString('utf8', 0, len));
200 ### buf.writeUIntLE(value, offset, byteLength[, noAssert])
201 ### buf.writeUIntBE(value, offset, byteLength[, noAssert])
202 ### buf.writeIntLE(value, offset, byteLength[, noAssert])
203 ### buf.writeIntBE(value, offset, byteLength[, noAssert])
205 * `value` {Number} Bytes to be written to buffer
206 * `offset` {Number} `0 <= offset <= buf.length`
207 * `byteLength` {Number} `0 < byteLength <= 6`
208 * `noAssert` {Boolean} Default: false
211 Writes `value` to the buffer at the specified `offset` and `byteLength`.
212 Supports up to 48 bits of accuracy. For example:
214 var b = new Buffer(6);
215 b.writeUIntBE(0x1234567890ab, 0, 6);
216 // <Buffer 12 34 56 78 90 ab>
218 Set `noAssert` to `true` to skip validation of `value` and `offset`. Defaults
221 ### buf.readUIntLE(offset, byteLength[, noAssert])
222 ### buf.readUIntBE(offset, byteLength[, noAssert])
223 ### buf.readIntLE(offset, byteLength[, noAssert])
224 ### buf.readIntBE(offset, byteLength[, noAssert])
226 * `offset` {Number} `0 <= offset <= buf.length`
227 * `byteLength` {Number} `0 < byteLength <= 6`
228 * `noAssert` {Boolean} Default: false
231 A generalized version of all numeric read methods. Supports up to 48 bits of
232 accuracy. For example:
234 var b = new Buffer(6);
235 b.writeUint16LE(0x90ab, 0);
236 b.writeUInt32LE(0x12345678, 2);
237 b.readUIntLE(0, 6).toString(16); // Specify 6 bytes (48 bits)
238 // output: '1234567890ab'
240 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
241 may be beyond the end of the buffer. Defaults to `false`.
243 ### buf.toString([encoding][, start][, end])
245 * `encoding` String, Optional, Default: 'utf8'
246 * `start` Number, Optional, Default: 0
247 * `end` Number, Optional, Default: `buffer.length`
249 Decodes and returns a string from buffer data encoded using the specified
250 character set encoding. If `encoding` is `undefined` or `null`, then `encoding`
251 defaults to `'utf8'. The `start` and `end` parameters default to `0` and
252 `buffer.length` when `undefined`.
254 buf = new Buffer(26);
255 for (var i = 0 ; i < 26 ; i++) {
256 buf[i] = i + 97; // 97 is ASCII a
258 buf.toString('ascii'); // outputs: abcdefghijklmnopqrstuvwxyz
259 buf.toString('ascii',0,5); // outputs: abcde
260 buf.toString('utf8',0,5); // outputs: abcde
261 buf.toString(undefined,0,5); // encoding defaults to 'utf8', outputs abcde
263 See `buffer.write()` example, above.
268 Returns a JSON-representation of the Buffer instance. `JSON.stringify`
269 implicitly calls this function when stringifying a Buffer instance.
273 var buf = new Buffer('test');
274 var json = JSON.stringify(buf);
277 // '{"type":"Buffer","data":[116,101,115,116]}'
279 var copy = JSON.parse(json, function(key, value) {
280 return value && value.type === 'Buffer'
281 ? new Buffer(value.data)
286 // <Buffer 74 65 73 74>
293 Get and set the octet at `index`. The values refer to individual bytes,
294 so the legal range is between `0x00` and `0xFF` hex or `0` and `255`.
296 Example: copy an ASCII string into a buffer, one byte at a time:
299 buf = new Buffer(str.length);
301 for (var i = 0; i < str.length ; i++) {
302 buf[i] = str.charCodeAt(i);
309 ### buf.equals(otherBuffer)
311 * `otherBuffer` {Buffer}
313 Returns a boolean of whether `this` and `otherBuffer` have the same
316 ### buf.compare(otherBuffer)
318 * `otherBuffer` {Buffer}
320 Returns a number indicating whether `this` comes before or after or is
321 the same as the `otherBuffer` in sort order.
324 ### buf.copy(targetBuffer[, targetStart][, sourceStart][, sourceEnd])
326 * `targetBuffer` Buffer object - Buffer to copy into
327 * `targetStart` Number, Optional, Default: 0
328 * `sourceStart` Number, Optional, Default: 0
329 * `sourceEnd` Number, Optional, Default: `buffer.length`
331 Copies data from a region of this buffer to a region in the target buffer even
332 if the target memory region overlaps with the source. If `undefined` the
333 `targetStart` and `sourceStart` parameters default to `0` while `sourceEnd`
334 defaults to `buffer.length`.
336 Example: build two Buffers, then copy `buf1` from byte 16 through byte 19
337 into `buf2`, starting at the 8th byte in `buf2`.
339 buf1 = new Buffer(26);
340 buf2 = new Buffer(26);
342 for (var i = 0 ; i < 26 ; i++) {
343 buf1[i] = i + 97; // 97 is ASCII a
344 buf2[i] = 33; // ASCII !
347 buf1.copy(buf2, 8, 16, 20);
348 console.log(buf2.toString('ascii', 0, 25));
350 // !!!!!!!!qrst!!!!!!!!!!!!!
352 Example: Build a single buffer, then copy data from one region to an overlapping
353 region in the same buffer
355 buf = new Buffer(26);
357 for (var i = 0 ; i < 26 ; i++) {
358 buf[i] = i + 97; // 97 is ASCII a
361 buf.copy(buf, 0, 4, 10);
362 console.log(buf.toString());
364 // efghijghijklmnopqrstuvwxyz
367 ### buf.slice([start][, end])
369 * `start` Number, Optional, Default: 0
370 * `end` Number, Optional, Default: `buffer.length`
372 Returns a new buffer which references the same memory as the old, but offset
373 and cropped by the `start` (defaults to `0`) and `end` (defaults to
374 `buffer.length`) indexes. Negative indexes start from the end of the buffer.
376 **Modifying the new buffer slice will modify memory in the original buffer!**
378 Example: build a Buffer with the ASCII alphabet, take a slice, then modify one
379 byte from the original Buffer.
381 var buf1 = new Buffer(26);
383 for (var i = 0 ; i < 26 ; i++) {
384 buf1[i] = i + 97; // 97 is ASCII a
387 var buf2 = buf1.slice(0, 3);
388 console.log(buf2.toString('ascii', 0, buf2.length));
390 console.log(buf2.toString('ascii', 0, buf2.length));
395 ### buf.readUInt8(offset[, noAssert])
398 * `noAssert` Boolean, Optional, Default: false
401 Reads an unsigned 8 bit integer from the buffer at the specified offset.
403 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
404 may be beyond the end of the buffer. Defaults to `false`.
408 var buf = new Buffer(4);
415 for (ii = 0; ii < buf.length; ii++) {
416 console.log(buf.readUInt8(ii));
424 ### buf.readUInt16LE(offset[, noAssert])
425 ### buf.readUInt16BE(offset[, noAssert])
428 * `noAssert` Boolean, Optional, Default: false
431 Reads an unsigned 16 bit integer from the buffer at the specified offset with
432 specified endian format.
434 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
435 may be beyond the end of the buffer. Defaults to `false`.
439 var buf = new Buffer(4);
446 console.log(buf.readUInt16BE(0));
447 console.log(buf.readUInt16LE(0));
448 console.log(buf.readUInt16BE(1));
449 console.log(buf.readUInt16LE(1));
450 console.log(buf.readUInt16BE(2));
451 console.log(buf.readUInt16LE(2));
460 ### buf.readUInt32LE(offset[, noAssert])
461 ### buf.readUInt32BE(offset[, noAssert])
464 * `noAssert` Boolean, Optional, Default: false
467 Reads an unsigned 32 bit integer from the buffer at the specified offset with
468 specified endian format.
470 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
471 may be beyond the end of the buffer. Defaults to `false`.
475 var buf = new Buffer(4);
482 console.log(buf.readUInt32BE(0));
483 console.log(buf.readUInt32LE(0));
488 ### buf.readInt8(offset[, noAssert])
491 * `noAssert` Boolean, Optional, Default: false
494 Reads a signed 8 bit integer from the buffer at the specified offset.
496 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
497 may be beyond the end of the buffer. Defaults to `false`.
499 Works as `buffer.readUInt8`, except buffer contents are treated as two's
500 complement signed values.
502 ### buf.readInt16LE(offset[, noAssert])
503 ### buf.readInt16BE(offset[, noAssert])
506 * `noAssert` Boolean, Optional, Default: false
509 Reads a signed 16 bit integer from the buffer at the specified offset with
510 specified endian format.
512 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
513 may be beyond the end of the buffer. Defaults to `false`.
515 Works as `buffer.readUInt16*`, except buffer contents are treated as two's
516 complement signed values.
518 ### buf.readInt32LE(offset[, noAssert])
519 ### buf.readInt32BE(offset[, noAssert])
522 * `noAssert` Boolean, Optional, Default: false
525 Reads a signed 32 bit integer from the buffer at the specified offset with
526 specified endian format.
528 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
529 may be beyond the end of the buffer. Defaults to `false`.
531 Works as `buffer.readUInt32*`, except buffer contents are treated as two's
532 complement signed values.
534 ### buf.readFloatLE(offset[, noAssert])
535 ### buf.readFloatBE(offset[, noAssert])
538 * `noAssert` Boolean, Optional, Default: false
541 Reads a 32 bit float from the buffer at the specified offset with specified
544 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
545 may be beyond the end of the buffer. Defaults to `false`.
549 var buf = new Buffer(4);
556 console.log(buf.readFloatLE(0));
560 ### buf.readDoubleLE(offset[, noAssert])
561 ### buf.readDoubleBE(offset[, noAssert])
564 * `noAssert` Boolean, Optional, Default: false
567 Reads a 64 bit double from the buffer at the specified offset with specified
570 Set `noAssert` to true to skip validation of `offset`. This means that `offset`
571 may be beyond the end of the buffer. Defaults to `false`.
575 var buf = new Buffer(8);
586 console.log(buf.readDoubleLE(0));
588 // 0.3333333333333333
590 ### buf.writeUInt8(value, offset[, noAssert])
594 * `noAssert` Boolean, Optional, Default: false
596 Writes `value` to the buffer at the specified offset. Note, `value` must be a
597 valid unsigned 8 bit integer.
599 Set `noAssert` to true to skip validation of `value` and `offset`. This means
600 that `value` may be too large for the specific function and `offset` may be
601 beyond the end of the buffer leading to the values being silently dropped. This
602 should not be used unless you are certain of correctness. Defaults to `false`.
606 var buf = new Buffer(4);
607 buf.writeUInt8(0x3, 0);
608 buf.writeUInt8(0x4, 1);
609 buf.writeUInt8(0x23, 2);
610 buf.writeUInt8(0x42, 3);
614 // <Buffer 03 04 23 42>
616 ### buf.writeUInt16LE(value, offset[, noAssert])
617 ### buf.writeUInt16BE(value, offset[, noAssert])
621 * `noAssert` Boolean, Optional, Default: false
623 Writes `value` to the buffer at the specified offset with specified endian
624 format. Note, `value` must be a valid unsigned 16 bit integer.
626 Set `noAssert` to true to skip validation of `value` and `offset`. This means
627 that `value` may be too large for the specific function and `offset` may be
628 beyond the end of the buffer leading to the values being silently dropped. This
629 should not be used unless you are certain of correctness. Defaults to `false`.
633 var buf = new Buffer(4);
634 buf.writeUInt16BE(0xdead, 0);
635 buf.writeUInt16BE(0xbeef, 2);
639 buf.writeUInt16LE(0xdead, 0);
640 buf.writeUInt16LE(0xbeef, 2);
644 // <Buffer de ad be ef>
645 // <Buffer ad de ef be>
647 ### buf.writeUInt32LE(value, offset[, noAssert])
648 ### buf.writeUInt32BE(value, offset[, noAssert])
652 * `noAssert` Boolean, Optional, Default: false
654 Writes `value` to the buffer at the specified offset with specified endian
655 format. Note, `value` must be a valid unsigned 32 bit integer.
657 Set `noAssert` to true to skip validation of `value` and `offset`. This means
658 that `value` may be too large for the specific function and `offset` may be
659 beyond the end of the buffer leading to the values being silently dropped. This
660 should not be used unless you are certain of correctness. Defaults to `false`.
664 var buf = new Buffer(4);
665 buf.writeUInt32BE(0xfeedface, 0);
669 buf.writeUInt32LE(0xfeedface, 0);
673 // <Buffer fe ed fa ce>
674 // <Buffer ce fa ed fe>
676 ### buf.writeInt8(value, offset[, noAssert])
680 * `noAssert` Boolean, Optional, Default: false
682 Writes `value` to the buffer at the specified offset. Note, `value` must be a
683 valid signed 8 bit integer.
685 Set `noAssert` to true to skip validation of `value` and `offset`. This means
686 that `value` may be too large for the specific function and `offset` may be
687 beyond the end of the buffer leading to the values being silently dropped. This
688 should not be used unless you are certain of correctness. Defaults to `false`.
690 Works as `buffer.writeUInt8`, except value is written out as a two's complement
691 signed integer into `buffer`.
693 ### buf.writeInt16LE(value, offset[, noAssert])
694 ### buf.writeInt16BE(value, offset[, noAssert])
698 * `noAssert` Boolean, Optional, Default: false
700 Writes `value` to the buffer at the specified offset with specified endian
701 format. Note, `value` must be a valid signed 16 bit integer.
703 Set `noAssert` to true to skip validation of `value` and `offset`. This means
704 that `value` may be too large for the specific function and `offset` may be
705 beyond the end of the buffer leading to the values being silently dropped. This
706 should not be used unless you are certain of correctness. Defaults to `false`.
708 Works as `buffer.writeUInt16*`, except value is written out as a two's
709 complement signed integer into `buffer`.
711 ### buf.writeInt32LE(value, offset[, noAssert])
712 ### buf.writeInt32BE(value, offset[, noAssert])
716 * `noAssert` Boolean, Optional, Default: false
718 Writes `value` to the buffer at the specified offset with specified endian
719 format. Note, `value` must be a valid signed 32 bit integer.
721 Set `noAssert` to true to skip validation of `value` and `offset`. This means
722 that `value` may be too large for the specific function and `offset` may be
723 beyond the end of the buffer leading to the values being silently dropped. This
724 should not be used unless you are certain of correctness. Defaults to `false`.
726 Works as `buffer.writeUInt32*`, except value is written out as a two's
727 complement signed integer into `buffer`.
729 ### buf.writeFloatLE(value, offset[, noAssert])
730 ### buf.writeFloatBE(value, offset[, noAssert])
734 * `noAssert` Boolean, Optional, Default: false
736 Writes `value` to the buffer at the specified offset with specified endian
737 format. Note, behavior is unspecified if `value` is not a 32 bit float.
739 Set `noAssert` to true to skip validation of `value` and `offset`. This means
740 that `value` may be too large for the specific function and `offset` may be
741 beyond the end of the buffer leading to the values being silently dropped. This
742 should not be used unless you are certain of correctness. Defaults to `false`.
746 var buf = new Buffer(4);
747 buf.writeFloatBE(0xcafebabe, 0);
751 buf.writeFloatLE(0xcafebabe, 0);
755 // <Buffer 4f 4a fe bb>
756 // <Buffer bb fe 4a 4f>
758 ### buf.writeDoubleLE(value, offset[, noAssert])
759 ### buf.writeDoubleBE(value, offset[, noAssert])
763 * `noAssert` Boolean, Optional, Default: false
765 Writes `value` to the buffer at the specified offset with specified endian
766 format. Note, `value` must be a valid 64 bit double.
768 Set `noAssert` to true to skip validation of `value` and `offset`. This means
769 that `value` may be too large for the specific function and `offset` may be
770 beyond the end of the buffer leading to the values being silently dropped. This
771 should not be used unless you are certain of correctness. Defaults to `false`.
775 var buf = new Buffer(8);
776 buf.writeDoubleBE(0xdeadbeefcafebabe, 0);
780 buf.writeDoubleLE(0xdeadbeefcafebabe, 0);
784 // <Buffer 43 eb d5 b7 dd f9 5f d7>
785 // <Buffer d7 5f f9 dd b7 d5 eb 43>
787 ### buf.fill(value[, offset][, end])
790 * `offset` Number, Optional
791 * `end` Number, Optional
793 Fills the buffer with the specified value. If the `offset` (defaults to `0`)
794 and `end` (defaults to `buffer.length`) are not given it will fill the entire
797 var b = new Buffer(50);
802 Creates iterator for buffer values (bytes). This function is called automatically
803 when `buffer` is used in a `for..of` statement.
807 Creates iterator for buffer keys (indices).
811 Creates iterator for `[index, byte]` arrays.
813 ## buffer.INSPECT_MAX_BYTES
815 * Number, Default: 50
817 How many bytes will be returned when `buffer.inspect()` is called. This can
818 be overridden by user modules.
820 Note that this is a property on the buffer module returned by
821 `require('buffer')`, not on the Buffer global, or a buffer instance.
825 Buffers can be iterated over using `for..of` syntax:
827 var buf = new Buffer([1, 2, 3]);
836 Additionally, `buffer.values()`, `buffer.keys()` and `buffer.entries()`
837 methods can be used to create iterators.
841 Returns an un-pooled `Buffer`.
843 In order to avoid the garbage collection overhead of creating many individually
844 allocated Buffers, by default allocations under 4KB are sliced from a single
845 larger allocated object. This approach improves both performance and memory
846 usage since v8 does not need to track and cleanup as many `Persistent` objects.
848 In the case where a developer may need to retain a small chunk of memory from a
849 pool for an indeterminate amount of time it may be appropriate to create an
850 un-pooled Buffer instance using SlowBuffer and copy out the relevant bits.
852 // need to keep around a few small chunks of memory
855 socket.on('readable', function() {
856 var data = socket.read();
857 // allocate for retained data
858 var sb = new SlowBuffer(10);
859 // copy the data into the new allocation
860 data.copy(sb, 0, 0, 10);
864 Though this should used sparingly and only be a last resort *after* a developer
865 has actively observed undue memory retention in their applications.