2 Copyright 2011-2016 Beman Dawes
4 Distributed under the Boost Software License, Version 1.0.
5 (http://www.boost.org/LICENSE_1_0.txt)
9 # Endian Arithmetic Types
10 :idprefix: arithmetic_
14 Header `boost/endian/arithmetic.hpp` provides integer binary types with
15 control over byte order, value type, size, and alignment. Typedefs provide
16 easy-to-use names for common configurations.
18 These types provide portable byte-holders for integer data, independent of
19 particular computer architectures. Use cases almost always involve I/O, either
20 via files or network connections. Although data portability is the primary
21 motivation, these integer byte-holders may also be used to reduce memory use,
22 file size, or network activity since they provide binary integer sizes not
25 Such integer byte-holder types are traditionally called *endian* types. See the
26 http://en.wikipedia.org/wiki/Endian[Wikipedia] for a full exploration of
27 *endianness*, including definitions of *big endian* and *little endian*.
29 Boost endian integers provide the same full set of {cpp} assignment, arithmetic,
30 and relational operators as {cpp} standard integral types, with the standard
33 Unary arithmetic operators are `+`, `-`, `~`, `!`, plus both prefix and postfix
34 `--` and `++`. Binary arithmetic operators are `+`, `+=`, `-`, `-=`, `\*`,
35 ``*=``, `/`, `/=`, `&`, `&=`, `|`, `|=`, `^`, `^=`, `<<`, `<\<=`, `>>`, and
36 `>>=`. Binary relational operators are `==`, `!=`, `<`, `\<=`, `>`, and `>=`.
38 Implicit conversion to the underlying value type is provided. An implicit
39 constructor converting from the underlying value type is provided.
42 The `endian_example.cpp` program writes a binary file containing four-byte,
43 big-endian and little-endian integers:
48 #include <boost/endian/arithmetic.hpp>
49 #include <boost/static_assert.hpp>
51 using namespace boost::endian;
55 // This is an extract from a very widely used GIS file format.
56 // Why the designer decided to mix big and little endians in
57 // the same file is not known. But this is a real-world format
58 // and users wishing to write low level code manipulating these
59 // files have to deal with the mixed endianness.
63 big_int32_t file_code;
64 big_int32_t file_length;
65 little_int32_t version;
66 little_int32_t shape_type;
69 const char* filename = "test.dat";
72 int main(int, char* [])
76 BOOST_STATIC_ASSERT(sizeof(h) == 16U); // reality check
78 h.file_code = 0x01020304;
79 h.file_length = sizeof(header);
81 h.shape_type = 0x01020304;
83 // Low-level I/O such as POSIX read/write or <cstdio>
84 // fread/fwrite is sometimes used for binary file operations
85 // when ultimate efficiency is important. Such I/O is often
86 // performed in some C++ wrapper class, but to drive home the
87 // point that endian integers are often used in fairly
88 // low-level code that does bulk I/O operations, <cstdio>
89 // fopen/fwrite is used for I/O in this example.
91 std::FILE* fi = std::fopen(filename, "wb"); // MUST BE BINARY
95 std::cout << "could not open " << filename << '\n';
99 if (std::fwrite(&h, sizeof(header), 1, fi) != 1)
101 std::cout << "write failure for " << filename << '\n';
107 std::cout << "created file " << filename << '\n';
113 After compiling and executing `endian_example.cpp`, a hex dump of `test.dat`
117 01020304 00000010 01000000 04030201
120 Notice that the first two 32-bit integers are big endian while the second two
121 are little endian, even though the machine this was compiled and run on was
126 Requires `<climits>`, `CHAR_BIT == 8`. If `CHAR_BIT` is some other value,
127 compilation will result in an `#error`. This restriction is in place because the
128 design, implementation, testing, and documentation has only considered issues
129 related to 8-bit bytes, and there have been no real-world use cases presented
132 In {cpp}03, `endian_arithmetic` does not meet the requirements for POD types
133 because it has constructors, private data members, and a base class. This means
134 that common use cases are relying on unspecified behavior in that the {cpp}
135 Standard does not guarantee memory layout for non-POD types. This has not been a
136 problem in practice since all known {cpp} compilers lay out memory as if
137 `endian` were a POD type. In {cpp}11, it is possible to specify the default
138 constructor as trivial, and private data members and base classes no longer
139 disqualify a type from being a POD type. Thus under {cpp}11, `endian_arithmetic`
140 will no longer be relying on unspecified behavior.
144 * Big endian| little endian | native endian byte ordering.
146 * Unaligned | aligned
147 * 1-8 byte (unaligned) | 1, 2, 4, 8 byte (aligned)
148 * Choice of value type
150 ## Enums and typedefs
152 Two scoped enums are provided:
155 enum class order { big, little, native };
157 enum class align { no, yes };
160 One class template is provided:
163 template <order Order, typename T, std::size_t n_bits,
164 align Align = align::no>
165 class endian_arithmetic;
168 Typedefs, such as `big_int32_t`, provide convenient naming conventions for
173 |Name |Alignment |Endianness |Sign |Sizes in bits (n)
174 |`big_intN_t` |no |big |signed |8,16,24,32,40,48,56,64
175 |`big_uintN_t` |no |big |unsigned |8,16,24,32,40,48,56,64
176 |`little_intN_t` |no |little |signed |8,16,24,32,40,48,56,64
177 |`little_uintN_t` |no |little |unsigned |8,16,24,32,40,48,56,64
178 |`native_intN_t` |no |native |signed |8,16,24,32,40,48,56,64
179 |`native_uintN_t` |no |native |unsigned |8,16,24,32,40,48,56,64
180 |`big_intN_at` |yes |big |signed |8,16,32,64
181 |`big_uintN_at` |yes |big |unsigned |8,16,32,64
182 |`little_intN_at` |yes |little |signed |8,16,32,64
183 |`little_uintN_at` |yes |little |unsigned |8,16,32,64
186 The unaligned types do not cause compilers to insert padding bytes in classes
187 and structs. This is an important characteristic that can be exploited to
188 minimize wasted space in memory, files, and network transmissions.
190 CAUTION: Code that uses aligned types is possibly non-portable because
191 alignment requirements vary between hardware architectures and because
192 alignment may be affected by compiler switches or pragmas. For example,
193 alignment of an 64-bit integer may be to a 32-bit boundary on a 32-bit machine.
194 Furthermore, aligned types are only available on architectures with 8, 16, 32,
195 and 64-bit integer types.
197 TIP: Prefer unaligned arithmetic types.
199 TIP: Protect yourself against alignment ills. For example:
204 static_assert(sizeof(containing_struct) == 12, "sizeof(containing_struct) is wrong");
207 NOTE: One-byte arithmetic types have identical layout on all platforms, so they
208 never actually reverse endianness. They are provided to enable generic code,
209 and to improve code readability and searchability.
211 ## Class template `endian_arithmetic`
213 An `endian_integer` is an integer byte-holder with user-specified endianness,
214 value type, size, and alignment. The usual operations on arithmetic types are
220 #include <boost/endian/buffers.hpp>
226 // C++11 features emulated if not available
228 enum class align { no, yes };
230 template <order Order, class T, std::size_t n_bits,
231 align Align = align::no>
232 class endian_arithmetic
233 : public endian_buffer<Order, T, n_bits, Align>
237 typedef T value_type;
239 // if BOOST_ENDIAN_FORCE_PODNESS is defined && C++11 PODs are not
240 // available then these two constructors will not be present
241 endian_arithmetic() noexcept = default;
242 endian_arithmetic(T v) noexcept;
244 endian_arithmetic& operator=(T v) noexcept;
245 operator value_type() const noexcept;
246 value_type value() const noexcept; // for exposition; see endian_buffer
247 unsigned char* data() noexcept; // for exposition; see endian_buffer
248 unsigned char const* data() const noexcept; // for exposition; see endian_buffer
250 // arithmetic operations
251 // note that additional operations are provided by the value_type
252 value_type operator+() const noexcept;
253 endian_arithmetic& operator+=(value_type y) noexcept;
254 endian_arithmetic& operator-=(value_type y) noexcept;
255 endian_arithmetic& operator*=(value_type y) noexcept;
256 endian_arithmetic& operator/=(value_type y) noexcept;
257 endian_arithmetic& operator%=(value_type y) noexcept;
258 endian_arithmetic& operator&=(value_type y) noexcept;
259 endian_arithmetic& operator|=(value_type y) noexcept;
260 endian_arithmetic& operator^=(value_type y) noexcept;
261 endian_arithmetic& operator<<=(value_type y) noexcept;
262 endian_arithmetic& operator>>=(value_type y) noexcept;
263 endian_arithmetic& operator++() noexcept;
264 endian_arithmetic& operator--() noexcept;
265 endian_arithmetic operator++(int) noexcept;
266 endian_arithmetic operator--(int) noexcept;
269 template <class charT, class traits>
270 friend std::basic_ostream<charT, traits>&
271 operator<<(std::basic_ostream<charT, traits>& os, const endian_arithmetic& x);
274 template <class charT, class traits>
275 friend std::basic_istream<charT, traits>&
276 operator>>(std::basic_istream<charT, traits>& is, endian_arithmetic& x);
281 // unaligned big endian signed integer types
282 typedef endian_arithmetic<order::big, int_least8_t, 8> big_int8_t;
283 typedef endian_arithmetic<order::big, int_least16_t, 16> big_int16_t;
284 typedef endian_arithmetic<order::big, int_least32_t, 24> big_int24_t;
285 typedef endian_arithmetic<order::big, int_least32_t, 32> big_int32_t;
286 typedef endian_arithmetic<order::big, int_least64_t, 40> big_int40_t;
287 typedef endian_arithmetic<order::big, int_least64_t, 48> big_int48_t;
288 typedef endian_arithmetic<order::big, int_least64_t, 56> big_int56_t;
289 typedef endian_arithmetic<order::big, int_least64_t, 64> big_int64_t;
291 // unaligned big endian unsigned integer types
292 typedef endian_arithmetic<order::big, uint_least8_t, 8> big_uint8_t;
293 typedef endian_arithmetic<order::big, uint_least16_t, 16> big_uint16_t;
294 typedef endian_arithmetic<order::big, uint_least32_t, 24> big_uint24_t;
295 typedef endian_arithmetic<order::big, uint_least32_t, 32> big_uint32_t;
296 typedef endian_arithmetic<order::big, uint_least64_t, 40> big_uint40_t;
297 typedef endian_arithmetic<order::big, uint_least64_t, 48> big_uint48_t;
298 typedef endian_arithmetic<order::big, uint_least64_t, 56> big_uint56_t;
299 typedef endian_arithmetic<order::big, uint_least64_t, 64> big_uint64_t;
301 // unaligned big endian floating point types
302 typedef endian_arithmetic<order::big, float, 32> big_float32_t;
303 typedef endian_arithmetic<order::big, double, 64> big_float64_t;
305 // unaligned little endian signed integer types
306 typedef endian_arithmetic<order::little, int_least8_t, 8> little_int8_t;
307 typedef endian_arithmetic<order::little, int_least16_t, 16> little_int16_t;
308 typedef endian_arithmetic<order::little, int_least32_t, 24> little_int24_t;
309 typedef endian_arithmetic<order::little, int_least32_t, 32> little_int32_t;
310 typedef endian_arithmetic<order::little, int_least64_t, 40> little_int40_t;
311 typedef endian_arithmetic<order::little, int_least64_t, 48> little_int48_t;
312 typedef endian_arithmetic<order::little, int_least64_t, 56> little_int56_t;
313 typedef endian_arithmetic<order::little, int_least64_t, 64> little_int64_t;
315 // unaligned little endian unsigned integer types
316 typedef endian_arithmetic<order::little, uint_least8_t, 8> little_uint8_t;
317 typedef endian_arithmetic<order::little, uint_least16_t, 16> little_uint16_t;
318 typedef endian_arithmetic<order::little, uint_least32_t, 24> little_uint24_t;
319 typedef endian_arithmetic<order::little, uint_least32_t, 32> little_uint32_t;
320 typedef endian_arithmetic<order::little, uint_least64_t, 40> little_uint40_t;
321 typedef endian_arithmetic<order::little, uint_least64_t, 48> little_uint48_t;
322 typedef endian_arithmetic<order::little, uint_least64_t, 56> little_uint56_t;
323 typedef endian_arithmetic<order::little, uint_least64_t, 64> little_uint64_t;
325 // unaligned little endian floating point types
326 typedef endian_arithmetic<order::little, float, 32> little_float32_t;
327 typedef endian_arithmetic<order::little, double, 64> little_float64_t;
329 // unaligned native endian signed integer types
330 typedef implementation-defined_int8_t native_int8_t;
331 typedef implementation-defined_int16_t native_int16_t;
332 typedef implementation-defined_int24_t native_int24_t;
333 typedef implementation-defined_int32_t native_int32_t;
334 typedef implementation-defined_int40_t native_int40_t;
335 typedef implementation-defined_int48_t native_int48_t;
336 typedef implementation-defined_int56_t native_int56_t;
337 typedef implementation-defined_int64_t native_int64_t;
339 // unaligned native endian unsigned integer types
340 typedef implementation-defined_uint8_t native_uint8_t;
341 typedef implementation-defined_uint16_t native_uint16_t;
342 typedef implementation-defined_uint24_t native_uint24_t;
343 typedef implementation-defined_uint32_t native_uint32_t;
344 typedef implementation-defined_uint40_t native_uint40_t;
345 typedef implementation-defined_uint48_t native_uint48_t;
346 typedef implementation-defined_uint56_t native_uint56_t;
347 typedef implementation-defined_uint64_t native_uint64_t;
349 // unaligned native endian floating point types
350 typedef implementation-defined_float32_t native_float32_t;
351 typedef implementation-defined_float64_t native_float64_t;
353 // aligned big endian signed integer types
354 typedef endian_arithmetic<order::big, int8_t, 8, align::yes> big_int8_at;
355 typedef endian_arithmetic<order::big, int16_t, 16, align::yes> big_int16_at;
356 typedef endian_arithmetic<order::big, int32_t, 32, align::yes> big_int32_at;
357 typedef endian_arithmetic<order::big, int64_t, 64, align::yes> big_int64_at;
359 // aligned big endian unsigned integer types
360 typedef endian_arithmetic<order::big, uint8_t, 8, align::yes> big_uint8_at;
361 typedef endian_arithmetic<order::big, uint16_t, 16, align::yes> big_uint16_at;
362 typedef endian_arithmetic<order::big, uint32_t, 32, align::yes> big_uint32_at;
363 typedef endian_arithmetic<order::big, uint64_t, 64, align::yes> big_uint64_at;
365 // aligned big endian floating point types
366 typedef endian_arithmetic<order::big, float, 32, align::yes> big_float32_at;
367 typedef endian_arithmetic<order::big, double, 64, align::yes> big_float64_at;
369 // aligned little endian signed integer types
370 typedef endian_arithmetic<order::little, int8_t, 8, align::yes> little_int8_at;
371 typedef endian_arithmetic<order::little, int16_t, 16, align::yes> little_int16_at;
372 typedef endian_arithmetic<order::little, int32_t, 32, align::yes> little_int32_at;
373 typedef endian_arithmetic<order::little, int64_t, 64, align::yes> little_int64_at;
375 // aligned little endian unsigned integer types
376 typedef endian_arithmetic<order::little, uint8_t, 8, align::yes> little_uint8_at;
377 typedef endian_arithmetic<order::little, uint16_t, 16, align::yes> little_uint16_at;
378 typedef endian_arithmetic<order::little, uint32_t, 32, align::yes> little_uint32_at;
379 typedef endian_arithmetic<order::little, uint64_t, 64, align::yes> little_uint64_at;
381 // aligned little endian floating point types
382 typedef endian_arithmetic<order::little, float, 32, align::yes> little_float32_at;
383 typedef endian_arithmetic<order::little, double, 64, align::yes> little_float64_at;
385 // aligned native endian typedefs are not provided because
386 // <cstdint> types are superior for that use case
388 } // namespace endian
392 The `implementation-defined` text above is either `big` or `little` according
393 to the endianness of the platform.
395 The only supported value of `CHAR_BIT` is 8.
397 The valid values of `Nbits` are as follows:
399 * When `sizeof(T)` is 1, `Nbits` shall be 8;
400 * When `sizeof(T)` is 2, `Nbits` shall be 16;
401 * When `sizeof(T)` is 4, `Nbits` shall be 24 or 32;
402 * When `sizeof(T)` is 8, `Nbits` shall be 40, 48, 56, or 64.
404 Other values of `sizeof(T)` are not supported.
406 When `Nbits` is equal to `sizeof(T)*8`, `T` must be a standard arithmetic type.
408 When `Nbits` is less than `sizeof(T)*8`, `T` must be a standard integral type
409 ({cpp}std, [basic.fundamental]) that is not `bool`.
414 endian_arithmetic() noexcept = default; // C++03: endian(){}
419 Effects:: Constructs an uninitialized object.
422 endian_arithmetic(T v) noexcept;
427 Effects:: See `endian_buffer::endian_buffer(T)`.
430 endian_arithmetic& operator=(T v) noexcept;
435 Effects:: See `endian_buffer::operator=(T)`.
439 operator T() const noexcept;
449 Other operators on endian objects are forwarded to the equivalent operator on
455 template <class charT, class traits>
456 friend std::basic_ostream<charT, traits>&
457 operator<<(std::basic_ostream<charT, traits>& os, const endian_arithmetic& x);
463 Returns:: `os << +x`.
469 template <class charT, class traits>
470 friend std::basic_istream<charT, traits>&
471 operator>>(std::basic_istream<charT, traits>& is, endian_arithmetic& x);
487 See the <<overview_faq,Overview FAQ>> for a library-wide FAQ.
489 Why not just use Boost.Serialization?::
490 Serialization involves a conversion for every object involved in I/O. Endian
491 integers require no conversion or copying. They are already in the desired
492 format for binary I/O. Thus they can be read or written in bulk.
494 Are endian types PODs?::
495 Yes for {cpp}11. No for {cpp}03, although several
496 <<arithmetic_compilation,macros>> are available to force PODness in all cases.
498 What are the implications of endian integer types not being PODs with {cpp}03 compilers?::
499 They can't be used in unions. Also, compilers aren't required to align or lay
500 out storage in portable ways, although this potential problem hasn't prevented
501 use of Boost.Endian with real compilers.
503 What good is native endianness?::
504 It provides alignment and size guarantees not available from the built-in
505 types. It eases generic programming.
507 Why bother with the aligned endian types?::
508 Aligned integer operations may be faster (as much as 10 to 20 times faster)
509 if the endianness and alignment of the type matches the endianness and
510 alignment requirements of the machine. The code, however, will be somewhat less
511 portable than with the unaligned types.
513 Why provide the arithmetic operations?::
514 Providing a full set of operations reduces program clutter and makes code
515 both easier to write and to read. Consider incrementing a variable in a record.
516 It is very convenient to write:
525 int temp(record.foo);
530 ## Design considerations for Boost.Endian types
532 * Must be suitable for I/O - in other words, must be memcpyable.
533 * Must provide exactly the size and internal byte ordering specified.
534 * Must work correctly when the internal integer representation has more bits
535 that the sum of the bits in the external byte representation. Sign extension
536 must work correctly when the internal integer representation type has more
537 bits than the sum of the bits in the external bytes. For example, using
538 a 64-bit integer internally to represent 40-bit (5 byte) numbers must work for
539 both positive and negative values.
540 * Must work correctly (including using the same defined external
541 representation) regardless of whether a compiler treats char as signed or
543 * Unaligned types must not cause compilers to insert padding bytes.
544 * The implementation should supply optimizations with great care. Experience
545 has shown that optimizations of endian integers often become pessimizations
546 when changing machines or compilers. Pessimizations can also happen when
547 changing compiler switches, compiler versions, or CPU models of the same
552 Classes with similar functionality have been independently developed by
553 several Boost programmers and used very successful in high-value, high-use
554 applications for many years. These independently developed endian libraries
555 often evolved from C libraries that were also widely used. Endian types have
556 proven widely useful across a wide range of computer architectures and
559 ## Motivating use cases
561 Neil Mayhew writes: "I can also provide a meaningful use-case for this
562 library: reading TrueType font files from disk and processing the contents. The
563 data format has fixed endianness (big) and has unaligned values in various
564 places. Using Boost.Endian simplifies and cleans the code wonderfully."
568 The availability of the {cpp}11
569 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted
570 Functions] feature is detected automatically, and will be used if present to
571 ensure that objects of `class endian_arithmetic` are trivial, and thus PODs.
574 Boost.Endian is implemented entirely within headers, with no need to link to any
575 Boost object libraries.
577 Several macros allow user control over features:
579 * BOOST_ENDIAN_NO_CTORS causes `class endian_arithmetic` to have no
580 constructors. The intended use is for compiling user code that must be portable
581 between compilers regardless of {cpp}11
582 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted
583 Functions] support. Use of constructors will always fail,
584 * BOOST_ENDIAN_FORCE_PODNESS causes BOOST_ENDIAN_NO_CTORS to be defined if
585 the compiler does not support {cpp}11
586 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2346.htm[Defaulted
587 Functions]. This is ensures that objects of `class endian_arithmetic` are PODs,
588 and so can be used in {cpp}03 unions. In {cpp}11, `class endian_arithmetic`
589 objects are PODs, even though they have constructors, so can always be used in
594 Original design developed by Darin Adler based on classes developed by Mark
595 Borgerding. Four original class templates combined into a single
596 `endian_arithmetic` class template by Beman Dawes, who put the library together,
597 provided documentation, added the typedefs, and also added the
598 `unrolled_byte_loops` sign partial specialization to correctly extend the sign
599 when cover integer size differs from endian representation size.