1 **********************************************************************
2 libaec - Adaptive Entropy Coding library
3 **********************************************************************
5 Libaec provides fast lossless compression of 1 up to 32 bit wide
6 signed or unsigned integers (samples). The library achieves best
7 results for low entropy data as often encountered in space imaging
8 instrument data or numerical model output from weather or climate
9 simulations. While floating point representations are not directly
10 supported, they can also be efficiently coded by grouping exponents
13 Libaec implements Golomb Rice coding as defined in the Space Data
14 System Standard documents 121.0-B-2 [1] and 120.0-G-2[2].
17 **********************************************************************
19 **********************************************************************
21 In doc/license.txt a clarification on potentially applying
22 intellectual property rights is given.
25 **********************************************************************
27 **********************************************************************
29 See INSTALL for details.
32 **********************************************************************
34 **********************************************************************
36 In this context efficiency refers to the size of the encoded
37 data. Performance refers to the time it takes to encode data.
39 Suppose you have an array of 32 bit signed integers you want to
40 compress. The pointer pointing to the data shall be called *source,
41 output goes into *dest.
46 struct aec_stream strm;
50 /* input data is 32 bits wide */
51 strm.bits_per_sample = 32;
53 /* define a block size of 16 */
56 /* the reference sample interval is set to 128 blocks */
59 /* input data is signed and needs to be preprocessed */
60 strm.flags = AEC_DATA_SIGNED | AEC_DATA_PREPROCESS;
62 /* pointer to input */
63 strm.next_in = (unsigned char *)source;
65 /* length of input in bytes */
66 strm.avail_in = source_length * sizeof(int32_t);
68 /* pointer to output buffer */
71 /* length of output buffer in bytes */
72 strm.avail_out = dest_length;
74 /* initialize encoding */
75 if (aec_encode_init(&strm) != AEC_OK)
78 /* Perform encoding in one call and flush output. */
79 /* In this example you must be sure that the output */
80 /* buffer is large enough for all compressed output */
81 if (aec_encode(&strm, AEC_FLUSH) != AEC_OK)
84 /* free all resources used by encoder */
85 aec_encode_end(&strm);
88 block_size can vary from 8 to 64 samples. Smaller blocks allow the
89 compression to adapt to rapid changes in entropy. Larger blocks create
90 less overhead but can be less efficient if entropy changes across the
93 rsi sets the reference sample interval. A large RSI will improve
94 performance and efficiency. It will also increase memory requirements
95 since internal buffering is based on RSI size. A smaller RSI may be
96 desirable in situations where each RSI will be packetized and possible
97 error propagation has to be minimized (e.g. on board a spacecraft[2]).
101 AEC_DATA_SIGNED: input data are signed integers. Specifying this
102 correctly increases compression efficiency. Default is unsigned.
104 AEC_DATA_PREPROCESS: preprocessing input will improve compression
105 efficiency if data samples are correlated. It will only cost
106 performance for no gain in efficiency if the data is already
109 AEC_DATA_MSB: input data is stored most significant byte first
110 i.e. big endian. You have to specify AEC_DATA_MSB even if your host
111 architecture is big endian. Default is little endian on all
114 AEC_DATA_3BYTE: the 24 bit input data is stored in three bytes.
116 AEC_DATA_RESTRICT: use a restricted set of code options.
120 Except for the AEC_DATA_3BYTE case for 24 bit data, the following
121 rules apply for deducing storage size from sample size
124 sample size storage size
127 17 - 32 bits 4 bytes (also for 24bit if AEC_DATA_3BYTE is not set)
129 If a sample requires less bits than the storage size provides, then
130 you have to make sure that unused bits are not set. Libaec does not
131 check this for performance reasons and will produce undefined output
132 if unused bits are set. All input data must be a multiple of the
133 storage size in bytes. Remaining bytes which do not form a complete
134 sample will be ignored.
136 Libaec accesses next_in and next_out buffers only bytewise. There are
137 no alignment requirements for these buffers.
141 aec_encode can be used in a streaming fashion by chunking input and
142 output and specifying AEC_NO_FLUSH. The function will return if either
143 the input runs empty or the output buffer is full. The calling
144 function can check avail_in and avail_out to see what occcurred. The
145 last call to aec_encode() must set AEC_FLUSH to drain all
146 output. aec.c is an example of streaming usage of encoding and
151 Encoded data will be written to the buffer submitted with
152 next_out. The length of the compressed data is total_out.
154 See libaec.h for a detailed description of all relevant structure
155 members and constants.
158 **********************************************************************
160 **********************************************************************
162 Using decoding is very similar to encoding, only the meaning of input
163 and output is reversed.
168 struct aec_stream strm;
169 /* this is now the compressed data */
170 unsigned char *source;
171 /* here goes the uncompressed result */
174 strm.bits_per_sample = 32;
175 strm.block_size = 16;
177 strm.flags = AEC_DATA_SIGNED | AEC_DATA_PREPROCESS;
178 strm.next_in = source;
179 strm.avail_in = source_length;
180 strm.next_out = (unsigned char *)dest;
181 strm.avail_out = dest_lenth * sizeof(int32_t);
182 if (aec_decode_init(&strm) != AEC_OK)
184 if (aec_decode(&strm, AEC_FLUSH) != AEC_OK)
186 aec_decode_end(&strm);
189 It is essential for decoding that parameters like bits_per_sample,
190 block_size, rsi, and flags are exactly the same as they were for
191 encoding. Libaec does not store these parameters in the coded stream
192 so it is up to the calling program to keep the correct parameters
193 between encoding and decoding.
195 The actual values of coding parameters are in fact only relevant for
196 efficiency and performance. Data integrity only depends on consistency
200 **********************************************************************
202 **********************************************************************
204 [1] Consultative Committee for Space Data Systems. Lossless Data
205 Compression. Recommendation for Space Data System Standards, CCSDS
206 121.0-B-2. Blue Book. Issue 2. Washington, D.C.: CCSDS, May 2012.
207 http://public.ccsds.org/publications/archive/121x0b2.pdf
209 [2] Consultative Committee for Space Data Systems. Lossless Data
210 Compression. Recommendation for Space Data System Standards, CCSDS
211 120.0-G-2. Green Book. Issue 2. Washington, D.C.: CCSDS, December
213 http://public.ccsds.org/publications/archive/120x0g2s.pdf