platform/upstream/lz4.git
5 years agocorrectly use unisgned int for index
gabrielstedman [Sat, 20 Apr 2019 23:20:38 +0000 (00:20 +0100)]
correctly use unisgned int for index

5 years agoAdd --list option to display compressed file information.
gabrielstedman [Sat, 20 Apr 2019 21:06:10 +0000 (22:06 +0100)]
Add --list option to display compressed file information.

5 years agobumped version number
Yann Collet [Fri, 19 Apr 2019 18:54:01 +0000 (11:54 -0700)]
bumped version number

to v1.9.1

5 years agoMerge pull request #684 from lz4/cxxlz4f
Yann Collet [Fri, 19 Apr 2019 18:51:29 +0000 (11:51 -0700)]
Merge pull request #684 from lz4/cxxlz4f

lz4frame: initializers compatibility with C++

5 years agolz4frame: initializers compatibility with C++
Yann Collet [Fri, 19 Apr 2019 17:23:50 +0000 (10:23 -0700)]
lz4frame: initializers compatibility with C++

fix #679, reported by @degski

5 years agoMerge pull request #683 from lz4/fuzasan
Yann Collet [Fri, 19 Apr 2019 03:57:56 +0000 (20:57 -0700)]
Merge pull request #683 from lz4/fuzasan

fixed read-after input in LZ4_decompress_safe()

5 years agoMerge pull request #682 from lz4/visual
Yann Collet [Fri, 19 Apr 2019 01:51:28 +0000 (18:51 -0700)]
Merge pull request #682 from lz4/visual

fix minor Visual warnings

5 years agofixed read-after input in LZ4_decompress_safe()
Yann Collet [Fri, 19 Apr 2019 01:50:51 +0000 (18:50 -0700)]
fixed read-after input in LZ4_decompress_safe()

5 years agoensure list of names is large enough
Yann Collet [Fri, 19 Apr 2019 00:26:01 +0000 (17:26 -0700)]
ensure list of names is large enough

5 years agofuzzer : reduced stack usage
Yann Collet [Fri, 19 Apr 2019 00:12:14 +0000 (17:12 -0700)]
fuzzer : reduced stack usage

5 years agofuzzer: reduce stack usage to please Visual static analyzer
Yann Collet [Thu, 18 Apr 2019 23:41:27 +0000 (16:41 -0700)]
fuzzer: reduce stack usage to please Visual static analyzer

5 years agoaddress a few minor Visual warnings
Yann Collet [Thu, 18 Apr 2019 23:06:02 +0000 (16:06 -0700)]
address a few minor Visual warnings

and created target cxx17build

5 years agoMerge pull request #678 from lz4/decFast
Yann Collet [Thu, 18 Apr 2019 21:29:20 +0000 (14:29 -0700)]
Merge pull request #678 from lz4/decFast

Fix out-of-bound read in LZ4_decompress_fast()

5 years agofix several minor static analyzer warnings
Yann Collet [Thu, 18 Apr 2019 19:05:50 +0000 (12:05 -0700)]
fix several minor static analyzer warnings

5 years agofuzzer: fixed scan-build leak warning
Yann Collet [Thu, 18 Apr 2019 17:38:51 +0000 (10:38 -0700)]
fuzzer: fixed scan-build leak warning

the program exit(),
so there is no need to track and dealloc every buffer.

5 years agofuzzer: fix explicit cast for C++
Yann Collet [Wed, 17 Apr 2019 22:44:14 +0000 (15:44 -0700)]
fuzzer: fix explicit cast for C++

5 years agore-enable FORCE_INLINE
Yann Collet [Wed, 17 Apr 2019 22:33:37 +0000 (15:33 -0700)]
re-enable FORCE_INLINE

was disabled for tests

5 years agofix out-of-bound read within LZ4_decompress_fast()
Yann Collet [Wed, 17 Apr 2019 22:01:53 +0000 (15:01 -0700)]
fix out-of-bound read within LZ4_decompress_fast()

and deprecate LZ4_decompress_fast(),
with deprecation warnings enabled by default.

Note that, as a consequence of the fix,
LZ4_decompress_fast is now __slower__ than LZ4_decompress_safe().
That's because, since it doesn't know the input buffer size,
it must progress more cautiously into the input buffer
to ensure to out-of-bound read.

5 years agofuzzer: added tests to detect LZ4_decompress_fast() out of bound read
Yann Collet [Wed, 17 Apr 2019 20:31:24 +0000 (13:31 -0700)]
fuzzer: added tests to detect LZ4_decompress_fast() out of bound read

5 years agoMerge pull request #674 from nrgiii/dev
Yann Collet [Wed, 17 Apr 2019 17:43:38 +0000 (10:43 -0700)]
Merge pull request #674 from nrgiii/dev

Fix

5 years agoMerge branch 'fuzzerAix' into dev
Norm Green [Wed, 17 Apr 2019 16:35:18 +0000 (09:35 -0700)]
Merge branch 'fuzzerAix' into dev

5 years agoFix AIX errors/warnings
Norm Green [Wed, 17 Apr 2019 16:20:09 +0000 (09:20 -0700)]
Fix AIX errors/warnings

5 years agofixed version number for lz4hc
Yann Collet [Wed, 17 Apr 2019 03:46:04 +0000 (20:46 -0700)]
fixed version number for lz4hc

5 years agominor benchmark update
Yann Collet [Tue, 16 Apr 2019 18:52:54 +0000 (11:52 -0700)]
minor benchmark update

was using wrong memory timing

5 years agoensure consistent definition and usage of FREEMEM
Yann Collet [Tue, 16 Apr 2019 18:26:03 +0000 (11:26 -0700)]
ensure consistent definition and usage of FREEMEM

as suggested by @sloutsky in #671

5 years agominor news update
Yann Collet [Tue, 16 Apr 2019 18:20:31 +0000 (11:20 -0700)]
minor news update

for msys2

5 years agoupdated NEWS for v1.9.0
Yann Collet [Tue, 16 Apr 2019 00:30:31 +0000 (17:30 -0700)]
updated NEWS for v1.9.0

5 years agoMerge pull request #669 from lz4/fixframe
Yann Collet [Mon, 15 Apr 2019 22:32:03 +0000 (15:32 -0700)]
Merge pull request #669 from lz4/fixframe

Fix lz4frame inefficiency

5 years agosimplified output_directive
Yann Collet [Mon, 15 Apr 2019 21:13:10 +0000 (14:13 -0700)]
simplified output_directive

5 years agofix comma for pedantic
Yann Collet [Mon, 15 Apr 2019 18:22:25 +0000 (11:22 -0700)]
fix comma for pedantic

5 years agoMerge branch 'dev' into fixframe
Yann Collet [Mon, 15 Apr 2019 18:18:33 +0000 (11:18 -0700)]
Merge branch 'dev' into fixframe

5 years agoadded a fake initialization to please an analyzer
Yann Collet [Mon, 15 Apr 2019 18:17:35 +0000 (11:17 -0700)]
added a fake initialization to please an analyzer

which is unable to understand that the variable is necessarily initialized
in spite of an assert just before.

5 years agofix fuzzer tests on dirty context
Yann Collet [Mon, 15 Apr 2019 18:12:54 +0000 (11:12 -0700)]
fix fuzzer tests on dirty context

context is no longer dirty after a failed compressed block.
Actually, all indexes are fine.
It remains compatible with continued streaming, and reset*_fast().

5 years agounified limitedOutput_directive
Yann Collet [Mon, 15 Apr 2019 18:09:56 +0000 (11:09 -0700)]
unified limitedOutput_directive

between lz4.c and lz4hc.c .
was left in a strange state after the "amalgamation" patch.

Now only 3 directives remain,
same name across both implementations,
single definition place.

Might allow some light simplification due to reduced nb of states possible.

5 years agoMerge branch 'dev' into fixframe
Yann Collet [Mon, 15 Apr 2019 17:46:48 +0000 (10:46 -0700)]
Merge branch 'dev' into fixframe

5 years agoadded more margin for canary in frametest
Yann Collet [Mon, 15 Apr 2019 17:44:46 +0000 (10:44 -0700)]
added more margin for canary in frametest

5 years agodecompress*_fast() function do not generate deprecation warnings
Yann Collet [Mon, 15 Apr 2019 17:33:40 +0000 (10:33 -0700)]
decompress*_fast() function do not generate deprecation warnings

they are classified as deprecated in the API documentation (lz4.h)
but do not yet trigger a warning,
to give time to existing applications to move away.

Also, the _fast() variants are still ~5% faster than the _safe() ones
after Dave's patch.

5 years agofixed lz4frame with linked blocks
Yann Collet [Mon, 15 Apr 2019 17:28:36 +0000 (10:28 -0700)]
fixed lz4frame with linked blocks

when one block was not compressible,
it would tag the context as `dirty`,
resulting in compression automatically bailing out of all future blocks,
making the rest of the frame uncompressible.

5 years agoframetest: removed uninitialized warning
Yann Collet [Mon, 15 Apr 2019 17:14:36 +0000 (10:14 -0700)]
frametest: removed uninitialized warning

was a false positive, but better remove it anyway

5 years agoassert() the failed test
Yann Collet [Mon, 15 Apr 2019 09:11:59 +0000 (02:11 -0700)]
assert() the failed test

ensure canary remains within buffer limits

5 years agofixed frametest
Yann Collet [Mon, 15 Apr 2019 08:54:40 +0000 (01:54 -0700)]
fixed frametest

One test could write a canary value out of bound
in exceptional conditions involving multiple flushes
triggered by -s3421 -t462948.

5 years agoMerge branch 'dev' of github.com:Cyan4973/lz4 into dev
Yann Collet [Sat, 13 Apr 2019 22:55:33 +0000 (15:55 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev

5 years agofixed incorrect assertion condition
Yann Collet [Sat, 13 Apr 2019 22:55:06 +0000 (15:55 -0700)]
fixed incorrect assertion condition

output can use the full length of output buffer

5 years agoMerge pull request #668 from lz4/visual
Yann Collet [Sat, 13 Apr 2019 21:15:48 +0000 (14:15 -0700)]
Merge pull request #668 from lz4/visual

Fix some Visual conversion warnings

5 years agoupdated benchmark table
Yann Collet [Sat, 13 Apr 2019 01:00:43 +0000 (18:00 -0700)]
updated benchmark table

5 years agoand even more visual conversion warnings
Yann Collet [Sat, 13 Apr 2019 00:40:23 +0000 (17:40 -0700)]
and even more visual conversion warnings

5 years agoMerge pull request #667 from lz4/pipeMultiple
Yann Collet [Sat, 13 Apr 2019 00:29:44 +0000 (17:29 -0700)]
Merge pull request #667 from lz4/pipeMultiple

Fix combination -m -c

5 years agosome more Visual conversion warning
Yann Collet [Sat, 13 Apr 2019 00:06:53 +0000 (17:06 -0700)]
some more Visual conversion warning

5 years agofixed minor Visual conversion warnings
Yann Collet [Sat, 13 Apr 2019 00:03:28 +0000 (17:03 -0700)]
fixed minor Visual conversion warnings

5 years agofix minor visual warning
Yann Collet [Fri, 12 Apr 2019 23:56:22 +0000 (16:56 -0700)]
fix minor visual warning

yet some overly cautious overflow risk flag,
while it's actually impossible, due to previous test just one line above.
Changing the cast position, just to be please the thing.

5 years agofixed minor Visual warnings
Yann Collet [Fri, 12 Apr 2019 23:49:01 +0000 (16:49 -0700)]
fixed minor Visual warnings

since Visual 2017,
worries about potential overflow, which are actually impossible.
Replaced (c * a) by (c ? a : 0).
Will likely replaced a * by a cmov.
Probably harmless for performance.

5 years agoadded tests for -d -m -c
Yann Collet [Fri, 12 Apr 2019 22:55:38 +0000 (15:55 -0700)]
added tests for -d -m -c

seems to work properly

5 years agofixed lz4 -m -c
Yann Collet [Fri, 12 Apr 2019 22:17:48 +0000 (15:17 -0700)]
fixed lz4 -m -c

can compress multiple files into stdout

5 years agoMerge pull request #666 from lz4/blockIndep
Yann Collet [Fri, 12 Apr 2019 21:28:44 +0000 (14:28 -0700)]
Merge pull request #666 from lz4/blockIndep

added command -BI for "Independent Blocks"

5 years agoadded command -BI for "Independent Blocks"
Yann Collet [Fri, 12 Apr 2019 20:27:11 +0000 (13:27 -0700)]
added command -BI for "Independent Blocks"

This is the reverse of `-BD`, and the current default.

This command can be useful to reverse a previous `-BD` command.
It may in the future be more important
if `lz4` switches to generating dependent blocks by default.

5 years agoupdated man page
Yann Collet [Fri, 12 Apr 2019 20:15:05 +0000 (13:15 -0700)]
updated man page

with --favor-decSpeed

5 years agoupdated doc to underline difference between block and frame
Yann Collet [Fri, 12 Apr 2019 18:27:44 +0000 (11:27 -0700)]
updated doc to underline difference between block and frame

as this is a very frequent source of confusion for new users.

5 years agoMerge pull request #665 from lz4/dict
Yann Collet [Thu, 11 Apr 2019 22:43:40 +0000 (15:43 -0700)]
Merge pull request #665 from lz4/dict

Dictionary compression

5 years agoMerge pull request #664 from lz4/maxdist
Yann Collet [Thu, 11 Apr 2019 22:42:54 +0000 (15:42 -0700)]
Merge pull request #664 from lz4/maxdist

introduce LZ4_DISTANCE_MAX build macro

5 years agoimproved documentation for LZ4 dictionary compression
Yann Collet [Thu, 11 Apr 2019 22:19:48 +0000 (15:19 -0700)]
improved documentation for LZ4 dictionary compression

5 years agointroduce LZ4_DISTANCE_MAX build macro
Yann Collet [Thu, 11 Apr 2019 21:15:33 +0000 (14:15 -0700)]
introduce LZ4_DISTANCE_MAX build macro

make it possible to generate LZ4-compressed block
with a controlled maximum offset (necessarily <= 65535).

This could be useful for compatibility with decoders
using a very limited memory budget (<64 KB).

Answer #154

5 years agoupdated block format with more details on MF_LIMIT
Yann Collet [Thu, 11 Apr 2019 20:55:35 +0000 (13:55 -0700)]
updated block format with more details on MF_LIMIT

aka distance from end of block for the last match.

5 years agoMerge pull request #663 from lz4/headerSize
Yann Collet [Wed, 10 Apr 2019 04:30:14 +0000 (21:30 -0700)]
Merge pull request #663 from lz4/headerSize

made LZ4F_getHeaderSize() public

5 years agoadded versions in comments
Yann Collet [Wed, 10 Apr 2019 01:23:32 +0000 (18:23 -0700)]
added versions in comments

5 years agomade LZ4F_getHeaderSize() public
Yann Collet [Wed, 10 Apr 2019 01:10:02 +0000 (18:10 -0700)]
made LZ4F_getHeaderSize() public

5 years agoMerge pull request #659 from lz4/resetFast
Yann Collet [Tue, 9 Apr 2019 23:26:49 +0000 (16:26 -0700)]
Merge pull request #659 from lz4/resetFast

LZ4_resetStream*_fast()

5 years agofixed loadDictHC
Yann Collet [Tue, 9 Apr 2019 22:37:59 +0000 (15:37 -0700)]
fixed loadDictHC

by making a full initialization
instead of a fast reset.

5 years agore-enable LZ4_resetStreamHC()
Yann Collet [Tue, 9 Apr 2019 21:00:30 +0000 (14:00 -0700)]
re-enable LZ4_resetStreamHC()

towards deprecation, but still available and fully supported

5 years agomodified LZ4_initStreamHC() to look like LZ4_initStream()
Yann Collet [Tue, 9 Apr 2019 20:55:42 +0000 (13:55 -0700)]
modified LZ4_initStreamHC() to look like LZ4_initStream()

it is now a pure initializer, for statically allocated states.
It can initialize any memory area, and because of this, requires size.

5 years agocheck some more initialization result
Yann Collet [Mon, 8 Apr 2019 23:51:22 +0000 (16:51 -0700)]
check some more initialization result

ensure it's not NULL.

5 years agoremoved LZ4_stream_t alignment test on Visual
Yann Collet [Mon, 8 Apr 2019 23:47:21 +0000 (16:47 -0700)]
removed LZ4_stream_t alignment test on Visual

it fails on x86 32-bit mode :
Visual reports an alignment of 8-bytes (even with alignof())
but actually only align LZ4_stream_t on 4 bytes.
The alignment check then fails, resulting in missed initialization.

5 years agocheckFrame: removed %zu
Yann Collet [Mon, 8 Apr 2019 21:21:09 +0000 (14:21 -0700)]
checkFrame: removed %zu

not liked by mingw

5 years agofullbench: assert lz4_stream_t initialization
Yann Collet [Mon, 8 Apr 2019 21:10:59 +0000 (14:10 -0700)]
fullbench: assert lz4_stream_t initialization

5 years agofixed cleaning tmp directory
Yann Collet [Mon, 8 Apr 2019 21:01:32 +0000 (14:01 -0700)]
fixed cleaning tmp directory

5 years agoslightly faster huge file test
Yann Collet [Mon, 8 Apr 2019 20:49:26 +0000 (13:49 -0700)]
slightly faster huge file test

level down 5->4
size down 6G->5G

5 years agoadded make list target to tests
Yann Collet [Mon, 8 Apr 2019 20:45:01 +0000 (13:45 -0700)]
added make list target to tests

5 years agoLZ4_initStream() checks alignment restriction
Yann Collet [Mon, 8 Apr 2019 19:49:54 +0000 (12:49 -0700)]
LZ4_initStream() checks alignment restriction

updated associated documentation

5 years agoadded comment on initStream + _extState_
Yann Collet [Fri, 5 Apr 2019 22:35:19 +0000 (15:35 -0700)]
added comment on initStream + _extState_

as suggested by @felixhandte

5 years agocreated LZ4_initStream()
Yann Collet [Fri, 5 Apr 2019 19:54:13 +0000 (12:54 -0700)]
created LZ4_initStream()

- promoted LZ4_resetStream_fast() to stable
- moved LZ4_resetStream() into deprecate, but without triggering a compiler warning
- update all sources to no longer rely on LZ4_resetStream()

note : LZ4_initStream() proposal is slightly different :
it's able to initialize any buffer, provided that it's large enough.
To this end, it accepts a void*, and returns an LZ4_stream_t*.

5 years agofixed _GNU_SOURCE duplicate
Yann Collet [Fri, 5 Apr 2019 18:55:34 +0000 (11:55 -0700)]
fixed _GNU_SOURCE duplicate

5 years agofuzzer: fixed strict c99 with mmap+MAP_ANONYMOUS
Yann Collet [Fri, 5 Apr 2019 18:47:06 +0000 (11:47 -0700)]
fuzzer: fixed strict c99 with mmap+MAP_ANONYMOUS

5 years agofixed strict iso C90
Yann Collet [Fri, 5 Apr 2019 17:41:16 +0000 (10:41 -0700)]
fixed strict iso C90

5 years agocreated LZ4_initStreamHC()
Yann Collet [Fri, 5 Apr 2019 00:03:05 +0000 (17:03 -0700)]
created LZ4_initStreamHC()

- promoted LZ4_resetStreamHC_fast() to stable
- moved LZ4_resetStreamHC() to deprecated (but do not generate a warning yet)
- Updated doc, to highlight difference between init and reset
- switched all invocations of LZ4_resetStreamHC() onto LZ4_initStreamHC()
- misc: ensure `make all` also builds /tests

5 years agoMerge pull request #658 from lz4/_fast
Yann Collet [Thu, 4 Apr 2019 20:42:58 +0000 (13:42 -0700)]
Merge pull request #658 from lz4/_fast

Deprecated LZ4_decompres_fast*() functions

5 years agomake `_fast*()` decoder generate a deprecation warning
Yann Collet [Thu, 4 Apr 2019 19:47:36 +0000 (12:47 -0700)]
make `_fast*()` decoder generate a deprecation warning

updated modification

5 years agomoved LZ4_decompress_fast*() into deprecated section
Yann Collet [Thu, 4 Apr 2019 19:24:46 +0000 (12:24 -0700)]
moved LZ4_decompress_fast*() into deprecated section

5 years agoMerge pull request #657 from lz4/destSize
Yann Collet [Thu, 4 Apr 2019 00:15:06 +0000 (17:15 -0700)]
Merge pull request #657 from lz4/destSize

moved _destSize() into "stable API" status

5 years agofixed an old bug in LZ4F_flush()
Yann Collet [Wed, 3 Apr 2019 23:28:42 +0000 (16:28 -0700)]
fixed an old bug in LZ4F_flush()

which remained undetected so far,
as it requires a fairly large number of conditions to be triggered,
starting with enabling Block checksum, which is disabled by default,
and which usage is known to be extremely rare.

5 years agofixed doc
Yann Collet [Wed, 3 Apr 2019 21:27:21 +0000 (14:27 -0700)]
fixed doc

and bumped version number fo v1.9.0

5 years agomoved _destSize() into "stable API" status
Yann Collet [Wed, 3 Apr 2019 21:18:00 +0000 (14:18 -0700)]
moved _destSize() into "stable API" status

as requested in #642

5 years agoMerge pull request #656 from lz4/armtest
Yann Collet [Wed, 3 Apr 2019 17:03:26 +0000 (10:03 -0700)]
Merge pull request #656 from lz4/armtest

LZ4_FAST_DEC_LOOP macros

5 years agominor comments and reformatting
Yann Collet [Wed, 3 Apr 2019 15:59:29 +0000 (08:59 -0700)]
minor comments and reformatting

5 years agofixed minor conversion warnings
Yann Collet [Wed, 3 Apr 2019 00:16:43 +0000 (17:16 -0700)]
fixed minor conversion warnings

5 years agocreated LZ4_FAST_DEC_LOOP build macro
Yann Collet [Tue, 2 Apr 2019 23:22:11 +0000 (16:22 -0700)]
created LZ4_FAST_DEC_LOOP build macro

5 years agofixed a few minor conversion warnings
Yann Collet [Tue, 2 Apr 2019 23:06:37 +0000 (16:06 -0700)]
fixed a few minor conversion warnings

5 years agoMerge pull request #652 from vtorri/dev
Yann Collet [Sun, 3 Mar 2019 21:40:45 +0000 (13:40 -0800)]
Merge pull request #652 from vtorri/dev

Allow installation of lz4 for Windows 10 with MSYS2

5 years agoAllow installation of lz4 for Windows 10 with MSYS2
Vincent Torri [Sun, 3 Mar 2019 21:06:38 +0000 (22:06 +0100)]
Allow installation of lz4 for Windows 10 with MSYS2

5 years agoMerge pull request #645 from djwatson/optimize_decompress_generic
Yann Collet [Tue, 12 Feb 2019 00:58:53 +0000 (16:58 -0800)]
Merge pull request #645 from djwatson/optimize_decompress_generic

Optimize decompress generic

5 years agodecompress_generic: Limit fastpath to x86
Dave Watson [Fri, 8 Feb 2019 21:57:43 +0000 (13:57 -0800)]
decompress_generic: Limit fastpath to x86

New fastpath currently shows a regression on qualcomm
arm chips.  Restrict it to x86 for now

5 years agodecompress_generic: Add fastpath for small offsets
Dave Watson [Mon, 28 Jan 2019 17:36:25 +0000 (09:36 -0800)]
decompress_generic: Add fastpath for small offsets

For small offsets of size 1, 2, 4 and 8, we can set a single uint64_t,
and then use it to do a memset() variation.  In particular, this makes
the somewhat-common RLE (offset 1) about 2-4x faster than the previous
implementation - we avoid not only the load blocked by store, but also
avoid the loads entirely.