Yann Collet [Mon, 4 Dec 2017 22:38:00 +0000 (14:38 -0800)]
Merge pull request #426 from aliceatlas/cli_chmod_fix
Fix cli sometimes calling UTIL_setFileStat on /dev/null
Alice Atlas [Mon, 4 Dec 2017 21:07:31 +0000 (16:07 -0500)]
Fix bug which could sometimes result in the lz4 cli chmodding /dev/null if running as root
Yann Collet [Sat, 25 Nov 2017 01:18:46 +0000 (17:18 -0800)]
minor updates to examples
see https://github.com/lz4/lz4/commit/
810e2ca27b3561e0f6bfa7a88e0fde6faf807064#commitcomment-
25810887
Yann Collet [Mon, 20 Nov 2017 18:27:05 +0000 (10:27 -0800)]
API : changed a few variables' names for clarity
updated relevant doc.
This patch has no impact on ABI/API, nor on binary generation.
Yann Collet [Mon, 13 Nov 2017 23:54:55 +0000 (15:54 -0800)]
updated NEWS
Yann Collet [Thu, 9 Nov 2017 22:13:30 +0000 (14:13 -0800)]
Merge pull request #416 from lz4/newopt
Improve Optimal parser
Yann Collet [Thu, 9 Nov 2017 01:56:20 +0000 (17:56 -0800)]
added code comments
Yann Collet [Thu, 9 Nov 2017 01:47:24 +0000 (17:47 -0800)]
added constant TRAILING_LITERALS
which is more explicit than its value `3`.
reported by @terrelln
Yann Collet [Thu, 9 Nov 2017 01:11:51 +0000 (17:11 -0800)]
lz4opt: simplified match finder invocation to LZ4HC_FindLongerMatch()
Yann Collet [Wed, 8 Nov 2017 16:42:59 +0000 (08:42 -0800)]
added code comments
Yann Collet [Wed, 8 Nov 2017 01:58:59 +0000 (17:58 -0800)]
fixed LZ4HC_reverseCountPattern()
for multi-bytes patterns
(which is not useful for the time being)
Yann Collet [Wed, 8 Nov 2017 01:37:31 +0000 (17:37 -0800)]
removed the ip++ at the beginning of block
The first byte used to be skipped
to avoid a infinite self-comparison.
This is no longer necessary, since init() ensures that index starts at 64K.
The first byte is also useless to search when each block is independent,
but it's no longer the case when blocks are linked.
Removing the first-byte-skip saves
about 10 bytes / MB on files compressed with -BD4 (linked blocks 64Kb),
which feels correct as each MB has 16 blocks of 64KB.
Yann Collet [Tue, 7 Nov 2017 20:09:32 +0000 (12:09 -0800)]
Merge pull request #418 from lz4/fasterCount
faster LZ4_count()
Yann Collet [Tue, 7 Nov 2017 19:33:40 +0000 (11:33 -0800)]
removed legacy version of LZ4HC_InsertAndFindBestMatch()
Yann Collet [Tue, 7 Nov 2017 19:29:28 +0000 (11:29 -0800)]
ensure `pattern` is a 1-byte repetition
Yann Collet [Tue, 7 Nov 2017 19:22:57 +0000 (11:22 -0800)]
removed useless `(1 && ...)` condition
as reported by @terrelln
Yann Collet [Tue, 7 Nov 2017 19:05:48 +0000 (11:05 -0800)]
improved LZ4HC_reverseCountPattern() :
works for any repetitive pattern of length 1, 2 or 4 (but not 3!)
works for any endianess
Yann Collet [Tue, 7 Nov 2017 18:53:29 +0000 (10:53 -0800)]
fixed LZ4HC_countPattern()
- works with byte values other than `0`
- works for any repetitive pattern of length 1, 2 or 4 (but not 3!)
- works for little and big endian systems
- preserve speed of previous implementation
Yann Collet [Tue, 7 Nov 2017 01:29:27 +0000 (17:29 -0800)]
added LZ4_FORCEINLINE to counter gcc regression
as recommended by @terrelln
Yann Collet [Mon, 6 Nov 2017 23:42:50 +0000 (15:42 -0800)]
2-stages LZ4_count
separate first branch from the rest of the compare loop
to get dedicated prediction.
measured a 3-4% compression speed improvement.
Yann Collet [Mon, 6 Nov 2017 23:20:08 +0000 (15:20 -0800)]
build: minor : `make lz4` doesn't compile liblz4 anymore
since it's not needed.
Yann Collet [Mon, 6 Nov 2017 21:56:44 +0000 (13:56 -0800)]
Merge pull request #417 from sylvestre/dev
When building with a C++ compiler, remove the 'register' keyword to silent a warning
Sylvestre Ledru [Mon, 6 Nov 2017 15:16:02 +0000 (16:16 +0100)]
Only ignore with C++17
Sylvestre Ledru [Sun, 5 Nov 2017 10:48:00 +0000 (11:48 +0100)]
When building with a C++ compiler, remove the 'register' keyword to silent a warning
For example, with clang:
lz4.c:XXX:36: error: 'register' storage class specifier is deprecated and incompatible with C++17 [-Werror,-Wdeprecated-register]
static unsigned LZ4_NbCommonBytes (register reg_t val)
^~~~~~~~~
Yann Collet [Fri, 3 Nov 2017 19:33:55 +0000 (12:33 -0700)]
fixed minor static analyzer warning
dead assignment
Yann Collet [Fri, 3 Nov 2017 18:49:56 +0000 (11:49 -0700)]
minor comment edit
Yann Collet [Fri, 3 Nov 2017 18:28:28 +0000 (11:28 -0700)]
unified HC levels
LZ4_setCompressionLevel() can be users accross the whole range of HC levels
No more transition issue between Optimal and HC modes
Yann Collet [Fri, 3 Nov 2017 17:48:55 +0000 (10:48 -0700)]
moved ctx->end handling from parsers
responsibility better handled one layer above (LZ4HC_compress_generic())
Yann Collet [Fri, 3 Nov 2017 17:30:52 +0000 (10:30 -0700)]
removed ctx->searchNum
nbSearches now transmitted directly as function parameter
easier to track and debug
Yann Collet [Fri, 3 Nov 2017 09:01:20 +0000 (02:01 -0700)]
LZ4_compress_HC_continue_destSize() now compatible with optimal parser
levels 11+
Yann Collet [Fri, 3 Nov 2017 08:37:43 +0000 (01:37 -0700)]
removes matches[] table
saves stack space
clearer match finder interface (no more table to fill)
Yann Collet [Fri, 3 Nov 2017 08:20:30 +0000 (01:20 -0700)]
removed useless parameter from hash chain matchfinder
used to be present for compatibility with binary tree matchfinder
Yann Collet [Fri, 3 Nov 2017 08:18:12 +0000 (01:18 -0700)]
removed code and reference to binary tree match finder
reduced size of LZ4HC state
Yann Collet [Fri, 3 Nov 2017 07:59:05 +0000 (00:59 -0700)]
improved level 11 speed
Yann Collet [Fri, 3 Nov 2017 07:15:52 +0000 (00:15 -0700)]
optimized skip strategy for level 12
Yann Collet [Fri, 3 Nov 2017 02:50:08 +0000 (19:50 -0700)]
new level 11 uses 512 attempts
Yann Collet [Fri, 3 Nov 2017 01:54:18 +0000 (18:54 -0700)]
more generic skip formula
improving speed
Yann Collet [Thu, 2 Nov 2017 23:25:10 +0000 (16:25 -0700)]
small adaptations for intermediate level 11
Yann Collet [Thu, 2 Nov 2017 22:37:18 +0000 (15:37 -0700)]
partial search, while preserving compression ratio
tag interesting places
Yann Collet [Thu, 2 Nov 2017 22:05:45 +0000 (15:05 -0700)]
searching match leading strictly farther does not work
sometimes, it's better to re-use same match but start it later,
in order to get shorter matchlength code
Yann Collet [Thu, 2 Nov 2017 21:53:06 +0000 (14:53 -0700)]
fixed last lost bytes in maximal mode
even gained 2 bytes on calgary.tar...
added conditional traces `g_debuglog_enable`
Yann Collet [Thu, 2 Nov 2017 20:44:57 +0000 (13:44 -0700)]
changed strategy : opt[] path is complete after each match
previous strategy would leave a few "bad choices"
on the ground they would be fixed later,
but that requires passing through each position to make the fix
and cannot give the end position of the last useful match.
Yann Collet [Wed, 1 Nov 2017 16:58:49 +0000 (09:58 -0700)]
Merge pull request #415 from lz4/fasterDecodingXp
Faster decoding xp
Yann Collet [Wed, 1 Nov 2017 00:55:01 +0000 (17:55 -0700)]
Merge branch 'dev' into btopt
Yann Collet [Tue, 31 Oct 2017 22:51:56 +0000 (15:51 -0700)]
minor change, to help store forwarding
in a marginal case (offset==4)
Yann Collet [Tue, 31 Oct 2017 21:20:25 +0000 (14:20 -0700)]
extended shortcut match length to 18
Yann Collet [Tue, 31 Oct 2017 20:49:29 +0000 (13:49 -0700)]
Merge pull request #408 from terrelln/time
[bench] Use higher resolution timer on POSIX
Yann Collet [Tue, 31 Oct 2017 19:22:15 +0000 (12:22 -0700)]
minor : coding style : use ML_MASK constant
Yann Collet [Tue, 31 Oct 2017 18:49:57 +0000 (11:49 -0700)]
added comments, as suggested by @terrelln
Yann Collet [Tue, 31 Oct 2017 00:47:54 +0000 (17:47 -0700)]
fixed minor overflow mistake in optimal parser
saving 20 bytes on calgary.tar
Yann Collet [Mon, 30 Oct 2017 23:10:25 +0000 (16:10 -0700)]
fixed minor initialization warning
Yann Collet [Mon, 30 Oct 2017 23:07:15 +0000 (16:07 -0700)]
more complete shortcut - passes tests
Yann Collet [Mon, 30 Oct 2017 22:57:43 +0000 (15:57 -0700)]
Merge pull request #412 from mikir/symbolsVisibility
Separated visibility from LZ4LIB_API macro.
mikir [Mon, 30 Oct 2017 12:44:24 +0000 (13:44 +0100)]
Separated visibility from LZ4LIB_API macro.
Yann Collet [Wed, 25 Oct 2017 08:10:53 +0000 (10:10 +0200)]
small modification of lz4 decoder to shortcut common case (short branch).
Yann Collet [Wed, 25 Oct 2017 05:07:08 +0000 (07:07 +0200)]
added hash chain with conditional length
not a success yet
Yann Collet [Sat, 21 Oct 2017 00:04:29 +0000 (17:04 -0700)]
lz4opt: added hash chain search
Yann Collet [Fri, 20 Oct 2017 22:30:50 +0000 (15:30 -0700)]
switched many types to int
Yann Collet [Fri, 20 Oct 2017 20:44:49 +0000 (13:44 -0700)]
removed SET_PRICE macro
Yann Collet [Fri, 20 Oct 2017 20:32:45 +0000 (13:32 -0700)]
removed one macro usage
Yann Collet [Fri, 20 Oct 2017 19:05:00 +0000 (12:05 -0700)]
minor refactor
reduce variable scope
remove one macro usage
Yann Collet [Fri, 20 Oct 2017 18:32:15 +0000 (11:32 -0700)]
lz4opt: refactor sequence reverse traversal
Yann Collet [Fri, 20 Oct 2017 18:24:56 +0000 (11:24 -0700)]
refactor variable matchnum
separate initial and iterative search
renamed nb_matches
Yann Collet [Fri, 20 Oct 2017 18:00:10 +0000 (11:00 -0700)]
simplified initial cost conditions
llen integrated in opt[]
Yann Collet [Thu, 19 Oct 2017 23:47:25 +0000 (16:47 -0700)]
added assert
Yann Collet [Thu, 19 Oct 2017 23:43:36 +0000 (16:43 -0700)]
renamed last_pos into last_match_pos
Yann Collet [Thu, 19 Oct 2017 23:39:40 +0000 (16:39 -0700)]
simplified early exit when single solution
Nick Terrell [Tue, 17 Oct 2017 19:42:27 +0000 (12:42 -0700)]
[bench] Use higher resolution timer on POSIX
The timer used was only accurate up to 0.01 seconds. This timer is accurate up to 1 ns.
It is a monotonic timer that measures the real time difference, not on CPU time.
Copied the benchmark code from https://github.com/facebook/zstd/commit/
6ab4d5e9041aba962a810ffee191f95897c6208e
Yann Collet [Sun, 15 Oct 2017 06:50:07 +0000 (23:50 -0700)]
lz4cli : minor rewrite of lz4c legacy commands
for clarity
Yann Collet [Sun, 15 Oct 2017 01:49:06 +0000 (18:49 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
Yann Collet [Sun, 15 Oct 2017 01:48:00 +0000 (18:48 -0700)]
lz4cli : removed extension artefacts
It used to be useful for an old Windows variant which is no longer maintained.
Yann Collet [Fri, 13 Oct 2017 23:21:46 +0000 (16:21 -0700)]
Merge pull request #407 from odaira/useO2ppc64le
Use O2 for the decompression functions on ppc64le with gcc
Rei Odaira [Fri, 13 Oct 2017 19:53:37 +0000 (14:53 -0500)]
Use the optimization level of O2 for the decompression functions on ppc64le with gcc, to avoid harmful unrolling and SIMDization with O3
Yann Collet [Tue, 10 Oct 2017 23:18:16 +0000 (16:18 -0700)]
Merge pull request #403 from felixhandte/lz4-cli-dict-support-tests
Support Dictionaries on the Command Line
W. Felix Handte [Tue, 3 Oct 2017 16:50:28 +0000 (12:50 -0400)]
Read the Dictionary into a Circular Buffer
W. Felix Handte [Fri, 22 Sep 2017 21:50:11 +0000 (14:50 -0700)]
Add some tests verifying command line dictionary functionality
W. Felix Handte [Fri, 22 Sep 2017 18:55:42 +0000 (11:55 -0700)]
Add Dictionary Support to the Command Line Tool
Yann Collet [Mon, 9 Oct 2017 08:50:28 +0000 (01:50 -0700)]
fused getLongerMatch and getWiderMatch
Yann Collet [Mon, 9 Oct 2017 08:44:05 +0000 (01:44 -0700)]
re-inserted last byte test in widerMatch
Yann Collet [Mon, 9 Oct 2017 07:36:47 +0000 (00:36 -0700)]
early out is not better
Yann Collet [Mon, 9 Oct 2017 07:31:12 +0000 (00:31 -0700)]
optional fuse
Yann Collet [Mon, 9 Oct 2017 06:55:42 +0000 (23:55 -0700)]
improved search of rep-1 patterns
Yann Collet [Mon, 9 Oct 2017 06:40:21 +0000 (23:40 -0700)]
insertAndFindBestMatch defers to insertAndGetWiderMatch
Yann Collet [Wed, 4 Oct 2017 22:24:08 +0000 (15:24 -0700)]
fixed decoding block checksum in lz4frame
Yann Collet [Sat, 30 Sep 2017 17:35:55 +0000 (10:35 -0700)]
fix #404
static analyzer `cppcheck` complains about a shift-by-32 on a 32 bits value,
which is an undefined behavior.
However, the flagged code path is never triggered in 32-bits mode,
(actually, it's not even generated if DCE kicks in),
the shift-by-32 is necessarily performed on a 64-bits value.
While it doesn't change anything regarding lz4 code generation, for both 32 and 64 bits mode,
(can be checked by md5sum on the generated binary),
the shift has been rewritten in a way which should please this static analyzer,
since it now pretends to shift by 16 on 32-bits cpu (note : it doesn't matter since the code will not even be generated in this case).
Note : this is a blind fix, the new code has not been tested with cppcheck, because cppcheck only works on Windows.
Other static analyzer, such as scan-build, do not trigger this false positive.
Yann Collet [Sat, 23 Sep 2017 22:06:24 +0000 (15:06 -0700)]
minor lz4frame code refactor
try to improve code readability.
minor optimization on condition to preserve history.
Yann Collet [Sat, 23 Sep 2017 21:02:22 +0000 (14:02 -0700)]
Merge pull request #402 from felixhandte/fix-dict-segfault
Fix dict segfault
W. Felix Handte [Fri, 22 Sep 2017 18:52:29 +0000 (11:52 -0700)]
Fix Segfault When Copying Dict
dctx must have been initialized before we can copy the dictionary in.
Yann Collet [Mon, 11 Sep 2017 17:25:47 +0000 (10:25 -0700)]
minor improvements to examples
cosmetic : better display
added optional variable MOREFLAGS
Yann Collet [Sun, 10 Sep 2017 21:35:16 +0000 (14:35 -0700)]
made clang warnings fail (-Werror)
in order to catch them in CI tests
Yann Collet [Sun, 10 Sep 2017 21:32:38 +0000 (14:32 -0700)]
fixed a bunch of -Wcomma warnings
reported by @rvandermeulen (#398)
Yann Collet [Sun, 10 Sep 2017 21:23:18 +0000 (14:23 -0700)]
added -Wcomma to travisCI clang test
Yann Collet [Thu, 7 Sep 2017 19:48:24 +0000 (12:48 -0700)]
fix #397 : decompression failed when using a combination of extDict + low memory address
Reported and fixed by @jscheid
Note : we are missing a test case to include it in the CI
Yann Collet [Thu, 7 Sep 2017 19:12:36 +0000 (12:12 -0700)]
bench : made decompression speed evaluation same time as compression
minor : slightly modified an example do avoid disabling a gcc warning through #pragma
Yann Collet [Thu, 7 Sep 2017 00:41:44 +0000 (17:41 -0700)]
minor Makefile fixes
Yann Collet [Wed, 6 Sep 2017 18:22:45 +0000 (11:22 -0700)]
updated lib/README
clarifications, improved wording
Yann Collet [Wed, 30 Aug 2017 23:09:10 +0000 (16:09 -0700)]
complementary information for #394
Yann Collet [Wed, 30 Aug 2017 22:42:04 +0000 (15:42 -0700)]
clarified documentation of streaming decompression functions
(synchronous bufferless mode)
answering questions by @jtbandes (#394)
Yann Collet [Wed, 30 Aug 2017 22:02:04 +0000 (15:02 -0700)]
minor typo fix
Yann Collet [Tue, 29 Aug 2017 22:31:56 +0000 (15:31 -0700)]
fixed FS-independent file order in /lib
identified by @bmwiedemann