Jennifer Liu [Wed, 27 Jun 2018 20:36:38 +0000 (13:36 -0700)]
Fixed bugs about incorrect acceleration calculation and benchmarking negative compresion level
Jennifer Liu [Wed, 27 Jun 2018 00:53:12 +0000 (17:53 -0700)]
Fixed invalid argument test and reformatted else
Jennifer Liu [Wed, 27 Jun 2018 00:31:15 +0000 (17:31 -0700)]
Fixed code based on comments from pull request
Jennifer Liu [Tue, 26 Jun 2018 21:48:58 +0000 (14:48 -0700)]
Removed duplicated circile.yml file
Jennifer Liu [Tue, 26 Jun 2018 18:24:21 +0000 (11:24 -0700)]
Fixed lz4 not found error part 2
Jennifer Liu [Tue, 26 Jun 2018 18:15:55 +0000 (11:15 -0700)]
Fixed lz4 not found error
Jennifer Liu [Tue, 26 Jun 2018 17:49:39 +0000 (10:49 -0700)]
delete commented out code in bench.c
Jennifer Liu [Tue, 26 Jun 2018 17:33:15 +0000 (10:33 -0700)]
fix yml file
Jennifer Liu [Tue, 26 Jun 2018 01:51:47 +0000 (18:51 -0700)]
Fixed bug about file to be compressed is not present
Jennifer Liu [Tue, 26 Jun 2018 00:46:39 +0000 (17:46 -0700)]
Added --fast command to cli
Jennifer Liu [Wed, 20 Jun 2018 20:37:49 +0000 (13:37 -0700)]
set up sample .ysml
Yann Collet [Fri, 1 Jun 2018 21:40:22 +0000 (14:40 -0700)]
Merge pull request #543 from nrgiii/aix_32bit_fuzzer
Aix 32bit fuzzer
Norm Green [Fri, 1 Jun 2018 19:00:41 +0000 (12:00 -0700)]
Merge branch 'master' of https://github.com/nrgiii/lz4 into aix_32bit_fuzzer
Norm Green [Fri, 1 Jun 2018 18:52:22 +0000 (11:52 -0700)]
Fix AIX 32 bit build problem of fuzzer.c
Yann Collet [Tue, 29 May 2018 21:20:48 +0000 (14:20 -0700)]
Merge pull request #542 from wbx-github/dev
allow to override uname when cross-compiling
Yann Collet [Tue, 22 May 2018 21:20:45 +0000 (14:20 -0700)]
Merge pull request #541 from felixhandte/hc-extern-c
Add `extern "C"` Guard Around Experimental HC Declarations
Waldemar Brodkorb [Tue, 22 May 2018 18:38:20 +0000 (20:38 +0200)]
allow to override uname when cross-compiling
When cross-compiling for example from Darwin to Linux it might be
useful to override uname output to force Linux and create Linux
libraries instead of Darwin libraries.
W. Felix Handte [Tue, 22 May 2018 15:28:39 +0000 (11:28 -0400)]
Also Fix Appveyor Cast Warning
W. Felix Handte [Tue, 22 May 2018 01:43:54 +0000 (21:43 -0400)]
Add `extern "C"` Guards Around Experimental HC Declarations
W. Felix Handte [Tue, 22 May 2018 01:55:04 +0000 (21:55 -0400)]
Remove #define-rename of `LZ4_decompress_safe_forceExtDict`
W. Felix Handte [Tue, 22 May 2018 01:36:49 +0000 (21:36 -0400)]
Test Linking C-Compiled Library and C++-Compiled Tests
Yann Collet [Thu, 17 May 2018 17:48:32 +0000 (10:48 -0700)]
Merge pull request #540 from fbrosson/Haiku
Add Haiku as a validated target.
fbrosson [Thu, 17 May 2018 16:52:53 +0000 (16:52 +0000)]
Add Haiku as a validated target.
lz4 1.8.2 works fine on Haiku and passes all tests.
Yann Collet [Mon, 7 May 2018 21:38:45 +0000 (14:38 -0700)]
Merge pull request #531 from lz4/dev
Preparing v1.8.2
Yann Collet [Mon, 7 May 2018 20:06:43 +0000 (13:06 -0700)]
Merge pull request #537 from lz4/xpHCmf2
Speed optimization for optimal parser
Yann Collet [Mon, 7 May 2018 19:14:26 +0000 (12:14 -0700)]
renamed variable for clarity
Yann Collet [Mon, 7 May 2018 18:33:53 +0000 (11:33 -0700)]
Merge pull request #538 from lz4/frameTestError
Fix frametest error
Yann Collet [Mon, 7 May 2018 01:26:14 +0000 (18:26 -0700)]
fixed minor conversion warning
Yann Collet [Sun, 6 May 2018 23:47:31 +0000 (16:47 -0700)]
small PA optimization
which measurably improves speed
on levels 9+
Yann Collet [Sun, 6 May 2018 02:59:00 +0000 (19:59 -0700)]
small extDict : fixed side-effect
don't fix dictionaries of size 0.
setting dictEnd == source triggers prefix mode,
thus removing possibility to use CDict.
Yann Collet [Sun, 6 May 2018 01:24:11 +0000 (18:24 -0700)]
fixed frametest error
The error can be reproduced using following command :
./frametest -v -i100000000 -s1659 -t31096808
It's actually a bug in the stream LZ4 API,
when starting a new stream
and providing a first chunk to complete with size < MINMATCH.
In which case, the chunk becomes a dictionary.
No hash was generated and stored,
but the chunk is accessible as default position 0 points to dictStart,
and position 0 is still within MAX_DISTANCE.
Then, next attempt to read 32-bits from position 0 fails.
The issue would have been mitigated by starting from index 64 KB,
effectively eliminating position 0 as too far away.
The proper fix is to eliminate such "dictionary" as too small.
Which is what this patch does.
Yann Collet [Sat, 5 May 2018 21:32:57 +0000 (14:32 -0700)]
lz4hc: fixed PA / SC parameter order
also :
reserved PA for levels 9+ (instead of 8+).
In most cases, speed is lower, and compression benefit is not worth.
Yann Collet [Sat, 5 May 2018 21:10:30 +0000 (14:10 -0700)]
lz4hc: SC only enabled for opt parser
the trade off is not good for regular HC parser :
compression is a little bit better, but speed cost is too large in comparison.
Yann Collet [Sat, 5 May 2018 20:46:45 +0000 (13:46 -0700)]
fixed SC.opt integration with regular HC parser
Only enabled when searching forward.
note : it slighly improves compression ratio,
but measurably decreases speed.
Trade-off to analyse.
Yann Collet [Sat, 5 May 2018 20:31:03 +0000 (13:31 -0700)]
lz4hc: fixed performance issue
when combining both PA and CS optimizations
Yann Collet [Sat, 5 May 2018 02:13:33 +0000 (19:13 -0700)]
integrated chain swapper into HC match finder
slower than expected
Pattern analyzer and Chain Swapper
work slower when both activated.
Reasons unclear.
Yann Collet [Fri, 4 May 2018 23:18:46 +0000 (16:18 -0700)]
Merge pull request #536 from terrelln/make-install
Fix make install
Nick Terrell [Fri, 4 May 2018 21:33:59 +0000 (14:33 -0700)]
Attempt to fix travis
Nick Terrell [Fri, 4 May 2018 20:35:10 +0000 (13:35 -0700)]
Fix make install
* Uninstall didn't remove the pkg-config correctly.
* Fix `mandir`
* Allow overriding either upper- or lower-case location variables, but
always use the lower case variables.
* Add test case that ensures overriding both upper- and lower-case
variables is the same, and that the directory is empty after uninstall.
Yann Collet [Thu, 3 May 2018 23:31:41 +0000 (16:31 -0700)]
implemented search accelerator
greatly improves speed compared to non-accelerated,
especially for slower files.
On my laptop, -b12 :
```
calgary.tar : 4.3 MB/s => 9.0 MB/s
enwik7 : 10.2 MB/s => 13.3 MB/s
silesia.tar : 4.0 MB/s => 8.7 MB/s
```
Note : this is the simplified version,
without handling dictionaries, external buffer, nor pattern analyzer.
Current `dev` branch on these samples gives :
```
calgary.tar : 4.2 MB/s
enwik7 : 9.7 MB/s
silesia.tar : 3.5 MB/s
```
interestingly, it's slower,
presumably due to handling of dictionaries.
Yann Collet [Thu, 3 May 2018 23:01:24 +0000 (16:01 -0700)]
Added CDict speed graph to be used for release statement
Yann Collet [Thu, 3 May 2018 22:40:01 +0000 (15:40 -0700)]
updated API documentation
Yann Collet [Thu, 3 May 2018 22:38:32 +0000 (15:38 -0700)]
created LZ4HC_FindLongestMatch()
simplified match finder
only searching forward and within current buffer,
for easier testing of optimizations.
Yann Collet [Thu, 3 May 2018 22:37:51 +0000 (15:37 -0700)]
Merge pull request #529 from felixhandte/lz4f-fast-reset-for-streaming-only
LZ4F: Only Reset the LZ4_stream_t when Init'ing a Streaming Block
Yann Collet [Thu, 3 May 2018 19:55:04 +0000 (12:55 -0700)]
Merge pull request #530 from lz4/lz4fRingBuffer
Random lz4f clarifications
Yann Collet [Thu, 3 May 2018 18:54:57 +0000 (11:54 -0700)]
Merge branch 'dev' into lz4fRingBuffer
Yann Collet [Thu, 3 May 2018 18:35:50 +0000 (11:35 -0700)]
Merge pull request #528 from lz4/complexShortcut
Faster decoding speed
Cyan4973 [Thu, 3 May 2018 14:56:33 +0000 (07:56 -0700)]
fix comments / indentation
as requested by @terrelln
W. Felix Handte [Thu, 3 May 2018 03:29:07 +0000 (23:29 -0400)]
Only Reset the LZ4 Stream when Init'ing a Streaming Block
Yann Collet [Wed, 2 May 2018 23:52:33 +0000 (16:52 -0700)]
updated NEWS in preparation for v1.8.2
Yann Collet [Wed, 2 May 2018 23:43:51 +0000 (16:43 -0700)]
Merge branch 'lz4fRingBuffer' of github.com:Cyan4973/lz4 into lz4fRingBuffer
Yann Collet [Wed, 2 May 2018 23:41:15 +0000 (16:41 -0700)]
updated benchmark for v1.8.2
Yann Collet [Wed, 2 May 2018 23:41:15 +0000 (16:41 -0700)]
updated benchmark for v1.8.2
Yann Collet [Wed, 2 May 2018 23:05:42 +0000 (16:05 -0700)]
random lz4f clarifications
the initial intention was to update lz4f ring buffer strategy,
but lz4f doesn't use ring buffer.
Instead, it uses the destination buffer as much as possible,
and merely copies just what's required to preserve history
into its own buffer, at the end.
Pretty efficient.
This patch just clarifies a few comments and add some assert().
It's built on top of #528.
It also updates doc.
Yann Collet [Wed, 2 May 2018 21:24:24 +0000 (14:24 -0700)]
Merge branch 'dev' into lz4fRingBuffer
Yann Collet [Wed, 2 May 2018 21:22:35 +0000 (14:22 -0700)]
increased nbAttempts for lz4 -12
shaves one more kilobyte from silesia.tar
Yann Collet [Wed, 2 May 2018 20:57:33 +0000 (13:57 -0700)]
removed test that might be optimized away
under UB rule "no overflow on int"
Yann Collet [Wed, 2 May 2018 19:56:37 +0000 (12:56 -0700)]
introduce LZ4_decoderRingBufferSize()
fuzzer : fix and robustify ring buffer tests
Yann Collet [Wed, 2 May 2018 17:33:12 +0000 (10:33 -0700)]
simplify shortcut
Yann Collet [Wed, 2 May 2018 17:08:30 +0000 (10:08 -0700)]
Merge branch 'dev' into complexShortcut
Yann Collet [Wed, 2 May 2018 17:06:07 +0000 (10:06 -0700)]
Merge pull request #525 from lz4/testDecMerge
added a test case for LZ4_decompress_fast_usingDict …
Yann Collet [Tue, 1 May 2018 03:40:34 +0000 (20:40 -0700)]
Merge pull request #521 from lz4/BD_deterministic
fix lz4hc -BD non-determinism
Cyan4973 [Tue, 1 May 2018 01:56:16 +0000 (18:56 -0700)]
renamed variable for clarity
lowLimit -> lowestMatchIndex
Cyan4973 [Tue, 1 May 2018 01:50:56 +0000 (18:50 -0700)]
added visual test dir to .gitignore
Yann Collet [Mon, 30 Apr 2018 23:08:16 +0000 (16:08 -0700)]
lz4hc changed variable
to reduce confusion
dictLowLimit => dictStart
Yann Collet [Mon, 30 Apr 2018 22:55:33 +0000 (15:55 -0700)]
clarified streaming decompression function
restrictions for ring buffer
Yann Collet [Mon, 30 Apr 2018 22:32:37 +0000 (15:32 -0700)]
Merge pull request #527 from svpv/fastDec
lz4.c: two-stage shortcut for LZ4_decompress_generic
Yann Collet [Sun, 29 Apr 2018 15:56:44 +0000 (08:56 -0700)]
Merge pull request #523 from svpv/makeV1
lib/Makefile: show commands with V=1
Cyan4973 [Sun, 29 Apr 2018 15:47:08 +0000 (08:47 -0700)]
Merge branch 'dev' of github.com:lz4/lz4 into dev
Cyan4973 [Sun, 29 Apr 2018 15:46:39 +0000 (08:46 -0700)]
updated NEWS for v1.8.2
mentioning work from @svpv
Yann Collet [Sun, 29 Apr 2018 15:45:16 +0000 (08:45 -0700)]
Merge pull request #526 from svpv/makeV1
lib/Makefile: show commands with V=1
Cyan4973 [Sun, 29 Apr 2018 14:42:24 +0000 (07:42 -0700)]
added a test case for LZ4_decompress_fast_usingDict
with a separated dictionary
since a joined dictionary is now detected as prefix64K.
Also : fixed a minor warning under msys
Yann Collet [Sun, 29 Apr 2018 14:41:35 +0000 (07:41 -0700)]
Merge pull request #515 from svpv/refactorDec
lz4.c: refactor the decoding routines
Cyan4973 [Sat, 28 Apr 2018 17:42:52 +0000 (10:42 -0700)]
ignore windows+msys artefacts
Alexey Tourbin [Sat, 28 Apr 2018 08:14:40 +0000 (11:14 +0300)]
lz4.c: two-stage shortcut for LZ4_decompress_generic
Alexey Tourbin [Sat, 28 Apr 2018 04:16:46 +0000 (07:16 +0300)]
lib/Makefile: show commands with V=1
`make V=1` will now show the commands executed to build the library.
A similar technique is used in e.g. linux/Makefile.
The bulk of this change is produced with the following vim command:
:g!/^\t@echo\>/s/^\t@/\t\$(Q)/
Yann Collet [Sat, 28 Apr 2018 00:22:06 +0000 (17:22 -0700)]
Merge pull request #522 from svpv/refactorDec
Refactor dec
Yann Collet [Fri, 27 Apr 2018 20:52:30 +0000 (13:52 -0700)]
Merge pull request #520 from felixhandte/frame-dict-nits
Minor Fixes to Dictionary Preparation in LZ4 Frame
Yann Collet [Fri, 27 Apr 2018 19:59:20 +0000 (12:59 -0700)]
Merge branch 'dev' into BD_deterministic
Yann Collet [Fri, 27 Apr 2018 19:46:49 +0000 (12:46 -0700)]
fix lz4hc -BD non-determinism
related to chain table update
Yann Collet [Fri, 27 Apr 2018 18:44:47 +0000 (11:44 -0700)]
lz4hc : minor editions for clarity
Yann Collet [Fri, 27 Apr 2018 18:46:29 +0000 (11:46 -0700)]
Merge pull request #519 from lz4/fdParser
Faster decoding speed
W. Felix Handte [Fri, 27 Apr 2018 18:10:27 +0000 (14:10 -0400)]
Avoid Possibly Redundant Table Clears When Loading HC Dict
W. Felix Handte [Fri, 27 Apr 2018 17:59:02 +0000 (13:59 -0400)]
Remove Redundant LZ4_resetStream() Call
W. Felix Handte [Fri, 27 Apr 2018 17:57:10 +0000 (13:57 -0400)]
Rename LZ4F_applyCDict() -> LZ4F_initStream()
Yann Collet [Fri, 27 Apr 2018 16:04:09 +0000 (09:04 -0700)]
ensure favorDecSpeed is properly initialized
also :
- fix a potential malloc error
- proper use of ALLOC macro inside lz4hc
- update html API doc
Yann Collet [Fri, 27 Apr 2018 15:43:40 +0000 (08:43 -0700)]
updated NEWS, in preparation for v1.8.2
Alexey Tourbin [Fri, 27 Apr 2018 12:00:11 +0000 (15:00 +0300)]
lz4.c: fixed the LZ4_decompress_fast_continue case
The change is very similar to that of the LZ4_decompress_safe_continue
case. The only reason a make this a separate change is to ensure that
the fuzzer, after it's been enhanced, can detect the flaw in
LZ4_decompress_fast_continue, and that the change indeed fixes the flaw.
Alexey Tourbin [Fri, 27 Apr 2018 04:06:37 +0000 (07:06 +0300)]
fuzzer.c: enabled ring buffer tests for decompress_fast
Ring buffer tests were performed only with LZ4_decompress_safe_continue,
leaving my buggy changes to LZ4_decompress_safe_continue undetected.
The tests are now replicated and performed in a similar manner for both
LZ4_decompress_safe_continue and LZ4_decompress_safe_continue (except
for the small buffer case where only one function can be tested,
because part of the dictionary is overwritten with the output).
I also updated function names in the messages (changed them to the
actual ones). The error was reported for LZ4_decompress_safe(),
which I found misleading.
Yann Collet [Fri, 27 Apr 2018 00:02:20 +0000 (17:02 -0700)]
fixed a number of minor cast warnings
Yann Collet [Thu, 26 Apr 2018 23:47:50 +0000 (16:47 -0700)]
Merge pull request #518 from felixhandte/fix-517-dict-size-truncation
Limit Dictionary Size During LZ4F Decompression
Yann Collet [Thu, 26 Apr 2018 23:40:33 +0000 (16:40 -0700)]
Merge pull request #516 from felixhandte/merge-dest-size
Merge _destSize Compress Variant into LZ4_compress_generic()
Yann Collet [Thu, 26 Apr 2018 22:49:32 +0000 (15:49 -0700)]
fasterDecSpeed can be triggered from cli with --favor-decSpeed
Yann Collet [Thu, 26 Apr 2018 22:18:44 +0000 (15:18 -0700)]
favorDecSpeed feature can be triggered from lz4frame
and lz4hc.
W. Felix Handte [Thu, 26 Apr 2018 19:42:16 +0000 (15:42 -0400)]
Merge _destSize Compress Variant into LZ4_compress_generic()
W. Felix Handte [Thu, 26 Apr 2018 21:25:12 +0000 (17:25 -0400)]
Add _destSize() to Fullbench
W. Felix Handte [Thu, 26 Apr 2018 20:53:40 +0000 (16:53 -0400)]
Limit Dictionary Size During LZ4F Decompression
Fixes lz4/lz4#517.
Yann Collet [Thu, 26 Apr 2018 20:01:59 +0000 (13:01 -0700)]
introduced ability to parse for decompression speed
triggered through an enum.
Now, it's still necessary to properly expose this capability
all the way up to the cli.
Alexey Tourbin [Thu, 26 Apr 2018 04:46:26 +0000 (07:46 +0300)]
lz4.c: fixed the LZ4_decompress_safe_continue case
The previous change broke decoding with a ring buffer. That's because
I didn't realize that the "double dictionary mode" was possible, i.e.
that the decoding routine can look both at the first part of the
dictionary passed as prefix and the second part passed via dictStart+dictSize.
So this change introduces the LZ4_decompress_safe_doubleDict helper,
which handles this "double dictionary" situation. (This is a bit of
a misnomer, there is only one dictionary, but I can't think of a better
name, and perhaps the designation is not all too bad.) The helper is
used only once, in LZ4_decompress_safe_continue, it should be inlined
with LZ4_FORCE_O2_GCC_PPC64LE attached to LZ4_decompress_safe_continue.
(Also, in the helper functions, I change the dictStart parameter type
to "const void*", to avoid a cast when calling helpers. In the helpers,
the upcast to "BYTE*" is still required, for compatibility with C++.)
So this fixes the case of LZ4_decompress_safe_continue, and I'm
surprised by the fact that the fuzzer is now happy and does not detect
a similar problem with LZ4_decompress_fast_continue. So before fixing
LZ4_decompress_fast_continue, the next logical step is to enhance
the fuzzer.
Cyan4973 [Wed, 25 Apr 2018 13:42:57 +0000 (06:42 -0700)]
minor edit of block format
clarifying parsing restrictions near end of block.