platform/upstream/lz4.git
5 years agoMerge pull request #565 from lz4/lz4f_init
Yann Collet [Mon, 10 Sep 2018 20:11:56 +0000 (13:11 -0700)]
Merge pull request #565 from lz4/lz4f_init

Introduced constants LZ4F_INIT_*

5 years agoMerge pull request #567 from laffer1/dev
Yann Collet [Mon, 10 Sep 2018 18:59:09 +0000 (11:59 -0700)]
Merge pull request #567 from laffer1/dev

Add support for MidnightBSD

5 years agoAdd support for MidnightBSD
Lucas Holt [Sat, 8 Sep 2018 18:46:54 +0000 (14:46 -0400)]
Add support for MidnightBSD

5 years agoMerge pull request #563 from lz4/docDict
Yann Collet [Thu, 6 Sep 2018 19:43:29 +0000 (12:43 -0700)]
Merge pull request #563 from lz4/docDict

updated documentation for dictionary compression

5 years agoadded comments on version numbers
Yann Collet [Thu, 6 Sep 2018 00:57:15 +0000 (17:57 -0700)]
added comments on version numbers

5 years agoMerge pull request #564 from lz4/cTime
Yann Collet [Thu, 6 Sep 2018 00:48:40 +0000 (17:48 -0700)]
Merge pull request #564 from lz4/cTime

fixed compression time

5 years agoframeCompress : added an error detection case
Yann Collet [Wed, 5 Sep 2018 23:45:46 +0000 (16:45 -0700)]
frameCompress : added an error detection case

check for potential input data not consumed.

5 years agoMerge branch 'dev' into lz4f_init
Yann Collet [Wed, 5 Sep 2018 23:43:46 +0000 (16:43 -0700)]
Merge branch 'dev' into lz4f_init

5 years agoMerge pull request #548 from cd127/dev
Yann Collet [Wed, 5 Sep 2018 23:35:59 +0000 (16:35 -0700)]
Merge pull request #548 from cd127/dev

Fix bug in frame decompress example

5 years agoIntroduced constants LZ4F_INIT_*
Yann Collet [Wed, 5 Sep 2018 23:04:05 +0000 (16:04 -0700)]
Introduced constants LZ4F_INIT_*

to simplify initialization of lz4frame.h structures.
Partially in response to #546.

5 years agoMerge pull request #556 from kou/add-missing-exeext
Yann Collet [Wed, 5 Sep 2018 22:55:34 +0000 (15:55 -0700)]
Merge pull request #556 from kou/add-missing-exeext

Add missing $(EXT)

5 years agofixed compression time
Yann Collet [Wed, 5 Sep 2018 22:16:03 +0000 (15:16 -0700)]
fixed compression time

displayed at the end (#555)

5 years agoMerge pull request #562 from lz4/roundTrip
Yann Collet [Wed, 5 Sep 2018 21:07:53 +0000 (14:07 -0700)]
Merge pull request #562 from lz4/roundTrip

Round trip test for #560

5 years agoupdated documentation regarding dictionary compression
Yann Collet [Wed, 5 Sep 2018 21:05:08 +0000 (14:05 -0700)]
updated documentation regarding dictionary compression

following suggestion from @stbrumme (#558)

Also : bumped version number, regenerated man page and html doc

5 years agoremoved one assert() condition
Yann Collet [Wed, 5 Sep 2018 18:25:10 +0000 (11:25 -0700)]
removed one assert() condition

which is not correct when using LZ4_HC with dictionary and starting from a low address (<0x10000).

5 years agomade roundTripTest fully general
Yann Collet [Wed, 5 Sep 2018 01:23:21 +0000 (18:23 -0700)]
made roundTripTest fully general

no longer "locked" on level 9

5 years agofixed #560
Yann Collet [Wed, 5 Sep 2018 01:21:40 +0000 (18:21 -0700)]
fixed #560

it was a fairly complex scenario,
involving source files > 64K
and some extraordinary conditions related to specific layout of ranges of zeroes.
and only on level 9.

5 years agofix minor cast warning for C++ compilation
Yann Collet [Wed, 5 Sep 2018 00:37:56 +0000 (17:37 -0700)]
fix minor cast warning for C++ compilation

5 years agonew test program : roundTripTest
Yann Collet [Tue, 4 Sep 2018 23:50:34 +0000 (16:50 -0700)]
new test program : roundTripTest

make a round trip test with arbitrary input file,
generate an `abort()` on error,
to work in tandem with `afl`.

note : currently locked on level 9, to investigate #560.

5 years agoAdd missing $(EXT)
Kouhei Sutou [Mon, 13 Aug 2018 05:17:54 +0000 (14:17 +0900)]
Add missing $(EXT)

5 years agoMerge pull request #553 from mrjoel/mrjoel/cmake-optional-lz4c
Yann Collet [Thu, 9 Aug 2018 21:07:45 +0000 (14:07 -0700)]
Merge pull request #553 from mrjoel/mrjoel/cmake-optional-lz4c

Add CMake option to not build legacy lz4c program

5 years agoAdd CMake option to not build legacy lz4c program
Joel Johnson [Thu, 2 Aug 2018 06:23:00 +0000 (00:23 -0600)]
Add CMake option to not build legacy lz4c program

6 years agoMerge pull request #552 from jackluo923/dev
Yann Collet [Mon, 30 Jul 2018 10:53:11 +0000 (12:53 +0200)]
Merge pull request #552 from jackluo923/dev

fixed spelling mistake in lz4_manual and lz4.h

6 years agofixed spelling mistake in lz4.h
Jack Luo [Sun, 29 Jul 2018 02:21:57 +0000 (22:21 -0400)]
fixed spelling mistake in lz4.h

6 years agofixed spelling mistake in lz4_manual
Jack Luo [Sun, 29 Jul 2018 02:12:55 +0000 (22:12 -0400)]
fixed spelling mistake in lz4_manual

6 years agoMerge pull request #547 from jennifermliu/dev
Yann Collet [Tue, 17 Jul 2018 13:42:48 +0000 (15:42 +0200)]
Merge pull request #547 from jennifermliu/dev

Add --fast command to cli

6 years agoMerge pull request #550 from terrelln/compress-doc
Yann Collet [Mon, 16 Jul 2018 10:15:21 +0000 (03:15 -0700)]
Merge pull request #550 from terrelln/compress-doc

Fix LZ4_compress_fast_continue() docs

6 years agoFix LZ4_compress_fast_continue() docs
Nick Terrell [Tue, 10 Jul 2018 18:44:03 +0000 (11:44 -0700)]
Fix LZ4_compress_fast_continue() docs

Fixes #549.

6 years agoFix bug in frame decompress example
Cedric De Brito [Mon, 2 Jul 2018 12:35:04 +0000 (14:35 +0200)]
Fix bug in frame decompress example

The decompression was failing as the srcEnd pointer in
decompress_file_internal was wrongly computed beyond
the end of the memory block.
We need to account for the fact that the header ("info")
was already read in the calling function ("alreadyConsumed").

6 years agoFixed bugs about incorrect acceleration calculation and benchmarking negative compres...
Jennifer Liu [Wed, 27 Jun 2018 20:36:38 +0000 (13:36 -0700)]
Fixed bugs about incorrect acceleration calculation and benchmarking negative compresion level

6 years agoFixed invalid argument test and reformatted else
Jennifer Liu [Wed, 27 Jun 2018 00:53:12 +0000 (17:53 -0700)]
Fixed invalid argument test and reformatted else

6 years agoFixed code based on comments from pull request
Jennifer Liu [Wed, 27 Jun 2018 00:31:15 +0000 (17:31 -0700)]
Fixed code based on comments from pull request

6 years agoRemoved duplicated circile.yml file
Jennifer Liu [Tue, 26 Jun 2018 21:48:58 +0000 (14:48 -0700)]
Removed duplicated circile.yml file

6 years agoFixed lz4 not found error part 2
Jennifer Liu [Tue, 26 Jun 2018 18:24:21 +0000 (11:24 -0700)]
Fixed lz4 not found error part 2

6 years agoFixed lz4 not found error
Jennifer Liu [Tue, 26 Jun 2018 18:15:55 +0000 (11:15 -0700)]
Fixed lz4 not found error

6 years agodelete commented out code in bench.c
Jennifer Liu [Tue, 26 Jun 2018 17:49:39 +0000 (10:49 -0700)]
delete commented out code in bench.c

6 years agofix yml file
Jennifer Liu [Tue, 26 Jun 2018 17:33:15 +0000 (10:33 -0700)]
fix yml file

6 years agoFixed bug about file to be compressed is not present
Jennifer Liu [Tue, 26 Jun 2018 01:51:47 +0000 (18:51 -0700)]
Fixed bug about file to be compressed is not present

6 years agoAdded --fast command to cli
Jennifer Liu [Tue, 26 Jun 2018 00:46:39 +0000 (17:46 -0700)]
Added --fast command to cli

6 years agoset up sample .ysml
Jennifer Liu [Wed, 20 Jun 2018 20:37:49 +0000 (13:37 -0700)]
set up sample .ysml

6 years agoMerge pull request #543 from nrgiii/aix_32bit_fuzzer
Yann Collet [Fri, 1 Jun 2018 21:40:22 +0000 (14:40 -0700)]
Merge pull request #543 from nrgiii/aix_32bit_fuzzer

Aix 32bit fuzzer

6 years agoMerge branch 'master' of https://github.com/nrgiii/lz4 into aix_32bit_fuzzer
Norm Green [Fri, 1 Jun 2018 19:00:41 +0000 (12:00 -0700)]
Merge branch 'master' of https://github.com/nrgiii/lz4 into aix_32bit_fuzzer

6 years agoFix AIX 32 bit build problem of fuzzer.c
Norm Green [Fri, 1 Jun 2018 18:52:22 +0000 (11:52 -0700)]
Fix AIX 32 bit build problem of fuzzer.c

6 years agoMerge pull request #542 from wbx-github/dev
Yann Collet [Tue, 29 May 2018 21:20:48 +0000 (14:20 -0700)]
Merge pull request #542 from wbx-github/dev

allow to override uname when cross-compiling

6 years agoMerge pull request #541 from felixhandte/hc-extern-c
Yann Collet [Tue, 22 May 2018 21:20:45 +0000 (14:20 -0700)]
Merge pull request #541 from felixhandte/hc-extern-c

Add `extern "C"` Guard Around Experimental HC Declarations

6 years agoallow to override uname when cross-compiling
Waldemar Brodkorb [Tue, 22 May 2018 18:38:20 +0000 (20:38 +0200)]
allow to override uname when cross-compiling

When cross-compiling for example from Darwin to Linux it might be
useful to override uname output to force Linux and create Linux
libraries instead of Darwin libraries.

6 years agoAlso Fix Appveyor Cast Warning
W. Felix Handte [Tue, 22 May 2018 15:28:39 +0000 (11:28 -0400)]
Also Fix Appveyor Cast Warning

6 years agoAdd `extern "C"` Guards Around Experimental HC Declarations
W. Felix Handte [Tue, 22 May 2018 01:43:54 +0000 (21:43 -0400)]
Add `extern "C"` Guards Around Experimental HC Declarations

6 years agoRemove #define-rename of `LZ4_decompress_safe_forceExtDict`
W. Felix Handte [Tue, 22 May 2018 01:55:04 +0000 (21:55 -0400)]
Remove #define-rename of `LZ4_decompress_safe_forceExtDict`

6 years agoTest Linking C-Compiled Library and C++-Compiled Tests
W. Felix Handte [Tue, 22 May 2018 01:36:49 +0000 (21:36 -0400)]
Test Linking C-Compiled Library and C++-Compiled Tests

6 years agoMerge pull request #540 from fbrosson/Haiku
Yann Collet [Thu, 17 May 2018 17:48:32 +0000 (10:48 -0700)]
Merge pull request #540 from fbrosson/Haiku

Add Haiku as a validated target.

6 years agoAdd Haiku as a validated target.
fbrosson [Thu, 17 May 2018 16:52:53 +0000 (16:52 +0000)]
Add Haiku as a validated target.

lz4 1.8.2 works fine on Haiku and passes all tests.

6 years agoMerge pull request #531 from lz4/dev
Yann Collet [Mon, 7 May 2018 21:38:45 +0000 (14:38 -0700)]
Merge pull request #531 from lz4/dev

Preparing v1.8.2

6 years agoMerge pull request #537 from lz4/xpHCmf2
Yann Collet [Mon, 7 May 2018 20:06:43 +0000 (13:06 -0700)]
Merge pull request #537 from lz4/xpHCmf2

Speed optimization for optimal parser

6 years agorenamed variable for clarity
Yann Collet [Mon, 7 May 2018 19:14:26 +0000 (12:14 -0700)]
renamed variable for clarity

6 years agoMerge pull request #538 from lz4/frameTestError
Yann Collet [Mon, 7 May 2018 18:33:53 +0000 (11:33 -0700)]
Merge pull request #538 from lz4/frameTestError

Fix frametest error

6 years agofixed minor conversion warning
Yann Collet [Mon, 7 May 2018 01:26:14 +0000 (18:26 -0700)]
fixed minor conversion warning

6 years agosmall PA optimization
Yann Collet [Sun, 6 May 2018 23:47:31 +0000 (16:47 -0700)]
small PA optimization

which measurably improves speed
on levels 9+

6 years agosmall extDict : fixed side-effect
Yann Collet [Sun, 6 May 2018 02:59:00 +0000 (19:59 -0700)]
small extDict : fixed side-effect

don't fix dictionaries of size 0.
setting dictEnd == source triggers prefix mode,
thus removing possibility to use CDict.

6 years agofixed frametest error
Yann Collet [Sun, 6 May 2018 01:24:11 +0000 (18:24 -0700)]
fixed frametest error

The error can be reproduced using following command :
./frametest -v -i100000000 -s1659 -t31096808

It's actually a bug in the stream LZ4 API,
when starting a new stream
and providing a first chunk to complete with size < MINMATCH.
In which case, the chunk becomes a dictionary.
No hash was generated and stored,
but the chunk is accessible as default position 0 points to dictStart,
and position 0 is still within MAX_DISTANCE.
Then, next attempt to read 32-bits from position 0 fails.

The issue would have been mitigated by starting from index 64 KB,
effectively eliminating position 0 as too far away.

The proper fix is to eliminate such "dictionary" as too small.
Which is what this patch does.

6 years agolz4hc: fixed PA / SC parameter order
Yann Collet [Sat, 5 May 2018 21:32:57 +0000 (14:32 -0700)]
lz4hc: fixed PA / SC parameter order

also :
reserved PA for levels 9+ (instead of 8+).
In most cases, speed is lower, and compression benefit is not worth.

6 years agolz4hc: SC only enabled for opt parser
Yann Collet [Sat, 5 May 2018 21:10:30 +0000 (14:10 -0700)]
lz4hc: SC only enabled for opt parser

the trade off is not good for regular HC parser :
compression is a little bit better, but speed cost is too large in comparison.

6 years agofixed SC.opt integration with regular HC parser
Yann Collet [Sat, 5 May 2018 20:46:45 +0000 (13:46 -0700)]
fixed SC.opt integration with regular HC parser

Only enabled when searching forward.

note : it slighly improves compression ratio,
but measurably decreases speed.
Trade-off to analyse.

6 years agolz4hc: fixed performance issue
Yann Collet [Sat, 5 May 2018 20:31:03 +0000 (13:31 -0700)]
lz4hc: fixed performance issue

when combining both PA and CS optimizations

6 years agointegrated chain swapper into HC match finder
Yann Collet [Sat, 5 May 2018 02:13:33 +0000 (19:13 -0700)]
integrated chain swapper into HC match finder

slower than expected
Pattern analyzer and Chain Swapper
work slower when both activated.
Reasons unclear.

6 years agoMerge pull request #536 from terrelln/make-install
Yann Collet [Fri, 4 May 2018 23:18:46 +0000 (16:18 -0700)]
Merge pull request #536 from terrelln/make-install

Fix make install

6 years agoAttempt to fix travis
Nick Terrell [Fri, 4 May 2018 21:33:59 +0000 (14:33 -0700)]
Attempt to fix travis

6 years agoFix make install
Nick Terrell [Fri, 4 May 2018 20:35:10 +0000 (13:35 -0700)]
Fix make install

* Uninstall didn't remove the pkg-config correctly.
* Fix `mandir`
* Allow overriding either upper- or lower-case location variables, but
  always use the lower case variables.
* Add test case that ensures overriding both upper- and lower-case
  variables is the same, and that the directory is empty after uninstall.

6 years agoimplemented search accelerator
Yann Collet [Thu, 3 May 2018 23:31:41 +0000 (16:31 -0700)]
implemented search accelerator

greatly improves speed compared to non-accelerated,
especially for slower files.

On my laptop, -b12 :
```
calgary.tar :  4.3 MB/s =>  9.0 MB/s
enwik7      : 10.2 MB/s => 13.3 MB/s
silesia.tar :  4.0 MB/s =>  8.7 MB/s
```

Note : this is the simplified version,
without handling dictionaries, external buffer, nor pattern analyzer.
Current `dev` branch on these samples gives :
```
calgary.tar :  4.2 MB/s
enwik7      :  9.7 MB/s
silesia.tar :  3.5 MB/s
```

interestingly, it's slower,
presumably due to handling of dictionaries.

6 years agoAdded CDict speed graph to be used for release statement
Yann Collet [Thu, 3 May 2018 23:01:24 +0000 (16:01 -0700)]
Added CDict speed graph to be used for release statement

6 years agoupdated API documentation
Yann Collet [Thu, 3 May 2018 22:40:01 +0000 (15:40 -0700)]
updated API documentation

6 years agocreated LZ4HC_FindLongestMatch()
Yann Collet [Thu, 3 May 2018 22:38:32 +0000 (15:38 -0700)]
created LZ4HC_FindLongestMatch()

simplified match finder
only searching forward and within current buffer,
for easier testing of optimizations.

6 years agoMerge pull request #529 from felixhandte/lz4f-fast-reset-for-streaming-only
Yann Collet [Thu, 3 May 2018 22:37:51 +0000 (15:37 -0700)]
Merge pull request #529 from felixhandte/lz4f-fast-reset-for-streaming-only

LZ4F: Only Reset the LZ4_stream_t when Init'ing a Streaming Block

6 years agoMerge pull request #530 from lz4/lz4fRingBuffer
Yann Collet [Thu, 3 May 2018 19:55:04 +0000 (12:55 -0700)]
Merge pull request #530 from lz4/lz4fRingBuffer

Random lz4f clarifications

6 years agoMerge branch 'dev' into lz4fRingBuffer
Yann Collet [Thu, 3 May 2018 18:54:57 +0000 (11:54 -0700)]
Merge branch 'dev' into lz4fRingBuffer

6 years agoMerge pull request #528 from lz4/complexShortcut
Yann Collet [Thu, 3 May 2018 18:35:50 +0000 (11:35 -0700)]
Merge pull request #528 from lz4/complexShortcut

Faster decoding speed

6 years agofix comments / indentation
Cyan4973 [Thu, 3 May 2018 14:56:33 +0000 (07:56 -0700)]
fix comments / indentation

as requested by @terrelln

6 years agoOnly Reset the LZ4 Stream when Init'ing a Streaming Block
W. Felix Handte [Thu, 3 May 2018 03:29:07 +0000 (23:29 -0400)]
Only Reset the LZ4 Stream when Init'ing a Streaming Block

6 years agoupdated NEWS in preparation for v1.8.2
Yann Collet [Wed, 2 May 2018 23:52:33 +0000 (16:52 -0700)]
updated NEWS in preparation for v1.8.2

6 years agoMerge branch 'lz4fRingBuffer' of github.com:Cyan4973/lz4 into lz4fRingBuffer
Yann Collet [Wed, 2 May 2018 23:43:51 +0000 (16:43 -0700)]
Merge branch 'lz4fRingBuffer' of github.com:Cyan4973/lz4 into lz4fRingBuffer

6 years agoupdated benchmark for v1.8.2
Yann Collet [Wed, 2 May 2018 23:41:15 +0000 (16:41 -0700)]
updated benchmark for v1.8.2

6 years agoupdated benchmark for v1.8.2
Yann Collet [Wed, 2 May 2018 23:41:15 +0000 (16:41 -0700)]
updated benchmark for v1.8.2

6 years agorandom lz4f clarifications
Yann Collet [Wed, 2 May 2018 23:05:42 +0000 (16:05 -0700)]
random lz4f clarifications

the initial intention was to update lz4f ring buffer strategy,
but lz4f doesn't use ring buffer.
Instead, it uses the destination buffer as much as possible,
and merely copies just what's required to preserve history
into its own buffer, at the end.
Pretty efficient.

This patch just clarifies a few comments and add some assert().
It's built on top of #528.

It also updates doc.

6 years agoMerge branch 'dev' into lz4fRingBuffer
Yann Collet [Wed, 2 May 2018 21:24:24 +0000 (14:24 -0700)]
Merge branch 'dev' into lz4fRingBuffer

6 years agoincreased nbAttempts for lz4 -12
Yann Collet [Wed, 2 May 2018 21:22:35 +0000 (14:22 -0700)]
increased nbAttempts for lz4 -12

shaves one more kilobyte from silesia.tar

6 years agoremoved test that might be optimized away
Yann Collet [Wed, 2 May 2018 20:57:33 +0000 (13:57 -0700)]
removed test that might be optimized away

under UB rule "no overflow on int"

6 years agointroduce LZ4_decoderRingBufferSize()
Yann Collet [Wed, 2 May 2018 19:56:37 +0000 (12:56 -0700)]
introduce LZ4_decoderRingBufferSize()

fuzzer : fix and robustify ring buffer tests

6 years agosimplify shortcut
Yann Collet [Wed, 2 May 2018 17:33:12 +0000 (10:33 -0700)]
simplify shortcut

6 years agoMerge branch 'dev' into complexShortcut
Yann Collet [Wed, 2 May 2018 17:08:30 +0000 (10:08 -0700)]
Merge branch 'dev' into complexShortcut

6 years agoMerge pull request #525 from lz4/testDecMerge
Yann Collet [Wed, 2 May 2018 17:06:07 +0000 (10:06 -0700)]
Merge pull request #525 from lz4/testDecMerge

added a test case for LZ4_decompress_fast_usingDict  …

6 years agoMerge pull request #521 from lz4/BD_deterministic
Yann Collet [Tue, 1 May 2018 03:40:34 +0000 (20:40 -0700)]
Merge pull request #521 from lz4/BD_deterministic

fix lz4hc -BD non-determinism

6 years agorenamed variable for clarity
Cyan4973 [Tue, 1 May 2018 01:56:16 +0000 (18:56 -0700)]
renamed variable for clarity

lowLimit -> lowestMatchIndex

6 years agoadded visual test dir to .gitignore
Cyan4973 [Tue, 1 May 2018 01:50:56 +0000 (18:50 -0700)]
added visual test dir to .gitignore

6 years agolz4hc changed variable
Yann Collet [Mon, 30 Apr 2018 23:08:16 +0000 (16:08 -0700)]
lz4hc changed variable

to reduce confusion
dictLowLimit => dictStart

6 years agoclarified streaming decompression function
Yann Collet [Mon, 30 Apr 2018 22:55:33 +0000 (15:55 -0700)]
clarified streaming decompression function

restrictions for ring buffer

6 years agoMerge pull request #527 from svpv/fastDec
Yann Collet [Mon, 30 Apr 2018 22:32:37 +0000 (15:32 -0700)]
Merge pull request #527 from svpv/fastDec

lz4.c: two-stage shortcut for LZ4_decompress_generic

6 years agoMerge pull request #523 from svpv/makeV1
Yann Collet [Sun, 29 Apr 2018 15:56:44 +0000 (08:56 -0700)]
Merge pull request #523 from svpv/makeV1

lib/Makefile: show commands with V=1

6 years agoMerge branch 'dev' of github.com:lz4/lz4 into dev
Cyan4973 [Sun, 29 Apr 2018 15:47:08 +0000 (08:47 -0700)]
Merge branch 'dev' of github.com:lz4/lz4 into dev

6 years agoupdated NEWS for v1.8.2
Cyan4973 [Sun, 29 Apr 2018 15:46:39 +0000 (08:46 -0700)]
updated NEWS for v1.8.2

mentioning work from @svpv

6 years agoMerge pull request #526 from svpv/makeV1
Yann Collet [Sun, 29 Apr 2018 15:45:16 +0000 (08:45 -0700)]
Merge pull request #526 from svpv/makeV1

lib/Makefile: show commands with V=1