platform/upstream/lz4.git
6 years agorevert project name
Yann Collet [Sun, 14 Jan 2018 05:40:04 +0000 (21:40 -0800)]
revert project name

6 years agominor name update
Yann Collet [Sun, 14 Jan 2018 05:14:36 +0000 (21:14 -0800)]
minor name update

6 years agoMerge branch 'master' into coverity_scan
Yann Collet [Sun, 14 Jan 2018 05:09:02 +0000 (21:09 -0800)]
Merge branch 'master' into coverity_scan

6 years agoMerge pull request #449 from lz4/dev
Yann Collet [Sun, 14 Jan 2018 04:56:14 +0000 (20:56 -0800)]
Merge pull request #449 from lz4/dev

v1.8.1.2

6 years agoupdate man page
Yann Collet [Sun, 14 Jan 2018 03:06:33 +0000 (19:06 -0800)]
update man page

6 years agonicer console message for `make clean`
Yann Collet [Sun, 14 Jan 2018 02:51:33 +0000 (18:51 -0800)]
nicer console message for `make clean`

6 years agoupdated manuals to v1.8.1
Yann Collet [Sun, 14 Jan 2018 02:46:13 +0000 (18:46 -0800)]
updated manuals to v1.8.1

6 years agoMerge pull request #448 from sunpoet/dev
Yann Collet [Sun, 14 Jan 2018 00:33:56 +0000 (16:33 -0800)]
Merge pull request #448 from sunpoet/dev

Fix lz4 version

6 years agoFix lz4 version
Po-Chuan Hsieh [Sat, 13 Jan 2018 22:38:03 +0000 (06:38 +0800)]
Fix lz4 version

6 years agoupdated LZ4F_decompress() documentation
Yann Collet [Sat, 13 Jan 2018 21:16:31 +0000 (13:16 -0800)]
updated LZ4F_decompress() documentation

6 years agoMerge pull request #446 from lz4/dev
Yann Collet [Sat, 13 Jan 2018 00:23:16 +0000 (16:23 -0800)]
Merge pull request #446 from lz4/dev

LZ4 v1.8.1

6 years agoupdated NEWS for v1.8.1
Yann Collet [Fri, 12 Jan 2018 22:36:10 +0000 (14:36 -0800)]
updated NEWS for v1.8.1

6 years agoMerge pull request #442 from terrelln/441
Yann Collet [Wed, 10 Jan 2018 18:09:13 +0000 (19:09 +0100)]
Merge pull request #442 from terrelln/441

[lz4io] Fix decompression file stat with --rm

6 years agoMerge pull request #443 from terrelln/440
Yann Collet [Wed, 10 Jan 2018 18:06:21 +0000 (19:06 +0100)]
Merge pull request #443 from terrelln/440

[lz4f] Skip memcpy() on empty dictionary

6 years agoMerge pull request #444 from felixhandte/publish-static-functions
Yann Collet [Wed, 10 Jan 2018 17:48:11 +0000 (18:48 +0100)]
Merge pull request #444 from felixhandte/publish-static-functions

Add Option to Make lz4frame_static.h Functions Visible in Shared Objects

6 years agoAdd Option to Make lz4frame_static.h Functions Visible in Shared Objects
W. Felix Handte [Mon, 8 Jan 2018 19:46:22 +0000 (14:46 -0500)]
Add Option to Make lz4frame_static.h Functions Visible in Shared Objects

In some contexts, *cough*like at facebook*cough*, dynamic linking is used in
contexts which aren't truly dynamic. That is, the guarantee is maintained that
a program will only ever execute against the library version it was compiled
to interact with.

For those situations, introduce a compile-time flag that overrides hiding
these unstable APIs in shared objects.

6 years agoMerge pull request #434 from lz4/pattern
Yann Collet [Sat, 6 Jan 2018 05:58:41 +0000 (06:58 +0100)]
Merge pull request #434 from lz4/pattern

conditional pattern analysis

6 years agoMerge pull request #435 from lz4/destSize
Yann Collet [Sat, 6 Jan 2018 05:57:56 +0000 (06:57 +0100)]
Merge pull request #435 from lz4/destSize

lz4opt is now compatible with _destSize variant

6 years ago[lz4f] Skip memcpy() on empty dictionary
Nick Terrell [Fri, 5 Jan 2018 22:30:49 +0000 (14:30 -0800)]
[lz4f] Skip memcpy() on empty dictionary

6 years ago[lz4io] Refuse to set file stat for non-regular files
Nick Terrell [Fri, 5 Jan 2018 19:32:04 +0000 (11:32 -0800)]
[lz4io] Refuse to set file stat for non-regular files

6 years ago[lz4io] Fix decompression file stat with --rm
Nick Terrell [Fri, 5 Jan 2018 19:27:24 +0000 (11:27 -0800)]
[lz4io] Fix decompression file stat with --rm

6 years agoMerge pull request #439 from eli-b/doc-nohup-behavior
Yann Collet [Mon, 1 Jan 2018 23:19:59 +0000 (00:19 +0100)]
Merge pull request #439 from eli-b/doc-nohup-behavior

Docs: describe behavior under nohup

6 years agoMerge pull request #438 from eli-b/grammar-fix
Yann Collet [Mon, 1 Jan 2018 23:17:57 +0000 (00:17 +0100)]
Merge pull request #438 from eli-b/grammar-fix

Grammar: "to silent" -> "to silence"

6 years agoDocs: describe behavior under nohup
Eli Boyarski [Mon, 1 Jan 2018 13:34:16 +0000 (15:34 +0200)]
Docs: describe behavior under nohup

6 years agoGrammar: "to silent" -> "to silence"
Eli Boyarski [Mon, 1 Jan 2018 12:15:31 +0000 (14:15 +0200)]
Grammar: "to silent" -> "to silence"

6 years agoobject files in /tests
Yann Collet [Wed, 27 Dec 2017 12:02:40 +0000 (13:02 +0100)]
object files in /tests

object files created from /tests are compiled with specific flags (`-g -DLZ4_DEBUG=1`)
which are not welcomed in release binary.
`lib/*.o` files created from /tests are now stored in /tests, to avoid unintentional mix.

6 years agolz4opt supports _destSize
Yann Collet [Fri, 22 Dec 2017 11:47:59 +0000 (12:47 +0100)]
lz4opt supports _destSize

no longer limited to level 9

6 years ago/tests programs compiled with LZ4_DEBUG=1
Yann Collet [Fri, 22 Dec 2017 10:54:43 +0000 (11:54 +0100)]
/tests programs compiled with LZ4_DEBUG=1

to enable assert() within /lib

6 years agoconditional pattern analysis
Yann Collet [Fri, 22 Dec 2017 07:07:25 +0000 (08:07 +0100)]
conditional pattern analysis

Pattern analysis (currently limited to long ranges of identical bytes)
is actually detrimental to performance
when `nbSearches` is low.

Reason is : `nbSearches` provides a built-in protection for these cases.
The problem with patterns is that they dramatically increase the number of candidates to visit.
But with a low nbSearches, the match finder just aborts early.

In such cases, pattern analysis adds some complexity without reducing total nb of candidates.
It actually increases compression ratio a little bit, by filtering only "good" candidates,
but at a measurable speed cost, so it's not a good trade-off.

This patch makes pattern analysis optional.
It's enabled for levels 8+ only.

6 years agoMerge pull request #431 from lz4/newopt
Yann Collet [Thu, 21 Dec 2017 08:46:25 +0000 (09:46 +0100)]
Merge pull request #431 from lz4/newopt

new level 10

6 years agonew level 10
Yann Collet [Wed, 20 Dec 2017 13:14:01 +0000 (14:14 +0100)]
new level 10

lz4opt is only competitive vs lz4hc level 10.
Below that level, it doesn't match the speed / compression effectiveness of regular hc parser.

This patch propose to extend lz4opt to levels 10-12.
The new level 10 tend to compress a bit better and a bit faster than previous one (mileage vary depending on file)

The only downside is that `limitedDestSize` mode is now limited to max level 9 (vs 10),
since it's only compatible with regular HC parser.
(Note : I suspect it's possible to convert lz4opt to support it too, but haven't spent time into it).

6 years agoMerge pull request #427 from lz4/registerLess
Yann Collet [Fri, 15 Dec 2017 22:46:00 +0000 (14:46 -0800)]
Merge pull request #427 from lz4/registerLess

remove `register` keyword

6 years agoMerge pull request #424 from MaxRis/msbuild-properties-#423
Yann Collet [Mon, 11 Dec 2017 18:07:03 +0000 (10:07 -0800)]
Merge pull request #424 from MaxRis/msbuild-properties-#423

Msbuild properties #423

6 years agoEnableWholeProgramOptimization and UseStaticCRT msbuild custom properties
Max Risuhin [Mon, 11 Dec 2017 12:57:19 +0000 (14:57 +0200)]
EnableWholeProgramOptimization and UseStaticCRT msbuild custom properties

6 years agoremove `register` keyword
Yann Collet [Tue, 5 Dec 2017 01:10:23 +0000 (17:10 -0800)]
remove `register` keyword

deprecated in newer C++ versions,
and dubious utility

6 years agoMerge pull request #426 from aliceatlas/cli_chmod_fix
Yann Collet [Mon, 4 Dec 2017 22:38:00 +0000 (14:38 -0800)]
Merge pull request #426 from aliceatlas/cli_chmod_fix

Fix cli sometimes calling UTIL_setFileStat on /dev/null

6 years agoFix bug which could sometimes result in the lz4 cli chmodding /dev/null if running...
Alice Atlas [Mon, 4 Dec 2017 21:07:31 +0000 (16:07 -0500)]
Fix bug which could sometimes result in the lz4 cli chmodding /dev/null if running as root

6 years agominor updates to examples
Yann Collet [Sat, 25 Nov 2017 01:18:46 +0000 (17:18 -0800)]
minor updates to examples

see https://github.com/lz4/lz4/commit/810e2ca27b3561e0f6bfa7a88e0fde6faf807064#commitcomment-25810887

6 years agoAPI : changed a few variables' names for clarity
Yann Collet [Mon, 20 Nov 2017 18:27:05 +0000 (10:27 -0800)]
API : changed a few variables' names for clarity

updated relevant doc.
This patch has no impact on ABI/API, nor on binary generation.

6 years agoupdated NEWS
Yann Collet [Mon, 13 Nov 2017 23:54:55 +0000 (15:54 -0800)]
updated NEWS

6 years agoMerge pull request #416 from lz4/newopt
Yann Collet [Thu, 9 Nov 2017 22:13:30 +0000 (14:13 -0800)]
Merge pull request #416 from lz4/newopt

Improve Optimal parser

6 years agoadded code comments
Yann Collet [Thu, 9 Nov 2017 01:56:20 +0000 (17:56 -0800)]
added code comments

6 years agoadded constant TRAILING_LITERALS
Yann Collet [Thu, 9 Nov 2017 01:47:24 +0000 (17:47 -0800)]
added constant TRAILING_LITERALS

which is more explicit than its value `3`.
reported by @terrelln

6 years agolz4opt: simplified match finder invocation to LZ4HC_FindLongerMatch()
Yann Collet [Thu, 9 Nov 2017 01:11:51 +0000 (17:11 -0800)]
lz4opt: simplified match finder invocation to LZ4HC_FindLongerMatch()

6 years agoadded code comments
Yann Collet [Wed, 8 Nov 2017 16:42:59 +0000 (08:42 -0800)]
added code comments

6 years agofixed LZ4HC_reverseCountPattern()
Yann Collet [Wed, 8 Nov 2017 01:58:59 +0000 (17:58 -0800)]
fixed LZ4HC_reverseCountPattern()

for multi-bytes patterns
(which is not useful for the time being)

6 years agoremoved the ip++ at the beginning of block
Yann Collet [Wed, 8 Nov 2017 01:37:31 +0000 (17:37 -0800)]
removed the ip++ at the beginning of block

The first byte used to be skipped
to avoid a infinite self-comparison.
This is no longer necessary, since init() ensures that index starts at 64K.

The first byte is also useless to search when each block is independent,
but it's no longer the case when blocks are linked.

Removing the first-byte-skip saves
about 10 bytes / MB on files compressed with -BD4 (linked blocks 64Kb),
which feels correct as each MB has 16 blocks of 64KB.

6 years agoMerge pull request #418 from lz4/fasterCount
Yann Collet [Tue, 7 Nov 2017 20:09:32 +0000 (12:09 -0800)]
Merge pull request #418 from lz4/fasterCount

faster LZ4_count()

6 years agoremoved legacy version of LZ4HC_InsertAndFindBestMatch()
Yann Collet [Tue, 7 Nov 2017 19:33:40 +0000 (11:33 -0800)]
removed legacy version of LZ4HC_InsertAndFindBestMatch()

6 years agoensure `pattern` is a 1-byte repetition
Yann Collet [Tue, 7 Nov 2017 19:29:28 +0000 (11:29 -0800)]
ensure `pattern` is a 1-byte repetition

6 years agoremoved useless `(1 && ...)` condition
Yann Collet [Tue, 7 Nov 2017 19:22:57 +0000 (11:22 -0800)]
removed useless `(1 && ...)` condition

as reported by @terrelln

6 years agoimproved LZ4HC_reverseCountPattern() :
Yann Collet [Tue, 7 Nov 2017 19:05:48 +0000 (11:05 -0800)]
improved LZ4HC_reverseCountPattern() :

works for any repetitive pattern of length 1, 2 or 4 (but not 3!)
works for any endianess

6 years agofixed LZ4HC_countPattern()
Yann Collet [Tue, 7 Nov 2017 18:53:29 +0000 (10:53 -0800)]
fixed LZ4HC_countPattern()

- works with byte values other than `0`
- works for any repetitive pattern of length 1, 2 or 4 (but not 3!)
- works for little and big endian systems
- preserve speed of previous implementation

6 years agoadded LZ4_FORCEINLINE to counter gcc regression
Yann Collet [Tue, 7 Nov 2017 01:29:27 +0000 (17:29 -0800)]
added LZ4_FORCEINLINE to counter gcc regression

as recommended by @terrelln

6 years ago2-stages LZ4_count
Yann Collet [Mon, 6 Nov 2017 23:42:50 +0000 (15:42 -0800)]
2-stages LZ4_count

separate first branch from the rest of the compare loop
to get dedicated prediction.

measured a 3-4% compression speed improvement.

6 years agobuild: minor : `make lz4` doesn't compile liblz4 anymore
Yann Collet [Mon, 6 Nov 2017 23:20:08 +0000 (15:20 -0800)]
build: minor : `make lz4` doesn't compile liblz4 anymore

since it's not needed.

6 years agoMerge pull request #417 from sylvestre/dev
Yann Collet [Mon, 6 Nov 2017 21:56:44 +0000 (13:56 -0800)]
Merge pull request #417 from sylvestre/dev

When building with a C++ compiler, remove the 'register' keyword to silent a warning

6 years agoOnly ignore with C++17
Sylvestre Ledru [Mon, 6 Nov 2017 15:16:02 +0000 (16:16 +0100)]
Only ignore with C++17

6 years agoWhen building with a C++ compiler, remove the 'register' keyword to silent a warning
Sylvestre Ledru [Sun, 5 Nov 2017 10:48:00 +0000 (11:48 +0100)]
When building with a C++ compiler, remove the 'register' keyword to silent a warning

For example, with clang:

lz4.c:XXX:36: error: 'register' storage class specifier is deprecated and incompatible with C++17 [-Werror,-Wdeprecated-register]
static unsigned LZ4_NbCommonBytes (register reg_t val)
                                   ^~~~~~~~~

6 years agofixed minor static analyzer warning
Yann Collet [Fri, 3 Nov 2017 19:33:55 +0000 (12:33 -0700)]
fixed minor static analyzer warning

dead assignment

6 years agominor comment edit
Yann Collet [Fri, 3 Nov 2017 18:49:56 +0000 (11:49 -0700)]
minor comment edit

6 years agounified HC levels
Yann Collet [Fri, 3 Nov 2017 18:28:28 +0000 (11:28 -0700)]
unified HC levels

LZ4_setCompressionLevel() can be users accross the whole range of HC levels
No more transition issue between Optimal and HC modes

6 years agomoved ctx->end handling from parsers
Yann Collet [Fri, 3 Nov 2017 17:48:55 +0000 (10:48 -0700)]
moved ctx->end handling from parsers

responsibility better handled one layer above (LZ4HC_compress_generic())

6 years agoremoved ctx->searchNum
Yann Collet [Fri, 3 Nov 2017 17:30:52 +0000 (10:30 -0700)]
removed ctx->searchNum

nbSearches now transmitted directly as function parameter
easier to track and debug

6 years agoLZ4_compress_HC_continue_destSize() now compatible with optimal parser
Yann Collet [Fri, 3 Nov 2017 09:01:20 +0000 (02:01 -0700)]
LZ4_compress_HC_continue_destSize() now compatible with optimal parser

levels 11+

6 years agoremoves matches[] table
Yann Collet [Fri, 3 Nov 2017 08:37:43 +0000 (01:37 -0700)]
removes matches[] table

saves stack space
clearer match finder interface (no more table to fill)

6 years agoremoved useless parameter from hash chain matchfinder
Yann Collet [Fri, 3 Nov 2017 08:20:30 +0000 (01:20 -0700)]
removed useless parameter from hash chain matchfinder

used to be present for compatibility with binary tree matchfinder

6 years agoremoved code and reference to binary tree match finder
Yann Collet [Fri, 3 Nov 2017 08:18:12 +0000 (01:18 -0700)]
removed code and reference to binary tree match finder

reduced size of LZ4HC state

6 years agoimproved level 11 speed
Yann Collet [Fri, 3 Nov 2017 07:59:05 +0000 (00:59 -0700)]
improved level 11 speed

6 years agooptimized skip strategy for level 12
Yann Collet [Fri, 3 Nov 2017 07:15:52 +0000 (00:15 -0700)]
optimized skip strategy for level 12

6 years agonew level 11 uses 512 attempts
Yann Collet [Fri, 3 Nov 2017 02:50:08 +0000 (19:50 -0700)]
new level 11 uses 512 attempts

6 years agomore generic skip formula
Yann Collet [Fri, 3 Nov 2017 01:54:18 +0000 (18:54 -0700)]
more generic skip formula

improving speed

6 years agosmall adaptations for intermediate level 11
Yann Collet [Thu, 2 Nov 2017 23:25:10 +0000 (16:25 -0700)]
small adaptations for intermediate level 11

6 years agopartial search, while preserving compression ratio
Yann Collet [Thu, 2 Nov 2017 22:37:18 +0000 (15:37 -0700)]
partial search, while preserving compression ratio

tag interesting places

6 years agosearching match leading strictly farther does not work
Yann Collet [Thu, 2 Nov 2017 22:05:45 +0000 (15:05 -0700)]
searching match leading strictly farther does not work

sometimes, it's better to re-use same match but start it later,
in order to get shorter matchlength code

6 years agofixed last lost bytes in maximal mode
Yann Collet [Thu, 2 Nov 2017 21:53:06 +0000 (14:53 -0700)]
fixed last lost bytes in maximal mode

even gained 2 bytes on calgary.tar...
added conditional traces `g_debuglog_enable`

6 years agochanged strategy : opt[] path is complete after each match
Yann Collet [Thu, 2 Nov 2017 20:44:57 +0000 (13:44 -0700)]
changed strategy : opt[] path is complete after each match

previous strategy would leave a few "bad choices"
on the ground they would be fixed later,
but that requires passing through each position to make the fix
and cannot give the end position of the last useful match.

6 years agoMerge pull request #415 from lz4/fasterDecodingXp
Yann Collet [Wed, 1 Nov 2017 16:58:49 +0000 (09:58 -0700)]
Merge pull request #415 from lz4/fasterDecodingXp

Faster decoding xp

6 years agoMerge branch 'dev' into btopt
Yann Collet [Wed, 1 Nov 2017 00:55:01 +0000 (17:55 -0700)]
Merge branch 'dev' into btopt

6 years agominor change, to help store forwarding
Yann Collet [Tue, 31 Oct 2017 22:51:56 +0000 (15:51 -0700)]
minor change, to help store forwarding

in a marginal case (offset==4)

6 years agoextended shortcut match length to 18
Yann Collet [Tue, 31 Oct 2017 21:20:25 +0000 (14:20 -0700)]
extended shortcut match length to 18

6 years agoMerge pull request #408 from terrelln/time
Yann Collet [Tue, 31 Oct 2017 20:49:29 +0000 (13:49 -0700)]
Merge pull request #408 from terrelln/time

[bench] Use higher resolution timer on POSIX

6 years agominor : coding style : use ML_MASK constant
Yann Collet [Tue, 31 Oct 2017 19:22:15 +0000 (12:22 -0700)]
minor : coding style : use ML_MASK constant

6 years agoadded comments, as suggested by @terrelln
Yann Collet [Tue, 31 Oct 2017 18:49:57 +0000 (11:49 -0700)]
added comments, as suggested by @terrelln

6 years agofixed minor overflow mistake in optimal parser
Yann Collet [Tue, 31 Oct 2017 00:47:54 +0000 (17:47 -0700)]
fixed minor overflow mistake in optimal parser

saving 20 bytes on calgary.tar

6 years agofixed minor initialization warning
Yann Collet [Mon, 30 Oct 2017 23:10:25 +0000 (16:10 -0700)]
fixed minor initialization warning

6 years agomore complete shortcut - passes tests
Yann Collet [Mon, 30 Oct 2017 23:07:15 +0000 (16:07 -0700)]
more complete shortcut - passes tests

6 years agoMerge pull request #412 from mikir/symbolsVisibility
Yann Collet [Mon, 30 Oct 2017 22:57:43 +0000 (15:57 -0700)]
Merge pull request #412 from mikir/symbolsVisibility

Separated visibility from LZ4LIB_API macro.

6 years agoSeparated visibility from LZ4LIB_API macro.
mikir [Mon, 30 Oct 2017 12:44:24 +0000 (13:44 +0100)]
Separated visibility from LZ4LIB_API macro.

6 years agosmall modification of lz4 decoder to shortcut common case (short branch).
Yann Collet [Wed, 25 Oct 2017 08:10:53 +0000 (10:10 +0200)]
small modification of lz4 decoder to shortcut common case (short branch).

6 years agoadded hash chain with conditional length
Yann Collet [Wed, 25 Oct 2017 05:07:08 +0000 (07:07 +0200)]
added hash chain with conditional length

not a success yet

6 years agolz4opt: added hash chain search
Yann Collet [Sat, 21 Oct 2017 00:04:29 +0000 (17:04 -0700)]
lz4opt: added hash chain search

6 years agoswitched many types to int
Yann Collet [Fri, 20 Oct 2017 22:30:50 +0000 (15:30 -0700)]
switched many types to int

6 years agoremoved SET_PRICE macro
Yann Collet [Fri, 20 Oct 2017 20:44:49 +0000 (13:44 -0700)]
removed SET_PRICE macro

6 years agoremoved one macro usage
Yann Collet [Fri, 20 Oct 2017 20:32:45 +0000 (13:32 -0700)]
removed one macro usage

6 years agominor refactor
Yann Collet [Fri, 20 Oct 2017 19:05:00 +0000 (12:05 -0700)]
minor refactor

reduce variable scope
remove one macro usage

6 years agolz4opt: refactor sequence reverse traversal
Yann Collet [Fri, 20 Oct 2017 18:32:15 +0000 (11:32 -0700)]
lz4opt: refactor sequence reverse traversal

6 years agorefactor variable matchnum
Yann Collet [Fri, 20 Oct 2017 18:24:56 +0000 (11:24 -0700)]
refactor variable matchnum

separate initial and iterative search
renamed nb_matches

6 years agosimplified initial cost conditions
Yann Collet [Fri, 20 Oct 2017 18:00:10 +0000 (11:00 -0700)]
simplified initial cost conditions

llen integrated in opt[]

6 years agoadded assert
Yann Collet [Thu, 19 Oct 2017 23:47:25 +0000 (16:47 -0700)]
added assert