platform/upstream/lz4.git
6 years agofixed minor declaration issue with clang on msys
Cyan4973 [Mon, 23 Apr 2018 22:52:44 +0000 (15:52 -0700)]
fixed minor declaration issue with clang on msys

6 years agodisable shortcut for LZ4_decompress_fast()
Cyan4973 [Mon, 23 Apr 2018 22:47:08 +0000 (15:47 -0700)]
disable shortcut for LZ4_decompress_fast()

improving speed

6 years agofullbench compiled without assert()
Cyan4973 [Mon, 23 Apr 2018 22:42:27 +0000 (15:42 -0700)]
fullbench compiled without assert()

to better reflect release speed

6 years agoMerge pull request #510 from terrelln/bug-fix
Yann Collet [Mon, 23 Apr 2018 22:28:19 +0000 (15:28 -0700)]
Merge pull request #510 from terrelln/bug-fix

Fix input size validation edge cases

6 years agoFix compilation error and assert.
Nick Terrell [Mon, 23 Apr 2018 21:21:02 +0000 (14:21 -0700)]
Fix compilation error and assert.

6 years agoFix input size validation edge cases
Nick Terrell [Mon, 23 Apr 2018 20:14:19 +0000 (13:14 -0700)]
Fix input size validation edge cases

The bug is a read up to 2 bytes past the end of the buffer.
There are three cases for this bug, one for each test case added.

* An empty input causes `token = *ip++` to read one byte too far.
* A one byte input with `(token >> ML_BITS) == RUN_MASK` causes
  one extra byte to be read without validation. This could be
  combined with the first bug to cause 2 extra bytes to be read.
* The case pointed out in issue #508, where `ip == iend` at the
  beginning of the loop after taking the shortcut.

Benchmarks show no regressions on clang or gcc-7 on both my mac
and devserver.

Fixes #508.

6 years agoMerge pull request #509 from svpv/clarifyFastRisks
Yann Collet [Mon, 23 Apr 2018 02:30:24 +0000 (19:30 -0700)]
Merge pull request #509 from svpv/clarifyFastRisks

lz4.h: clarify the risks of using LZ4_decompress_fast()

6 years agolz4.h: clarify the risks of using LZ4_decompress_fast()
Alexey Tourbin [Sun, 22 Apr 2018 22:43:30 +0000 (01:43 +0300)]
lz4.h: clarify the risks of using LZ4_decompress_fast()

The notes about "security guarantee" and "malicious inputs" seemed
a bit non-technical to me, so I took the liberty to tone them down
and instead describe the actual risks in technical terms.  Namely,
the function never writes past the end of the output buffer, so
a direct hostile takeover (resulting in arbitrary code execution
soon after the return from the function) is not possible.  However,
the application can crash because of reads from unmapped pages.

I also took the liberty to describe what I believe is the only sensible
usage scenario for the function: "This function is only usable if the
originalSize of uncompressed data is known in advance," etc.

6 years agoMerge pull request #503 from lz4/l120
Yann Collet [Thu, 19 Apr 2018 18:50:20 +0000 (11:50 -0700)]
Merge pull request #503 from lz4/l120

minor length reduction of several large lines

6 years agoMerge pull request #502 from lhacc1/dev
Yann Collet [Thu, 19 Apr 2018 17:52:48 +0000 (10:52 -0700)]
Merge pull request #502 from lhacc1/dev

Wrap likely/unlikely macroses with #ifndef

6 years agomodified indentation for consistency
Yann Collet [Thu, 19 Apr 2018 17:50:40 +0000 (10:50 -0700)]
modified indentation for consistency

6 years agominor length reduction of several large lines
Yann Collet [Wed, 18 Apr 2018 23:49:27 +0000 (16:49 -0700)]
minor length reduction of several large lines

6 years agoMerge pull request #497 from lz4/lowAddr
Yann Collet [Wed, 18 Apr 2018 17:16:25 +0000 (10:16 -0700)]
Merge pull request #497 from lz4/lowAddr

Compatibility with low memory addresses

6 years agoWrap likely/unlikely macroses with #ifndef
Dmitrii Rodionov [Wed, 18 Apr 2018 09:20:56 +0000 (12:20 +0300)]
Wrap likely/unlikely macroses with #ifndef

It prevent redefine error when project using lz4 has its own likely/unlikely
macroses.

6 years agofixed LZ4_compress_fast_extState_fastReset() in 32-bit mode
Yann Collet [Tue, 17 Apr 2018 23:47:56 +0000 (16:47 -0700)]
fixed LZ4_compress_fast_extState_fastReset() in 32-bit mode

6 years agofix dictDelta setting error
Yann Collet [Tue, 17 Apr 2018 23:18:37 +0000 (16:18 -0700)]
fix dictDelta setting error

wrong test

6 years agofix matchIndex overflow
Yann Collet [Tue, 17 Apr 2018 22:29:17 +0000 (15:29 -0700)]
fix matchIndex overflow

can happen with dictCtx

6 years agoMerge branch 'dev' into lowAddr
Yann Collet [Tue, 17 Apr 2018 19:07:22 +0000 (12:07 -0700)]
Merge branch 'dev' into lowAddr

6 years agoMerge pull request #501 from felixhandte/fix-dict-load-offset
Yann Collet [Tue, 17 Apr 2018 19:06:44 +0000 (12:06 -0700)]
Merge pull request #501 from felixhandte/fix-dict-load-offset

Always Bump Offset by 64 KB in LZ4_loadDict()

6 years agoAlways Bump Offset by 64 KB in LZ4_loadDict()
W. Felix Handte [Tue, 17 Apr 2018 18:01:44 +0000 (14:01 -0400)]
Always Bump Offset by 64 KB in LZ4_loadDict()

This actually ensures the guarantee referred to in the comment in
LZ4_compress_fast_continue().

6 years agofixed dictCtx compression
Yann Collet [Tue, 17 Apr 2018 06:59:42 +0000 (23:59 -0700)]
fixed dictCtx compression

6 years agoedited a few traces for debugging
Yann Collet [Tue, 17 Apr 2018 00:15:02 +0000 (17:15 -0700)]
edited a few traces for debugging

6 years agofixed minor format warnings
Yann Collet [Mon, 16 Apr 2018 23:54:03 +0000 (16:54 -0700)]
fixed minor format warnings

6 years agofixed fuzzer tests
Yann Collet [Mon, 16 Apr 2018 23:14:28 +0000 (16:14 -0700)]
fixed fuzzer tests

which were modified in parallel within branc `dev`

6 years agoMerge branch 'dev' into lowAddr
Yann Collet [Mon, 16 Apr 2018 23:12:38 +0000 (16:12 -0700)]
Merge branch 'dev' into lowAddr

6 years agofixed gcc performance regression
Yann Collet [Mon, 16 Apr 2018 22:11:28 +0000 (15:11 -0700)]
fixed gcc performance regression

6 years agoMerge pull request #499 from felixhandte/lz4-attach-dict-tests
Yann Collet [Fri, 13 Apr 2018 20:22:38 +0000 (13:22 -0700)]
Merge pull request #499 from felixhandte/lz4-attach-dict-tests

Test LZ4_attach_dictionary() and Friends

6 years agoFurther Test that ExtDictCtx Mode Produces the Exact Same Output
W. Felix Handte [Thu, 12 Apr 2018 23:17:53 +0000 (19:17 -0400)]
Further Test that ExtDictCtx Mode Produces the Exact Same Output

6 years agoAdd Tests for LZ4_attach_dictionary and Friends
W. Felix Handte [Thu, 12 Apr 2018 22:23:01 +0000 (18:23 -0400)]
Add Tests for LZ4_attach_dictionary and Friends

6 years agofixed minor unused variable warning
Yann Collet [Fri, 13 Apr 2018 09:45:32 +0000 (02:45 -0700)]
fixed minor unused variable warning

6 years agoadded comment on variables required after _next_match
Yann Collet [Fri, 13 Apr 2018 09:26:14 +0000 (02:26 -0700)]
added comment on variables required after _next_match

6 years agofixed potential ptrdiff_t overflow (32-bits mode)
Yann Collet [Fri, 13 Apr 2018 09:10:53 +0000 (02:10 -0700)]
fixed potential ptrdiff_t overflow (32-bits mode)

Also removed pointer comparison, which should solve #485

6 years agocompatibility with gcc-4.4 string.h version
Cyan4973 [Fri, 13 Apr 2018 08:01:54 +0000 (01:01 -0700)]
compatibility with gcc-4.4 string.h version

Someone found it would be a great idea to define there a global variable under the very generic name "index".
Cause problem with shadow warnings, so no variable can be named "index" now ...

Also : automatically update API manual

6 years agoadded sudo rights for low-mem-address tests
Cyan4973 [Fri, 13 Apr 2018 07:59:27 +0000 (00:59 -0700)]
added sudo rights for low-mem-address tests

6 years agofixed : counting matches which overlap extDict and prefix
test4973 [Thu, 12 Apr 2018 23:12:21 +0000 (16:12 -0700)]
fixed : counting matches which overlap extDict and prefix

6 years agomodified a few traces for debug
test4973 [Thu, 12 Apr 2018 14:25:40 +0000 (07:25 -0700)]
modified a few traces for debug

6 years agoMerge pull request #496 from lz4/circleci
Yann Collet [Thu, 12 Apr 2018 20:23:51 +0000 (13:23 -0700)]
Merge pull request #496 from lz4/circleci

Reduced LZ4 test time on circle-ci

6 years agomodified versionsTest
Yann Collet [Thu, 12 Apr 2018 13:47:27 +0000 (06:47 -0700)]
modified versionsTest

to use MOREFLAGS rather CPPFLAGS
as some older versions of LZ4 overwrite CPPFLAGS environment variable.

6 years agofixed LZ4_compress_fast_extState_fastReset()
test4973 [Wed, 11 Apr 2018 23:49:40 +0000 (16:49 -0700)]
fixed LZ4_compress_fast_extState_fastReset()

6 years agoMerge branch 'dev' into lowAddr
test4973 [Wed, 11 Apr 2018 23:45:19 +0000 (16:45 -0700)]
Merge branch 'dev' into lowAddr

6 years agoallow system-defined CPPFLAGS in /tests
Yann Collet [Wed, 11 Apr 2018 23:41:25 +0000 (16:41 -0700)]
allow system-defined CPPFLAGS in /tests

6 years agoreduced test time on circle-ci
Yann Collet [Wed, 11 Apr 2018 23:31:43 +0000 (16:31 -0700)]
reduced test time on circle-ci

6 years agoMerge pull request #492 from felixhandte/avoid-prepare-in-continue
Yann Collet [Wed, 11 Apr 2018 23:15:42 +0000 (16:15 -0700)]
Merge pull request #492 from felixhandte/avoid-prepare-in-continue

Several Changes Concerning Table Preparation in LZ4 Fast

6 years agoFix Silly Warning (const-ness in declaration has no effect on value types!)
W. Felix Handte [Wed, 11 Apr 2018 22:42:09 +0000 (18:42 -0400)]
Fix Silly Warning (const-ness in declaration has no effect on value types!)

6 years agoMinor Fixes
W. Felix Handte [Wed, 11 Apr 2018 20:55:12 +0000 (16:55 -0400)]
Minor Fixes

6 years agoAdd a LZ4_STATIC_LINKING_ONLY Macro to Guard Experimental APIs
W. Felix Handte [Wed, 11 Apr 2018 20:31:52 +0000 (16:31 -0400)]
Add a LZ4_STATIC_LINKING_ONLY Macro to Guard Experimental APIs

6 years agoExpose dictCtx Functionality in LZ4
W. Felix Handte [Wed, 11 Apr 2018 20:04:24 +0000 (16:04 -0400)]
Expose dictCtx Functionality in LZ4

6 years agoRename _extState_noReset -> _extState_fastReset and Edit Comments
W. Felix Handte [Wed, 11 Apr 2018 19:13:01 +0000 (15:13 -0400)]
Rename _extState_noReset -> _extState_fastReset and Edit Comments

6 years agoRemove Extraneous Assignment (clearedTable == 0)
W. Felix Handte [Wed, 11 Apr 2018 19:12:34 +0000 (15:12 -0400)]
Remove Extraneous Assignment (clearedTable == 0)

6 years agoExpose a Faster Stream Reset Function
W. Felix Handte [Tue, 10 Apr 2018 17:12:30 +0000 (13:12 -0400)]
Expose a Faster Stream Reset Function

6 years agofix minor conversion warning
test4973 [Tue, 10 Apr 2018 03:38:00 +0000 (20:38 -0700)]
fix minor conversion warning

cast from void not implicit for C++

6 years agofixed minor conversion warning
test4973 [Tue, 10 Apr 2018 00:08:17 +0000 (17:08 -0700)]
fixed minor conversion warning

ptr diff -> U32

6 years agoMerge branch 'dev' into lowAddr
test4973 [Mon, 9 Apr 2018 23:23:39 +0000 (16:23 -0700)]
Merge branch 'dev' into lowAddr

6 years agoAvoid Calling LZ4_prepareTable() in LZ4_compress_fast_continue()
W. Felix Handte [Fri, 6 Apr 2018 20:52:29 +0000 (16:52 -0400)]
Avoid Calling LZ4_prepareTable() in LZ4_compress_fast_continue()

6 years agoMerge pull request #494 from felixhandte/kill-goto
Yann Collet [Sat, 7 Apr 2018 00:35:45 +0000 (17:35 -0700)]
Merge pull request #494 from felixhandte/kill-goto

Return to Allowing Early Returns in LZ4_compress_generic()

6 years agoReturn to Allowing Early Returns in LZ4_compress_generic()
W. Felix Handte [Fri, 6 Apr 2018 22:52:55 +0000 (18:52 -0400)]
Return to Allowing Early Returns in LZ4_compress_generic()

Or: `goto` Considered Harmful

Or: https://xkcd.com/292/

6 years agoMerge pull request #493 from lz4/statusLine
Yann Collet [Fri, 6 Apr 2018 22:33:28 +0000 (15:33 -0700)]
Merge pull request #493 from lz4/statusLine

fixed DISPLAYUPDATE()

6 years agofixed DISPLAYUPDATE()
Yann Collet [Fri, 6 Apr 2018 21:16:23 +0000 (14:16 -0700)]
fixed DISPLAYUPDATE()

wrong comparison, which was always overflowing (hence was always true)
except when it was not (i386, reported by pmc)
in which case it would never show any information.

6 years agonoticed a bug when re-using hash table
test4973 [Fri, 6 Apr 2018 02:05:49 +0000 (19:05 -0700)]
noticed a bug when re-using hash table

./fuzzer -vv -s4217 -t7518

6 years agoadded low-memory address test to travis
test4973 [Fri, 6 Apr 2018 01:39:22 +0000 (18:39 -0700)]
added low-memory address test to travis

requires modification linux configuration (sudo)

6 years agofixed byPtr mode
test4973 [Fri, 6 Apr 2018 01:29:42 +0000 (18:29 -0700)]
fixed byPtr mode

switch to byU32 when src address is < 64K

note : byPtr is still useful in 32-bits, as it's about ~10% faster

6 years agofixed byPtr match search
test4973 [Fri, 6 Apr 2018 00:52:54 +0000 (17:52 -0700)]
fixed byPtr match search

6 years agofixed immediate match search
test4973 [Fri, 6 Apr 2018 00:16:33 +0000 (17:16 -0700)]
fixed immediate match search

6 years agochanged LZ4_compress_generic() logic
test4973 [Thu, 5 Apr 2018 23:38:43 +0000 (16:38 -0700)]
changed LZ4_compress_generic() logic

to use indexes (U32) instead of Ptr.

byPtr is still present.

6 years agofixed lz4 compression starting at small address
test4973 [Thu, 5 Apr 2018 19:40:33 +0000 (12:40 -0700)]
fixed lz4 compression starting at small address

when using byU32 and byU16 modes

6 years agoMerge branch 'dev' into lowAddr
test4973 [Wed, 4 Apr 2018 18:38:55 +0000 (11:38 -0700)]
Merge branch 'dev' into lowAddr

6 years agoMerge pull request #490 from kenjichanhkg/dev
Yann Collet [Mon, 2 Apr 2018 03:33:42 +0000 (20:33 -0700)]
Merge pull request #490 from kenjichanhkg/dev

added vs2017 projects

6 years agoadded vs2017 projects
Kenji Chan [Mon, 2 Apr 2018 02:52:45 +0000 (10:52 +0800)]
added vs2017 projects

6 years agoMerge pull request #486 from felixhandte/fix-test-makefile-clean-up
Yann Collet [Wed, 21 Mar 2018 21:53:02 +0000 (14:53 -0700)]
Merge pull request #486 from felixhandte/fix-test-makefile-clean-up

Add Dependency to Fix Parallel `make test` Runs

6 years agoMerge pull request #487 from felixhandte/better-obsoletion-comment
Yann Collet [Wed, 21 Mar 2018 21:52:53 +0000 (14:52 -0700)]
Merge pull request #487 from felixhandte/better-obsoletion-comment

Better Describe Functionality of Obsolete Streaming Functions

6 years agoAlso Fix a Comment
W. Felix Handte [Wed, 21 Mar 2018 15:48:35 +0000 (11:48 -0400)]
Also Fix a Comment

6 years agoBetter Describe Functionality of Obsolete Streaming Functions
W. Felix Handte [Wed, 21 Mar 2018 15:39:41 +0000 (11:39 -0400)]
Better Describe Functionality of Obsolete Streaming Functions

6 years agoAdd Dependency to Fix Parallel `make test` Runs
W. Felix Handte [Wed, 21 Mar 2018 15:28:51 +0000 (11:28 -0400)]
Add Dependency to Fix Parallel `make test` Runs

When run with `-jN`, the `rm tmp*` can run in the middle of the `test-lz4-dict`
job, which will then fail, finding its files to have been axed. This adds a
dependency between the two.

6 years agoadded c90 test to c_standards
Yann Collet [Wed, 21 Mar 2018 14:19:48 +0000 (07:19 -0700)]
added c90 test to c_standards

to catch `//` comments

6 years agoadded low address fuzzer tests
test4973 [Wed, 21 Mar 2018 14:14:13 +0000 (07:14 -0700)]
added low address fuzzer tests

6 years agofix comment style
Yann Collet [Wed, 21 Mar 2018 14:07:24 +0000 (07:07 -0700)]
fix comment style

6 years agobench: introduced hidden command -S
Yann Collet [Tue, 20 Mar 2018 00:19:25 +0000 (17:19 -0700)]
bench: introduced hidden command -S

to benchmark multiple files with separate results

6 years agoMerge branch 'dev' of github.com:Cyan4973/lz4 into dev
Yann Collet [Mon, 19 Mar 2018 23:18:25 +0000 (16:18 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev

6 years agominor man fix on clevels
Yann Collet [Mon, 19 Mar 2018 23:18:10 +0000 (16:18 -0700)]
minor man fix on clevels

6 years agoMerge pull request #484 from lz4/fasterDict
Yann Collet [Mon, 19 Mar 2018 17:06:33 +0000 (10:06 -0700)]
Merge pull request #484 from lz4/fasterDict

Faster dictionary compression

6 years agoMerge pull request #406 from felixhandte/ref-dict-table
Yann Collet [Mon, 19 Mar 2018 02:07:55 +0000 (19:07 -0700)]
Merge pull request #406 from felixhandte/ref-dict-table

Use the Dictionary Hash Table in Place

6 years agoRemove Framebench Tool
W. Felix Handte [Wed, 31 Jan 2018 23:11:37 +0000 (18:11 -0500)]
Remove Framebench Tool

6 years agoMove LZ4_compress_fast_extState_noReset Declaration to Unstable Section
W. Felix Handte [Wed, 14 Mar 2018 19:58:38 +0000 (15:58 -0400)]
Move LZ4_compress_fast_extState_noReset Declaration to Unstable Section

6 years agoRestore the Other Old Streaming Functions in a Degraded Fashion
W. Felix Handte [Wed, 14 Mar 2018 19:51:59 +0000 (15:51 -0400)]
Restore the Other Old Streaming Functions in a Degraded Fashion

6 years agoSwitch ALLOC() to ALLOC_AND_ZERO() to Paper Over Existing Uninitialized Read
W. Felix Handte [Tue, 13 Mar 2018 21:47:34 +0000 (17:47 -0400)]
Switch ALLOC() to ALLOC_AND_ZERO() to Paper Over Existing Uninitialized Read

6 years agoSplit lz4CtxLevel into Two Fields
W. Felix Handte [Tue, 13 Mar 2018 21:45:09 +0000 (17:45 -0400)]
Split lz4CtxLevel into Two Fields

6 years agoAnother Allocation Fail Check
W. Felix Handte [Tue, 13 Mar 2018 21:35:44 +0000 (17:35 -0400)]
Another Allocation Fail Check

6 years agoRestore LZ4_sizeofStreamState, We Didn't Actually Need to Delete It
W. Felix Handte [Tue, 13 Mar 2018 19:42:03 +0000 (15:42 -0400)]
Restore LZ4_sizeofStreamState, We Didn't Actually Need to Delete It

6 years agoRestore checkTag Cleaning
W. Felix Handte [Tue, 13 Mar 2018 19:18:08 +0000 (15:18 -0400)]
Restore checkTag Cleaning

6 years agoRename Enums and Add Comment
W. Felix Handte [Tue, 13 Mar 2018 19:07:19 +0000 (15:07 -0400)]
Rename Enums and Add Comment

6 years agoWhitespace Fixes
W. Felix Handte [Mon, 12 Mar 2018 22:46:54 +0000 (18:46 -0400)]
Whitespace Fixes

6 years agoAdd NULL Checks
W. Felix Handte [Mon, 12 Mar 2018 22:32:24 +0000 (18:32 -0400)]
Add NULL Checks

6 years agoSimpler Ternary Statements
W. Felix Handte [Fri, 9 Mar 2018 17:05:31 +0000 (12:05 -0500)]
Simpler Ternary Statements

6 years agoRenames and Comment Fixes
W. Felix Handte [Mon, 12 Mar 2018 22:13:24 +0000 (18:13 -0400)]
Renames and Comment Fixes

6 years agoHoist LZ4F Dictionary Setup into Helper LZ4F_applyCDict()
W. Felix Handte [Mon, 12 Mar 2018 20:11:55 +0000 (16:11 -0400)]
Hoist LZ4F Dictionary Setup into Helper LZ4F_applyCDict()

6 years agoMinor Style Fixes
W. Felix Handte [Mon, 12 Mar 2018 20:11:44 +0000 (16:11 -0400)]
Minor Style Fixes

6 years agoPreserve currentOffset==0 When Possible
W. Felix Handte [Thu, 8 Mar 2018 19:09:06 +0000 (14:09 -0500)]
Preserve currentOffset==0 When Possible

6 years agoSpecialize _extState() for Clean Ctx Rather Than Calling _safeExtState()
W. Felix Handte [Fri, 9 Mar 2018 17:14:42 +0000 (12:14 -0500)]
Specialize _extState() for Clean Ctx Rather Than Calling _safeExtState()

6 years agoRemove Switch In Favor of Ternary Statement
W. Felix Handte [Thu, 8 Mar 2018 17:30:34 +0000 (12:30 -0500)]
Remove Switch In Favor of Ternary Statement

6 years agoFurther Avoid a dictionary==NULL Check
W. Felix Handte [Thu, 8 Mar 2018 17:29:45 +0000 (12:29 -0500)]
Further Avoid a dictionary==NULL Check