Yann Collet [Tue, 24 Apr 2018 18:25:57 +0000 (11:25 -0700)]
Merge pull request #511 from lz4/decFast
Fixed performance issue with LZ4_decompress_fast()
Yann Collet [Tue, 24 Apr 2018 06:44:04 +0000 (23:44 -0700)]
Merge pull request #504 from baruchsiach/static-only-support
lib: allow to disable shared libraries
Cyan4973 [Tue, 24 Apr 2018 02:26:02 +0000 (19:26 -0700)]
re-ordered parenthesis
to avoid mixing && and &
as suggested by @terrelln
Yann Collet [Mon, 23 Apr 2018 22:55:56 +0000 (15:55 -0700)]
Merge pull request #507 from lz4/clangPerf
fixed lz4_fast clang performance
Cyan4973 [Mon, 23 Apr 2018 22:52:44 +0000 (15:52 -0700)]
fixed minor declaration issue with clang on msys
Cyan4973 [Mon, 23 Apr 2018 22:47:08 +0000 (15:47 -0700)]
disable shortcut for LZ4_decompress_fast()
improving speed
Cyan4973 [Mon, 23 Apr 2018 22:42:27 +0000 (15:42 -0700)]
fullbench compiled without assert()
to better reflect release speed
Yann Collet [Mon, 23 Apr 2018 22:28:19 +0000 (15:28 -0700)]
Merge pull request #510 from terrelln/bug-fix
Fix input size validation edge cases
Nick Terrell [Mon, 23 Apr 2018 21:21:02 +0000 (14:21 -0700)]
Fix compilation error and assert.
Nick Terrell [Mon, 23 Apr 2018 20:14:19 +0000 (13:14 -0700)]
Fix input size validation edge cases
The bug is a read up to 2 bytes past the end of the buffer.
There are three cases for this bug, one for each test case added.
* An empty input causes `token = *ip++` to read one byte too far.
* A one byte input with `(token >> ML_BITS) == RUN_MASK` causes
one extra byte to be read without validation. This could be
combined with the first bug to cause 2 extra bytes to be read.
* The case pointed out in issue #508, where `ip == iend` at the
beginning of the loop after taking the shortcut.
Benchmarks show no regressions on clang or gcc-7 on both my mac
and devserver.
Fixes #508.
Yann Collet [Mon, 23 Apr 2018 02:30:24 +0000 (19:30 -0700)]
Merge pull request #509 from svpv/clarifyFastRisks
lz4.h: clarify the risks of using LZ4_decompress_fast()
Alexey Tourbin [Sun, 22 Apr 2018 22:43:30 +0000 (01:43 +0300)]
lz4.h: clarify the risks of using LZ4_decompress_fast()
The notes about "security guarantee" and "malicious inputs" seemed
a bit non-technical to me, so I took the liberty to tone them down
and instead describe the actual risks in technical terms. Namely,
the function never writes past the end of the output buffer, so
a direct hostile takeover (resulting in arbitrary code execution
soon after the return from the function) is not possible. However,
the application can crash because of reads from unmapped pages.
I also took the liberty to describe what I believe is the only sensible
usage scenario for the function: "This function is only usable if the
originalSize of uncompressed data is known in advance," etc.
Cyan4973 [Sat, 21 Apr 2018 07:11:51 +0000 (00:11 -0700)]
fixed incorrect comment
Yann Collet [Sat, 21 Apr 2018 01:09:51 +0000 (18:09 -0700)]
fixed clang performance in lz4_fast
The simple change from
`matchIndex+MAX_DISTANCE < current`
towards
`current - matchIndex > MAX_DISTANCE`
is enough to generate a 10% performance drop under clang.
Quite massive.
(I missed as my eyes were concentrated on gcc performance at that time).
The second version is more robust, because it also survives a situation where
`matchIndex > current`
due to overflows.
The first version requires matchIndex to not overflow.
Hence were added `assert()` conditions.
The only case where this can happen is with dictCtx compression,
in the case where the dictionary context is not initialized before loading the dictionary.
So it's enough to always initialize the context while loading the dictionary.
Yann Collet [Thu, 19 Apr 2018 18:50:20 +0000 (11:50 -0700)]
Merge pull request #503 from lz4/l120
minor length reduction of several large lines
Yann Collet [Thu, 19 Apr 2018 17:52:48 +0000 (10:52 -0700)]
Merge pull request #502 from lhacc1/dev
Wrap likely/unlikely macroses with #ifndef
Yann Collet [Thu, 19 Apr 2018 17:50:40 +0000 (10:50 -0700)]
modified indentation for consistency
Baruch Siach [Thu, 19 Apr 2018 09:28:11 +0000 (12:28 +0300)]
lib: allow to disable shared libraries
Just like BUILD_STATIC=no disables static libraries, BUILD_SHARED=no
disabled shared libraries. This is useful to support toolchains that do
not support shared libraries.
Yann Collet [Wed, 18 Apr 2018 23:49:27 +0000 (16:49 -0700)]
minor length reduction of several large lines
Yann Collet [Wed, 18 Apr 2018 17:16:25 +0000 (10:16 -0700)]
Merge pull request #497 from lz4/lowAddr
Compatibility with low memory addresses
Dmitrii Rodionov [Wed, 18 Apr 2018 09:20:56 +0000 (12:20 +0300)]
Wrap likely/unlikely macroses with #ifndef
It prevent redefine error when project using lz4 has its own likely/unlikely
macroses.
Yann Collet [Tue, 17 Apr 2018 23:47:56 +0000 (16:47 -0700)]
fixed LZ4_compress_fast_extState_fastReset() in 32-bit mode
Yann Collet [Tue, 17 Apr 2018 23:18:37 +0000 (16:18 -0700)]
fix dictDelta setting error
wrong test
Yann Collet [Tue, 17 Apr 2018 22:29:17 +0000 (15:29 -0700)]
fix matchIndex overflow
can happen with dictCtx
Yann Collet [Tue, 17 Apr 2018 19:07:22 +0000 (12:07 -0700)]
Merge branch 'dev' into lowAddr
Yann Collet [Tue, 17 Apr 2018 19:06:44 +0000 (12:06 -0700)]
Merge pull request #501 from felixhandte/fix-dict-load-offset
Always Bump Offset by 64 KB in LZ4_loadDict()
W. Felix Handte [Tue, 17 Apr 2018 18:01:44 +0000 (14:01 -0400)]
Always Bump Offset by 64 KB in LZ4_loadDict()
This actually ensures the guarantee referred to in the comment in
LZ4_compress_fast_continue().
Yann Collet [Tue, 17 Apr 2018 06:59:42 +0000 (23:59 -0700)]
fixed dictCtx compression
Yann Collet [Tue, 17 Apr 2018 00:15:02 +0000 (17:15 -0700)]
edited a few traces for debugging
Yann Collet [Mon, 16 Apr 2018 23:54:03 +0000 (16:54 -0700)]
fixed minor format warnings
Yann Collet [Mon, 16 Apr 2018 23:14:28 +0000 (16:14 -0700)]
fixed fuzzer tests
which were modified in parallel within branc `dev`
Yann Collet [Mon, 16 Apr 2018 23:12:38 +0000 (16:12 -0700)]
Merge branch 'dev' into lowAddr
Yann Collet [Mon, 16 Apr 2018 22:11:28 +0000 (15:11 -0700)]
fixed gcc performance regression
Yann Collet [Fri, 13 Apr 2018 20:22:38 +0000 (13:22 -0700)]
Merge pull request #499 from felixhandte/lz4-attach-dict-tests
Test LZ4_attach_dictionary() and Friends
W. Felix Handte [Thu, 12 Apr 2018 23:17:53 +0000 (19:17 -0400)]
Further Test that ExtDictCtx Mode Produces the Exact Same Output
W. Felix Handte [Thu, 12 Apr 2018 22:23:01 +0000 (18:23 -0400)]
Add Tests for LZ4_attach_dictionary and Friends
Yann Collet [Fri, 13 Apr 2018 09:45:32 +0000 (02:45 -0700)]
fixed minor unused variable warning
Yann Collet [Fri, 13 Apr 2018 09:26:14 +0000 (02:26 -0700)]
added comment on variables required after _next_match
Yann Collet [Fri, 13 Apr 2018 09:10:53 +0000 (02:10 -0700)]
fixed potential ptrdiff_t overflow (32-bits mode)
Also removed pointer comparison, which should solve #485
Cyan4973 [Fri, 13 Apr 2018 08:01:54 +0000 (01:01 -0700)]
compatibility with gcc-4.4 string.h version
Someone found it would be a great idea to define there a global variable under the very generic name "index".
Cause problem with shadow warnings, so no variable can be named "index" now ...
Also : automatically update API manual
Cyan4973 [Fri, 13 Apr 2018 07:59:27 +0000 (00:59 -0700)]
added sudo rights for low-mem-address tests
test4973 [Thu, 12 Apr 2018 23:12:21 +0000 (16:12 -0700)]
fixed : counting matches which overlap extDict and prefix
test4973 [Thu, 12 Apr 2018 14:25:40 +0000 (07:25 -0700)]
modified a few traces for debug
Yann Collet [Thu, 12 Apr 2018 20:23:51 +0000 (13:23 -0700)]
Merge pull request #496 from lz4/circleci
Reduced LZ4 test time on circle-ci
Yann Collet [Thu, 12 Apr 2018 13:47:27 +0000 (06:47 -0700)]
modified versionsTest
to use MOREFLAGS rather CPPFLAGS
as some older versions of LZ4 overwrite CPPFLAGS environment variable.
test4973 [Wed, 11 Apr 2018 23:49:40 +0000 (16:49 -0700)]
fixed LZ4_compress_fast_extState_fastReset()
test4973 [Wed, 11 Apr 2018 23:45:19 +0000 (16:45 -0700)]
Merge branch 'dev' into lowAddr
Yann Collet [Wed, 11 Apr 2018 23:41:25 +0000 (16:41 -0700)]
allow system-defined CPPFLAGS in /tests
Yann Collet [Wed, 11 Apr 2018 23:31:43 +0000 (16:31 -0700)]
reduced test time on circle-ci
Yann Collet [Wed, 11 Apr 2018 23:15:42 +0000 (16:15 -0700)]
Merge pull request #492 from felixhandte/avoid-prepare-in-continue
Several Changes Concerning Table Preparation in LZ4 Fast
W. Felix Handte [Wed, 11 Apr 2018 22:42:09 +0000 (18:42 -0400)]
Fix Silly Warning (const-ness in declaration has no effect on value types!)
W. Felix Handte [Wed, 11 Apr 2018 20:55:12 +0000 (16:55 -0400)]
Minor Fixes
W. Felix Handte [Wed, 11 Apr 2018 20:31:52 +0000 (16:31 -0400)]
Add a LZ4_STATIC_LINKING_ONLY Macro to Guard Experimental APIs
W. Felix Handte [Wed, 11 Apr 2018 20:04:24 +0000 (16:04 -0400)]
Expose dictCtx Functionality in LZ4
W. Felix Handte [Wed, 11 Apr 2018 19:13:01 +0000 (15:13 -0400)]
Rename _extState_noReset -> _extState_fastReset and Edit Comments
W. Felix Handte [Wed, 11 Apr 2018 19:12:34 +0000 (15:12 -0400)]
Remove Extraneous Assignment (clearedTable == 0)
W. Felix Handte [Tue, 10 Apr 2018 17:12:30 +0000 (13:12 -0400)]
Expose a Faster Stream Reset Function
test4973 [Tue, 10 Apr 2018 03:38:00 +0000 (20:38 -0700)]
fix minor conversion warning
cast from void not implicit for C++
test4973 [Tue, 10 Apr 2018 00:08:17 +0000 (17:08 -0700)]
fixed minor conversion warning
ptr diff -> U32
test4973 [Mon, 9 Apr 2018 23:23:39 +0000 (16:23 -0700)]
Merge branch 'dev' into lowAddr
W. Felix Handte [Fri, 6 Apr 2018 20:52:29 +0000 (16:52 -0400)]
Avoid Calling LZ4_prepareTable() in LZ4_compress_fast_continue()
Yann Collet [Sat, 7 Apr 2018 00:35:45 +0000 (17:35 -0700)]
Merge pull request #494 from felixhandte/kill-goto
Return to Allowing Early Returns in LZ4_compress_generic()
W. Felix Handte [Fri, 6 Apr 2018 22:52:55 +0000 (18:52 -0400)]
Return to Allowing Early Returns in LZ4_compress_generic()
Or: `goto` Considered Harmful
Or: https://xkcd.com/292/
Yann Collet [Fri, 6 Apr 2018 22:33:28 +0000 (15:33 -0700)]
Merge pull request #493 from lz4/statusLine
fixed DISPLAYUPDATE()
Yann Collet [Fri, 6 Apr 2018 21:16:23 +0000 (14:16 -0700)]
fixed DISPLAYUPDATE()
wrong comparison, which was always overflowing (hence was always true)
except when it was not (i386, reported by pmc)
in which case it would never show any information.
test4973 [Fri, 6 Apr 2018 02:05:49 +0000 (19:05 -0700)]
noticed a bug when re-using hash table
./fuzzer -vv -s4217 -t7518
test4973 [Fri, 6 Apr 2018 01:39:22 +0000 (18:39 -0700)]
added low-memory address test to travis
requires modification linux configuration (sudo)
test4973 [Fri, 6 Apr 2018 01:29:42 +0000 (18:29 -0700)]
fixed byPtr mode
switch to byU32 when src address is < 64K
note : byPtr is still useful in 32-bits, as it's about ~10% faster
test4973 [Fri, 6 Apr 2018 00:52:54 +0000 (17:52 -0700)]
fixed byPtr match search
test4973 [Fri, 6 Apr 2018 00:16:33 +0000 (17:16 -0700)]
fixed immediate match search
test4973 [Thu, 5 Apr 2018 23:38:43 +0000 (16:38 -0700)]
changed LZ4_compress_generic() logic
to use indexes (U32) instead of Ptr.
byPtr is still present.
test4973 [Thu, 5 Apr 2018 19:40:33 +0000 (12:40 -0700)]
fixed lz4 compression starting at small address
when using byU32 and byU16 modes
test4973 [Wed, 4 Apr 2018 18:38:55 +0000 (11:38 -0700)]
Merge branch 'dev' into lowAddr
Yann Collet [Mon, 2 Apr 2018 03:33:42 +0000 (20:33 -0700)]
Merge pull request #490 from kenjichanhkg/dev
added vs2017 projects
Kenji Chan [Mon, 2 Apr 2018 02:52:45 +0000 (10:52 +0800)]
added vs2017 projects
Yann Collet [Wed, 21 Mar 2018 21:53:02 +0000 (14:53 -0700)]
Merge pull request #486 from felixhandte/fix-test-makefile-clean-up
Add Dependency to Fix Parallel `make test` Runs
Yann Collet [Wed, 21 Mar 2018 21:52:53 +0000 (14:52 -0700)]
Merge pull request #487 from felixhandte/better-obsoletion-comment
Better Describe Functionality of Obsolete Streaming Functions
W. Felix Handte [Wed, 21 Mar 2018 15:48:35 +0000 (11:48 -0400)]
Also Fix a Comment
W. Felix Handte [Wed, 21 Mar 2018 15:39:41 +0000 (11:39 -0400)]
Better Describe Functionality of Obsolete Streaming Functions
W. Felix Handte [Wed, 21 Mar 2018 15:28:51 +0000 (11:28 -0400)]
Add Dependency to Fix Parallel `make test` Runs
When run with `-jN`, the `rm tmp*` can run in the middle of the `test-lz4-dict`
job, which will then fail, finding its files to have been axed. This adds a
dependency between the two.
Yann Collet [Wed, 21 Mar 2018 14:19:48 +0000 (07:19 -0700)]
added c90 test to c_standards
to catch `//` comments
test4973 [Wed, 21 Mar 2018 14:14:13 +0000 (07:14 -0700)]
added low address fuzzer tests
Yann Collet [Wed, 21 Mar 2018 14:07:24 +0000 (07:07 -0700)]
fix comment style
Yann Collet [Tue, 20 Mar 2018 00:19:25 +0000 (17:19 -0700)]
bench: introduced hidden command -S
to benchmark multiple files with separate results
Yann Collet [Mon, 19 Mar 2018 23:18:25 +0000 (16:18 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
Yann Collet [Mon, 19 Mar 2018 23:18:10 +0000 (16:18 -0700)]
minor man fix on clevels
Yann Collet [Mon, 19 Mar 2018 17:06:33 +0000 (10:06 -0700)]
Merge pull request #484 from lz4/fasterDict
Faster dictionary compression
Yann Collet [Mon, 19 Mar 2018 02:07:55 +0000 (19:07 -0700)]
Merge pull request #406 from felixhandte/ref-dict-table
Use the Dictionary Hash Table in Place
W. Felix Handte [Wed, 31 Jan 2018 23:11:37 +0000 (18:11 -0500)]
Remove Framebench Tool
W. Felix Handte [Wed, 14 Mar 2018 19:58:38 +0000 (15:58 -0400)]
Move LZ4_compress_fast_extState_noReset Declaration to Unstable Section
W. Felix Handte [Wed, 14 Mar 2018 19:51:59 +0000 (15:51 -0400)]
Restore the Other Old Streaming Functions in a Degraded Fashion
W. Felix Handte [Tue, 13 Mar 2018 21:47:34 +0000 (17:47 -0400)]
Switch ALLOC() to ALLOC_AND_ZERO() to Paper Over Existing Uninitialized Read
W. Felix Handte [Tue, 13 Mar 2018 21:45:09 +0000 (17:45 -0400)]
Split lz4CtxLevel into Two Fields
W. Felix Handte [Tue, 13 Mar 2018 21:35:44 +0000 (17:35 -0400)]
Another Allocation Fail Check
W. Felix Handte [Tue, 13 Mar 2018 19:42:03 +0000 (15:42 -0400)]
Restore LZ4_sizeofStreamState, We Didn't Actually Need to Delete It
W. Felix Handte [Tue, 13 Mar 2018 19:18:08 +0000 (15:18 -0400)]
Restore checkTag Cleaning
W. Felix Handte [Tue, 13 Mar 2018 19:07:19 +0000 (15:07 -0400)]
Rename Enums and Add Comment
W. Felix Handte [Mon, 12 Mar 2018 22:46:54 +0000 (18:46 -0400)]
Whitespace Fixes
W. Felix Handte [Mon, 12 Mar 2018 22:32:24 +0000 (18:32 -0400)]
Add NULL Checks
W. Felix Handte [Fri, 9 Mar 2018 17:05:31 +0000 (12:05 -0500)]
Simpler Ternary Statements