Yann Collet [Fri, 31 May 2019 19:23:52 +0000 (12:23 -0700)]
Merge pull request #719 from parheliamm/dev
FAST_DEC_LOOP: move offset check in specific condition.
Chenxi Mao [Fri, 31 May 2019 00:36:13 +0000 (08:36 +0800)]
FAST_DEC_LOOP: only did offset check in specific condition.
When I did FAST_DEC_LOOP performance test, I found the
offset check is much more than v1.8.3
You will see the condition check difference via lzbench with dickens test case.
v1.8.3 34959
v.1.9.x 1055885
After investigate the code, we could see the difference.
v.1.8.3 SKIP the condition check if
if condition is true in:
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1463
AND below condition is true
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1478\
The offset check should be invoked.
v1.9.3
The offset check code will be invoked in every loop which lead to downgrade.
So the fix would be move this check to specific condition
to avoid useless condition check.
After this change, the call number is same as v1.8.3
Yann Collet [Fri, 31 May 2019 00:28:59 +0000 (17:28 -0700)]
Merge pull request #714 from dreambottle/fix-lz4f-input-buf-use
Fix src temporary buffer use in lz4frame
Yann Collet [Mon, 27 May 2019 18:59:23 +0000 (11:59 -0700)]
Merge pull request #715 from gabrielstedman/ret
Handle file not existing case #704
gabrielstedman [Sat, 25 May 2019 18:57:04 +0000 (19:57 +0100)]
Handle file not existing case #704
Niko Dzhus [Fri, 24 May 2019 15:25:06 +0000 (18:25 +0300)]
fix temporary buffer use when input size hint is respected
Yann Collet [Fri, 17 May 2019 19:22:39 +0000 (12:22 -0700)]
Merge pull request #709 from lz4/circle
updated tests for CircleCI
Yann Collet [Thu, 16 May 2019 23:46:16 +0000 (16:46 -0700)]
updated tests
- only play listTest with `make test`, not `make all` which is limited to build
- update `clangtest`, so that it's possible to disable O3 optimization, for faster processing
Yann Collet [Thu, 16 May 2019 22:56:42 +0000 (15:56 -0700)]
Merge pull request #708 from gabrielstedman/list
Add multiframe report to --list command
gstedman [Thu, 16 May 2019 14:12:00 +0000 (15:12 +0100)]
List mode improvements.
Improve formatting
Include static assert
Use UTIL_fseek to handle potential fseek limitation
Be explicit when refusing to read from stdin
Properly free dctx after use
Include valgrind tests
gstedman [Fri, 10 May 2019 15:54:05 +0000 (16:54 +0100)]
Add multiframe report to --list command
Yann Collet [Tue, 14 May 2019 03:35:10 +0000 (20:35 -0700)]
Merge pull request #707 from prekageo/dev
enable LZ4_FAST_DEC_LOOP build macro on aarch64 by default
George Prekas [Fri, 3 May 2019 16:44:56 +0000 (11:44 -0500)]
enable LZ4_FAST_DEC_LOOP build macro on aarch64/GCC by default
Yann Collet [Tue, 30 Apr 2019 06:40:42 +0000 (23:40 -0700)]
Merge pull request #700 from lzutao/meson-non-failures
contrib: Make Meson build non optional
Lzu Tao [Tue, 30 Apr 2019 05:25:40 +0000 (12:25 +0700)]
meson: Rely only on extracted version in lz4.h
So now instead of warning when failing to extract version number
from lz4.h, we error and stop the build instead.
Lzu Tao [Fri, 26 Apr 2019 18:41:03 +0000 (01:41 +0700)]
contrib: Make Meson build non optional
* Update ninja version to 1.9 in CI.
* Update default project version in meson script.
Yann Collet [Fri, 26 Apr 2019 03:26:04 +0000 (20:26 -0700)]
Merge pull request #699 from brendene/compressBound
Include block checksum in worst case scenario calculation of dstCapacity
Brenden Eng [Fri, 26 Apr 2019 02:37:39 +0000 (22:37 -0400)]
Include block checksum in worst case scenario calculation of dstCapacity
Yann Collet [Wed, 24 Apr 2019 18:50:50 +0000 (11:50 -0700)]
Merge pull request #698 from lz4/tidy
several minor style changes recommended by clang-tidy
Yann Collet [Wed, 24 Apr 2019 17:03:02 +0000 (10:03 -0700)]
several minor style changes recommended by clang-tidy
Yann Collet [Wed, 24 Apr 2019 00:20:03 +0000 (17:20 -0700)]
Merge pull request #697 from ldv-alt/fixes/test-amalgamation
Fix test-amalgamation
Yann Collet [Tue, 23 Apr 2019 22:08:46 +0000 (15:08 -0700)]
Merge pull request #696 from lz4/mactest
Fix dynamic library compilation on Mac OS-X
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: split the rule
Change test-amalgamation to follow each-rule-makes-a-single-target idiom.
Fixes:
a7e8d394 ("[amalgamation] add test")
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: fix compilation options
Use the same compilation options to compile lz4_all.c and other object files.
Fixes:
a7e8d394 ("[amalgamation] add test")
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: use a single cat command
Use the list of prerequisites instead of listing those files manually,
this way they will never fall out of sync.
Also update the amalgamation example to use a single cat command.
Fixes:
a7e8d394 ("[amalgamation] add test")
Fixes:
b192c86b ("[amalgamation] lz4frame.c")
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: fix the list of prerequisites
Add $(LZ4DIR)/lz4frame.c to the list of prerequisites as the rule uses
that file.
Fixes:
b192c86b ("[amalgamation] lz4frame.c")
Yann Collet [Tue, 23 Apr 2019 21:09:40 +0000 (14:09 -0700)]
Merge pull request #695 from lembacon/makefile-os
lib/Makefile: Fix detection of `Darwin`.
Yann Collet [Tue, 23 Apr 2019 19:29:39 +0000 (12:29 -0700)]
added library build test on Mac OS-X
Chongyu Zhu [Tue, 23 Apr 2019 19:13:19 +0000 (03:13 +0800)]
lib/Makefile: Fix detection of `Darwin`.
Yann Collet [Tue, 23 Apr 2019 16:20:41 +0000 (09:20 -0700)]
Merge pull request #693 from JPeterMugaas/master
More build imrpvements
JPeterMugaas [Tue, 23 Apr 2019 11:44:00 +0000 (07:44 -0400)]
More build imrpvements
Moved a few other tests to Makefiles.inc. Other things might need to go there.
Made a test for symlink appropriateness. Windows can NOT handle them the same way Unix-like operating systems do (if at all). This is mostly the same as the Visual C projects.
embed version info into .dll and .exes that are redistributed.
Yann Collet [Tue, 23 Apr 2019 00:45:40 +0000 (17:45 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev
Yann Collet [Tue, 23 Apr 2019 00:45:24 +0000 (17:45 -0700)]
updated NEWS for v1.9.1
Yann Collet [Tue, 23 Apr 2019 00:43:01 +0000 (17:43 -0700)]
Merge pull request #691 from lz4/list
--list
Yann Collet [Mon, 22 Apr 2019 23:06:22 +0000 (16:06 -0700)]
fix: no leak when LZ4F_dctx creation fails
strange, because it previous implementation, it would `exit()`,
so it should not matter ...
Yann Collet [Mon, 22 Apr 2019 22:29:50 +0000 (15:29 -0700)]
Merge pull request #689 from JPeterMugaas/jpm-makefile
Jpm makefile - as described in https://github.com/lz4/lz4/issues/688
Yann Collet [Mon, 22 Apr 2019 22:24:44 +0000 (15:24 -0700)]
updated code comments
Yann Collet [Mon, 22 Apr 2019 22:14:53 +0000 (15:14 -0700)]
--list gives block type
Yann Collet [Mon, 22 Apr 2019 22:01:20 +0000 (15:01 -0700)]
--list can resume after a failed file
JPeterMugaas [Mon, 22 Apr 2019 21:38:43 +0000 (17:38 -0400)]
Make programs/Makefile use the includes.
Yann Collet [Mon, 22 Apr 2019 21:01:19 +0000 (14:01 -0700)]
refactored --list function
better handling of special conditions,
better scoping of variables.
Also : updated man page
JPeterMugaas [Mon, 22 Apr 2019 20:06:04 +0000 (16:06 -0400)]
Fix a test for mingw
JPeterMugaas [Mon, 22 Apr 2019 19:48:48 +0000 (15:48 -0400)]
Try to put some tests I made in ONE place. I also moved a test for "install" in one place to try to isolate it.
Yann Collet [Mon, 22 Apr 2019 19:27:25 +0000 (12:27 -0700)]
fixed C90 compliance
re-structure code, have everything into a single section of lz4io.c
Yann Collet [Mon, 22 Apr 2019 19:19:28 +0000 (12:19 -0700)]
Merge pull request #690 from gabrielstedman/dev
Implement --list
JPeterMugaas [Mon, 22 Apr 2019 17:48:59 +0000 (13:48 -0400)]
Initial commits from diff I submitted earlier
gabrielstedman [Mon, 22 Apr 2019 10:07:08 +0000 (11:07 +0100)]
FR #598 - Correctly initialize cfinfo & cast malloc res to (char*)
gabrielstedman [Mon, 22 Apr 2019 09:35:29 +0000 (10:35 +0100)]
FR #598 - Make fileSize unsigned long long
gabrielstedman [Mon, 22 Apr 2019 08:58:26 +0000 (09:58 +0100)]
FR #598 - Move LZ4F_compFileInfo_t def to lz4frame.h discard output_file
assert if in om_list mode
gabrielstedman [Mon, 22 Apr 2019 08:56:51 +0000 (09:56 +0100)]
Merge https://github.com/lz4/lz4 into dev
gabrielstedman [Mon, 22 Apr 2019 08:00:20 +0000 (09:00 +0100)]
FR #598 - Make LZ4IO_getCompressedFileInfo internal and reword func
Yann Collet [Mon, 22 Apr 2019 03:20:41 +0000 (20:20 -0700)]
Merge pull request #686 from lz4/clistdout
cli: warning for stdout as default output
Yann Collet [Mon, 22 Apr 2019 00:01:03 +0000 (17:01 -0700)]
ensure tests work when `stdout` is not the console
ensure this case is continuously tested on travis.
Update documentation on implicit output,
invite to not rely on implicit output in scripts.
Yann Collet [Sun, 21 Apr 2019 19:42:13 +0000 (12:42 -0700)]
tests/Makefile : created CMP variable
for potential redirection, if need be.
note : should probably converge all comparison operations onto CMP
gabrielstedman [Sun, 21 Apr 2019 16:43:57 +0000 (17:43 +0100)]
FR #598 Improve initial design, test mallocs, support C90.
gabrielstedman [Sat, 20 Apr 2019 23:20:38 +0000 (00:20 +0100)]
correctly use unisgned int for index
gabrielstedman [Sat, 20 Apr 2019 21:06:10 +0000 (22:06 +0100)]
Add --list option to display compressed file information.
Yann Collet [Sat, 20 Apr 2019 00:08:40 +0000 (17:08 -0700)]
cli: display a warning whenever default output is stdout while input != stdin
This behavior has been preserved for compatibility with existing ecosystem.
But it's problematic, as some environment start `lz4` without identifying stdout as console by default,
leading to a change of behavior for a same line of script.
A more sensible policy would be to default to stdout only when input is stdin.
Soft change for the time being : keep the behavior, just print a warning message.
User should prefer `-c` to explicitly select `stdout`.
Also : updated tests in Makefile to explicitly select `stdout` with `-c`.
Yann Collet [Fri, 19 Apr 2019 18:54:01 +0000 (11:54 -0700)]
bumped version number
to v1.9.1
Yann Collet [Fri, 19 Apr 2019 18:51:29 +0000 (11:51 -0700)]
Merge pull request #684 from lz4/cxxlz4f
lz4frame: initializers compatibility with C++
Yann Collet [Fri, 19 Apr 2019 17:23:50 +0000 (10:23 -0700)]
lz4frame: initializers compatibility with C++
fix #679, reported by @degski
Yann Collet [Fri, 19 Apr 2019 03:57:56 +0000 (20:57 -0700)]
Merge pull request #683 from lz4/fuzasan
fixed read-after input in LZ4_decompress_safe()
Yann Collet [Fri, 19 Apr 2019 01:51:28 +0000 (18:51 -0700)]
Merge pull request #682 from lz4/visual
fix minor Visual warnings
Yann Collet [Fri, 19 Apr 2019 01:50:51 +0000 (18:50 -0700)]
fixed read-after input in LZ4_decompress_safe()
Yann Collet [Fri, 19 Apr 2019 00:26:01 +0000 (17:26 -0700)]
ensure list of names is large enough
Yann Collet [Fri, 19 Apr 2019 00:12:14 +0000 (17:12 -0700)]
fuzzer : reduced stack usage
Yann Collet [Thu, 18 Apr 2019 23:41:27 +0000 (16:41 -0700)]
fuzzer: reduce stack usage to please Visual static analyzer
Yann Collet [Thu, 18 Apr 2019 23:06:02 +0000 (16:06 -0700)]
address a few minor Visual warnings
and created target cxx17build
Yann Collet [Thu, 18 Apr 2019 21:29:20 +0000 (14:29 -0700)]
Merge pull request #678 from lz4/decFast
Fix out-of-bound read in LZ4_decompress_fast()
Yann Collet [Thu, 18 Apr 2019 19:05:50 +0000 (12:05 -0700)]
fix several minor static analyzer warnings
Yann Collet [Thu, 18 Apr 2019 17:38:51 +0000 (10:38 -0700)]
fuzzer: fixed scan-build leak warning
the program exit(),
so there is no need to track and dealloc every buffer.
Yann Collet [Wed, 17 Apr 2019 22:44:14 +0000 (15:44 -0700)]
fuzzer: fix explicit cast for C++
Yann Collet [Wed, 17 Apr 2019 22:33:37 +0000 (15:33 -0700)]
re-enable FORCE_INLINE
was disabled for tests
Yann Collet [Wed, 17 Apr 2019 22:01:53 +0000 (15:01 -0700)]
fix out-of-bound read within LZ4_decompress_fast()
and deprecate LZ4_decompress_fast(),
with deprecation warnings enabled by default.
Note that, as a consequence of the fix,
LZ4_decompress_fast is now __slower__ than LZ4_decompress_safe().
That's because, since it doesn't know the input buffer size,
it must progress more cautiously into the input buffer
to ensure to out-of-bound read.
Yann Collet [Wed, 17 Apr 2019 20:31:24 +0000 (13:31 -0700)]
fuzzer: added tests to detect LZ4_decompress_fast() out of bound read
Yann Collet [Wed, 17 Apr 2019 17:43:38 +0000 (10:43 -0700)]
Merge pull request #674 from nrgiii/dev
Fix
Norm Green [Wed, 17 Apr 2019 16:35:18 +0000 (09:35 -0700)]
Merge branch 'fuzzerAix' into dev
Norm Green [Wed, 17 Apr 2019 16:20:09 +0000 (09:20 -0700)]
Fix AIX errors/warnings
Yann Collet [Wed, 17 Apr 2019 03:46:04 +0000 (20:46 -0700)]
fixed version number for lz4hc
Yann Collet [Tue, 16 Apr 2019 20:55:28 +0000 (13:55 -0700)]
Merge pull request #673 from lz4/dev
minor update for v1.9.0
Yann Collet [Tue, 16 Apr 2019 18:52:54 +0000 (11:52 -0700)]
minor benchmark update
was using wrong memory timing
Yann Collet [Tue, 16 Apr 2019 18:26:03 +0000 (11:26 -0700)]
ensure consistent definition and usage of FREEMEM
as suggested by @sloutsky in #671
Yann Collet [Tue, 16 Apr 2019 18:20:31 +0000 (11:20 -0700)]
minor news update
for msys2
Yann Collet [Tue, 16 Apr 2019 17:07:41 +0000 (10:07 -0700)]
Merge pull request #670 from lz4/dev
v1.9.0
Yann Collet [Tue, 16 Apr 2019 00:30:31 +0000 (17:30 -0700)]
updated NEWS for v1.9.0
Yann Collet [Mon, 15 Apr 2019 22:32:03 +0000 (15:32 -0700)]
Merge pull request #669 from lz4/fixframe
Fix lz4frame inefficiency
Yann Collet [Mon, 15 Apr 2019 21:13:10 +0000 (14:13 -0700)]
simplified output_directive
Yann Collet [Mon, 15 Apr 2019 18:22:25 +0000 (11:22 -0700)]
fix comma for pedantic
Yann Collet [Mon, 15 Apr 2019 18:18:33 +0000 (11:18 -0700)]
Merge branch 'dev' into fixframe
Yann Collet [Mon, 15 Apr 2019 18:17:35 +0000 (11:17 -0700)]
added a fake initialization to please an analyzer
which is unable to understand that the variable is necessarily initialized
in spite of an assert just before.
Yann Collet [Mon, 15 Apr 2019 18:12:54 +0000 (11:12 -0700)]
fix fuzzer tests on dirty context
context is no longer dirty after a failed compressed block.
Actually, all indexes are fine.
It remains compatible with continued streaming, and reset*_fast().
Yann Collet [Mon, 15 Apr 2019 18:09:56 +0000 (11:09 -0700)]
unified limitedOutput_directive
between lz4.c and lz4hc.c .
was left in a strange state after the "amalgamation" patch.
Now only 3 directives remain,
same name across both implementations,
single definition place.
Might allow some light simplification due to reduced nb of states possible.
Yann Collet [Mon, 15 Apr 2019 17:46:48 +0000 (10:46 -0700)]
Merge branch 'dev' into fixframe
Yann Collet [Mon, 15 Apr 2019 17:44:46 +0000 (10:44 -0700)]
added more margin for canary in frametest
Yann Collet [Mon, 15 Apr 2019 17:33:40 +0000 (10:33 -0700)]
decompress*_fast() function do not generate deprecation warnings
they are classified as deprecated in the API documentation (lz4.h)
but do not yet trigger a warning,
to give time to existing applications to move away.
Also, the _fast() variants are still ~5% faster than the _safe() ones
after Dave's patch.
Yann Collet [Mon, 15 Apr 2019 17:28:36 +0000 (10:28 -0700)]
fixed lz4frame with linked blocks
when one block was not compressible,
it would tag the context as `dirty`,
resulting in compression automatically bailing out of all future blocks,
making the rest of the frame uncompressible.
Yann Collet [Mon, 15 Apr 2019 17:14:36 +0000 (10:14 -0700)]
frametest: removed uninitialized warning
was a false positive, but better remove it anyway
Yann Collet [Mon, 15 Apr 2019 09:11:59 +0000 (02:11 -0700)]
assert() the failed test
ensure canary remains within buffer limits
Yann Collet [Mon, 15 Apr 2019 08:54:40 +0000 (01:54 -0700)]
fixed frametest
One test could write a canary value out of bound
in exceptional conditions involving multiple flushes
triggered by -s3421 -t462948.
Yann Collet [Sat, 13 Apr 2019 22:55:33 +0000 (15:55 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev