platform/upstream/lz4.git
5 years agofuzzer: added test to catch #738
Yann Collet [Sat, 29 Jun 2019 03:15:43 +0000 (20:15 -0700)]
fuzzer: added test to catch #738

5 years agoMerge pull request #735 from lz4/versionOut
Yann Collet [Tue, 25 Jun 2019 17:32:19 +0000 (10:32 -0700)]
Merge pull request #735 from lz4/versionOut

fix #734 : --version should output to stdout

5 years agofix #734 : --version should output to stdout
Yann Collet [Mon, 24 Jun 2019 23:08:30 +0000 (16:08 -0700)]
fix #734 : --version should output to stdout

instead of stderr

5 years agoupdate simple_buffer example
Yann Collet [Thu, 6 Jun 2019 21:17:44 +0000 (14:17 -0700)]
update simple_buffer example

there were a few tiny inaccuracies, especially in error conditions.

5 years agoprecise again that LZ4 decoder needs metadata
Yann Collet [Thu, 6 Jun 2019 20:20:30 +0000 (13:20 -0700)]
precise again that LZ4 decoder needs metadata

and that such metadata must be provided / sent / saved by the application.

5 years agoMerge pull request #727 from ephiepark/dev
Yann Collet [Wed, 5 Jun 2019 21:34:28 +0000 (14:34 -0700)]
Merge pull request #727 from ephiepark/dev

CircleCI : use custom Docker Image with pre-installed dependencies

5 years agocircleci : use custom docker image with pre-installed dependencies
Ephraim Park [Tue, 4 Jun 2019 19:45:31 +0000 (12:45 -0700)]
circleci : use custom docker image with pre-installed dependencies

5 years agorestored FORCE_INLINE
Yann Collet [Tue, 4 Jun 2019 21:04:49 +0000 (14:04 -0700)]
restored FORCE_INLINE

5 years agoMerge pull request #718 from lz4/fullbench
Yann Collet [Fri, 31 May 2019 22:18:12 +0000 (15:18 -0700)]
Merge pull request #718 from lz4/fullbench

fullbench: added test scenario LZ4F_decompress_followHint

5 years agofullbench: ensure decompressionFunction and dName are initialized
Yann Collet [Fri, 31 May 2019 20:25:12 +0000 (13:25 -0700)]
fullbench: ensure decompressionFunction and dName are initialized

Visual Studio seems to miss that they are necessarily initialized in the switch() { case: }

5 years agoMerge pull request #717 from lz4/inplace
Yann Collet [Fri, 31 May 2019 19:59:38 +0000 (12:59 -0700)]
Merge pull request #717 from lz4/inplace

Added documentation and macro to support in-place compression and decompression

5 years agoMerge pull request #719 from parheliamm/dev
Yann Collet [Fri, 31 May 2019 19:23:52 +0000 (12:23 -0700)]
Merge pull request #719 from parheliamm/dev

FAST_DEC_LOOP: move offset check in specific condition.

5 years agoreplaced while(1)
Yann Collet [Fri, 31 May 2019 19:01:33 +0000 (12:01 -0700)]
replaced while(1)

by for (;;)
just to please Visual Studio C4127 .

5 years agoadded more details for in-place documentation
Yann Collet [Fri, 31 May 2019 18:56:59 +0000 (11:56 -0700)]
added more details for in-place documentation

5 years agodecompress: changed final memcpy() into memmove()
Yann Collet [Fri, 31 May 2019 18:44:37 +0000 (11:44 -0700)]
decompress: changed final memcpy() into memmove()

for compatibility with in-place decompression scenarios.

5 years agoMakefile removed CFLAGS modifier
Yann Collet [Fri, 31 May 2019 18:31:48 +0000 (11:31 -0700)]
Makefile removed CFLAGS modifier

which was removing `-O3` as a side effect

5 years agoFAST_DEC_LOOP: only did offset check in specific condition.
Chenxi Mao [Fri, 31 May 2019 00:36:13 +0000 (08:36 +0800)]
FAST_DEC_LOOP: only did offset check in specific condition.

When I did FAST_DEC_LOOP performance test, I found the
offset check is much more than v1.8.3

You will see the condition check difference via lzbench with dickens test case.
v1.8.3 34959
v.1.9.x 1055885

After investigate the code, we could see the difference.
v.1.8.3 SKIP the condition check if
if condition is true in:
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1463
AND below condition is true
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1478\
The offset check should be invoked.

v1.9.3
The offset check code will be invoked in every loop which lead to downgrade.
So the fix would be move this check to specific condition
to avoid useless condition check.

After this change, the call number is same as v1.8.3

5 years agofullbench: added test scenario LZ4F_decompress_followHint
Yann Collet [Fri, 31 May 2019 00:29:51 +0000 (17:29 -0700)]
fullbench: added test scenario LZ4F_decompress_followHint

This emulates a streaming scenario,
where the caller follows rigorously the srcSize hints
provided as return value of LZ4F_decompress().

This is useful to show the issue in #714,
where data is uselessly copied in a tmp buffer first.

5 years agoMerge pull request #714 from dreambottle/fix-lz4f-input-buf-use
Yann Collet [Fri, 31 May 2019 00:28:59 +0000 (17:28 -0700)]
Merge pull request #714 from dreambottle/fix-lz4f-input-buf-use

Fix src temporary buffer use in lz4frame

5 years agoupdated API manual
Yann Collet [Thu, 30 May 2019 23:23:53 +0000 (16:23 -0700)]
updated API manual

5 years agoupdated LZ4_DECOMPRESS_INPLACE_MARGIN
Yann Collet [Thu, 30 May 2019 23:19:30 +0000 (16:19 -0700)]
updated LZ4_DECOMPRESS_INPLACE_MARGIN

to pass worst case scenario.
Now adds margin proportional to input size to counter local expansion.

5 years agoadded test case for in-place decompression
Yann Collet [Thu, 30 May 2019 23:17:47 +0000 (16:17 -0700)]
added test case for in-place decompression

worst case, designed to make the decoder overwrite into input

5 years agoadd more doc on in-place (de)compression
Yann Collet [Thu, 30 May 2019 16:45:21 +0000 (09:45 -0700)]
add more doc on in-place (de)compression

5 years agofuzzer: changed internal buffer size
Yann Collet [Wed, 29 May 2019 20:33:55 +0000 (13:33 -0700)]
fuzzer: changed internal buffer size

to ensure no overflow during unit tests

5 years agoone more conversion warning
Yann Collet [Wed, 29 May 2019 20:17:45 +0000 (13:17 -0700)]
one more conversion warning

5 years agosome more minor conversion warnings fixes
Yann Collet [Wed, 29 May 2019 20:14:52 +0000 (13:14 -0700)]
some more minor conversion warnings fixes

5 years agofixed minor conversion warning
Yann Collet [Wed, 29 May 2019 19:56:27 +0000 (12:56 -0700)]
fixed minor conversion warning

5 years agoensure lz4.h can be included with or without LZ4_STATIC_LINKING_ONLY in any order
Yann Collet [Wed, 29 May 2019 19:21:14 +0000 (12:21 -0700)]
ensure lz4.h can be included with or without LZ4_STATIC_LINKING_ONLY in any order

ensure correct propagation of LZ4_DISTANCE_MAX

5 years agoadded comments and macros for in-place (de)compression
Yann Collet [Wed, 29 May 2019 19:06:13 +0000 (12:06 -0700)]
added comments and macros for in-place (de)compression

5 years agoadded test case for in-place compression
Yann Collet [Wed, 29 May 2019 18:19:10 +0000 (11:19 -0700)]
added test case for in-place compression

5 years agoMerge pull request #715 from gabrielstedman/ret
Yann Collet [Mon, 27 May 2019 18:59:23 +0000 (11:59 -0700)]
Merge pull request #715 from gabrielstedman/ret

Handle file not existing case #704

5 years agoHandle file not existing case #704
gabrielstedman [Sat, 25 May 2019 18:57:04 +0000 (19:57 +0100)]
Handle file not existing case #704

5 years agofix temporary buffer use when input size hint is respected
Niko Dzhus [Fri, 24 May 2019 15:25:06 +0000 (18:25 +0300)]
fix temporary buffer use when input size hint is respected

5 years agoMerge pull request #709 from lz4/circle
Yann Collet [Fri, 17 May 2019 19:22:39 +0000 (12:22 -0700)]
Merge pull request #709 from lz4/circle

updated tests for CircleCI

5 years agoupdated tests
Yann Collet [Thu, 16 May 2019 23:46:16 +0000 (16:46 -0700)]
updated tests

- only play listTest with `make test`, not `make all` which is limited to build
- update `clangtest`, so that it's possible to disable O3 optimization, for faster processing

5 years agoMerge pull request #708 from gabrielstedman/list
Yann Collet [Thu, 16 May 2019 22:56:42 +0000 (15:56 -0700)]
Merge pull request #708 from gabrielstedman/list

Add multiframe report to --list command

5 years agoList mode improvements.
gstedman [Thu, 16 May 2019 14:12:00 +0000 (15:12 +0100)]
List mode improvements.
  Improve formatting
  Include static assert
  Use UTIL_fseek to handle potential fseek limitation
  Be explicit when refusing to read from stdin
  Properly free dctx after use
  Include valgrind tests

5 years agoAdd multiframe report to --list command
gstedman [Fri, 10 May 2019 15:54:05 +0000 (16:54 +0100)]
Add multiframe report to --list command

5 years agoMerge pull request #707 from prekageo/dev
Yann Collet [Tue, 14 May 2019 03:35:10 +0000 (20:35 -0700)]
Merge pull request #707 from prekageo/dev

enable LZ4_FAST_DEC_LOOP build macro on aarch64 by default

5 years agoenable LZ4_FAST_DEC_LOOP build macro on aarch64/GCC by default
George Prekas [Fri, 3 May 2019 16:44:56 +0000 (11:44 -0500)]
enable LZ4_FAST_DEC_LOOP build macro on aarch64/GCC by default

5 years agoMerge pull request #700 from lzutao/meson-non-failures
Yann Collet [Tue, 30 Apr 2019 06:40:42 +0000 (23:40 -0700)]
Merge pull request #700 from lzutao/meson-non-failures

contrib: Make Meson build non optional

5 years agomeson: Rely only on extracted version in lz4.h
Lzu Tao [Tue, 30 Apr 2019 05:25:40 +0000 (12:25 +0700)]
meson: Rely only on extracted version in lz4.h

So now instead of warning when failing to extract version number
from lz4.h, we error and stop the build instead.

5 years agocontrib: Make Meson build non optional
Lzu Tao [Fri, 26 Apr 2019 18:41:03 +0000 (01:41 +0700)]
contrib: Make Meson build non optional

* Update ninja version to 1.9 in CI.
* Update default project version in meson script.

5 years agoMerge pull request #699 from brendene/compressBound
Yann Collet [Fri, 26 Apr 2019 03:26:04 +0000 (20:26 -0700)]
Merge pull request #699 from brendene/compressBound

Include block checksum in worst case scenario calculation of dstCapacity

5 years agoInclude block checksum in worst case scenario calculation of dstCapacity
Brenden Eng [Fri, 26 Apr 2019 02:37:39 +0000 (22:37 -0400)]
Include block checksum in worst case scenario calculation of dstCapacity

5 years agoMerge pull request #698 from lz4/tidy
Yann Collet [Wed, 24 Apr 2019 18:50:50 +0000 (11:50 -0700)]
Merge pull request #698 from lz4/tidy

several minor style changes recommended by clang-tidy

5 years agoseveral minor style changes recommended by clang-tidy
Yann Collet [Wed, 24 Apr 2019 17:03:02 +0000 (10:03 -0700)]
several minor style changes recommended by clang-tidy

5 years agoMerge pull request #697 from ldv-alt/fixes/test-amalgamation
Yann Collet [Wed, 24 Apr 2019 00:20:03 +0000 (17:20 -0700)]
Merge pull request #697 from ldv-alt/fixes/test-amalgamation

Fix test-amalgamation

5 years agoMerge pull request #696 from lz4/mactest
Yann Collet [Tue, 23 Apr 2019 22:08:46 +0000 (15:08 -0700)]
Merge pull request #696 from lz4/mactest

Fix dynamic library compilation on Mac OS-X

5 years agotest-amalgamation: split the rule
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: split the rule

Change test-amalgamation to follow each-rule-makes-a-single-target idiom.

Fixes: a7e8d394 ("[amalgamation] add test")

5 years agotest-amalgamation: fix compilation options
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: fix compilation options

Use the same compilation options to compile lz4_all.c and other object files.

Fixes: a7e8d394 ("[amalgamation] add test")

5 years agotest-amalgamation: use a single cat command
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: use a single cat command

Use the list of prerequisites instead of listing those files manually,
this way they will never fall out of sync.

Also update the amalgamation example to use a single cat command.

Fixes: a7e8d394 ("[amalgamation] add test")
Fixes: b192c86b ("[amalgamation] lz4frame.c")

5 years agotest-amalgamation: fix the list of prerequisites
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: fix the list of prerequisites

Add $(LZ4DIR)/lz4frame.c to the list of prerequisites as the rule uses
that file.

Fixes: b192c86b ("[amalgamation] lz4frame.c")

5 years agoMerge pull request #695 from lembacon/makefile-os
Yann Collet [Tue, 23 Apr 2019 21:09:40 +0000 (14:09 -0700)]
Merge pull request #695 from lembacon/makefile-os

lib/Makefile: Fix detection of `Darwin`.

5 years agoadded library build test on Mac OS-X
Yann Collet [Tue, 23 Apr 2019 19:29:39 +0000 (12:29 -0700)]
added library build test on Mac OS-X

5 years agolib/Makefile: Fix detection of `Darwin`.
Chongyu Zhu [Tue, 23 Apr 2019 19:13:19 +0000 (03:13 +0800)]
lib/Makefile: Fix detection of `Darwin`.

5 years agoMerge pull request #693 from JPeterMugaas/master
Yann Collet [Tue, 23 Apr 2019 16:20:41 +0000 (09:20 -0700)]
Merge pull request #693 from JPeterMugaas/master

More build imrpvements

5 years agoMore build imrpvements
JPeterMugaas [Tue, 23 Apr 2019 11:44:00 +0000 (07:44 -0400)]
More build imrpvements
Moved a few other tests to Makefiles.inc.  Other things might need to go there.
Made a test for symlink appropriateness.  Windows can NOT handle them the same way Unix-like operating systems do (if at all).  This is mostly the same as the Visual C projects.
embed version info into .dll and .exes that are redistributed.

5 years agoMerge branch 'dev' of github.com:Cyan4973/lz4 into dev
Yann Collet [Tue, 23 Apr 2019 00:45:40 +0000 (17:45 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev

5 years agoupdated NEWS for v1.9.1
Yann Collet [Tue, 23 Apr 2019 00:45:24 +0000 (17:45 -0700)]
updated NEWS for v1.9.1

5 years agoMerge pull request #691 from lz4/list
Yann Collet [Tue, 23 Apr 2019 00:43:01 +0000 (17:43 -0700)]
Merge pull request #691 from lz4/list

--list

5 years agofix: no leak when LZ4F_dctx creation fails
Yann Collet [Mon, 22 Apr 2019 23:06:22 +0000 (16:06 -0700)]
fix: no leak when LZ4F_dctx creation fails

strange, because it previous implementation, it would `exit()`,
so it should not matter ...

5 years agoMerge pull request #689 from JPeterMugaas/jpm-makefile
Yann Collet [Mon, 22 Apr 2019 22:29:50 +0000 (15:29 -0700)]
Merge pull request #689 from JPeterMugaas/jpm-makefile

Jpm makefile - as described in https://github.com/lz4/lz4/issues/688

5 years agoupdated code comments
Yann Collet [Mon, 22 Apr 2019 22:24:44 +0000 (15:24 -0700)]
updated code comments

5 years ago--list gives block type
Yann Collet [Mon, 22 Apr 2019 22:14:53 +0000 (15:14 -0700)]
--list gives block type

5 years ago--list can resume after a failed file
Yann Collet [Mon, 22 Apr 2019 22:01:20 +0000 (15:01 -0700)]
--list can resume after a failed file

5 years agoMake programs/Makefile use the includes.
JPeterMugaas [Mon, 22 Apr 2019 21:38:43 +0000 (17:38 -0400)]
Make programs/Makefile use the includes.

5 years agorefactored --list function
Yann Collet [Mon, 22 Apr 2019 21:01:19 +0000 (14:01 -0700)]
refactored --list function

better handling of special conditions,
better scoping of variables.

Also : updated man page

5 years agoFix a test for mingw
JPeterMugaas [Mon, 22 Apr 2019 20:06:04 +0000 (16:06 -0400)]
Fix a test for mingw

5 years agoTry to put some tests I made in ONE place. I also moved a test for "install" in...
JPeterMugaas [Mon, 22 Apr 2019 19:48:48 +0000 (15:48 -0400)]
Try to put some tests I made in ONE place.  I also moved a test for "install" in one place to try to isolate it.

5 years agofixed C90 compliance
Yann Collet [Mon, 22 Apr 2019 19:27:25 +0000 (12:27 -0700)]
fixed C90 compliance

re-structure code, have everything into a single section of lz4io.c

5 years agoMerge pull request #690 from gabrielstedman/dev
Yann Collet [Mon, 22 Apr 2019 19:19:28 +0000 (12:19 -0700)]
Merge pull request #690 from gabrielstedman/dev

Implement --list

5 years agoInitial commits from diff I submitted earlier
JPeterMugaas [Mon, 22 Apr 2019 17:48:59 +0000 (13:48 -0400)]
Initial commits from diff I submitted earlier

5 years agoFR #598 - Correctly initialize cfinfo & cast malloc res to (char*)
gabrielstedman [Mon, 22 Apr 2019 10:07:08 +0000 (11:07 +0100)]
FR #598 - Correctly initialize cfinfo & cast malloc res to (char*)

5 years agoFR #598 - Make fileSize unsigned long long
gabrielstedman [Mon, 22 Apr 2019 09:35:29 +0000 (10:35 +0100)]
FR #598 - Make fileSize unsigned long long

5 years agoFR #598 - Move LZ4F_compFileInfo_t def to lz4frame.h discard output_file
gabrielstedman [Mon, 22 Apr 2019 08:58:26 +0000 (09:58 +0100)]
FR #598 - Move LZ4F_compFileInfo_t def to lz4frame.h discard output_file
assert if in om_list mode

5 years agoMerge https://github.com/lz4/lz4 into dev
gabrielstedman [Mon, 22 Apr 2019 08:56:51 +0000 (09:56 +0100)]
Merge https://github.com/lz4/lz4 into dev

5 years agoFR #598 - Make LZ4IO_getCompressedFileInfo internal and reword func
gabrielstedman [Mon, 22 Apr 2019 08:00:20 +0000 (09:00 +0100)]
FR #598 - Make LZ4IO_getCompressedFileInfo internal and reword func

5 years agoMerge pull request #686 from lz4/clistdout
Yann Collet [Mon, 22 Apr 2019 03:20:41 +0000 (20:20 -0700)]
Merge pull request #686 from lz4/clistdout

cli: warning for stdout as default output

5 years agoensure tests work when `stdout` is not the console
Yann Collet [Mon, 22 Apr 2019 00:01:03 +0000 (17:01 -0700)]
ensure tests work when `stdout` is not the console

ensure this case is continuously tested on travis.

Update documentation on implicit output,
invite to not rely on implicit output in scripts.

5 years agotests/Makefile : created CMP variable
Yann Collet [Sun, 21 Apr 2019 19:42:13 +0000 (12:42 -0700)]
tests/Makefile : created CMP variable

for potential redirection, if need be.

note : should probably converge all comparison operations onto CMP

5 years agoFR #598 Improve initial design, test mallocs, support C90.
gabrielstedman [Sun, 21 Apr 2019 16:43:57 +0000 (17:43 +0100)]
FR #598 Improve initial design, test mallocs, support C90.

5 years agocorrectly use unisgned int for index
gabrielstedman [Sat, 20 Apr 2019 23:20:38 +0000 (00:20 +0100)]
correctly use unisgned int for index

5 years agoAdd --list option to display compressed file information.
gabrielstedman [Sat, 20 Apr 2019 21:06:10 +0000 (22:06 +0100)]
Add --list option to display compressed file information.

5 years agocli: display a warning whenever default output is stdout while input != stdin
Yann Collet [Sat, 20 Apr 2019 00:08:40 +0000 (17:08 -0700)]
cli: display a warning whenever default output is stdout while input != stdin

This behavior has been preserved for compatibility with existing ecosystem.
But it's problematic, as some environment start `lz4` without identifying stdout as console by default,
leading to a change of behavior for a same line of script.

A more sensible policy would be to default to stdout only when input is stdin.

Soft change for the time being : keep the behavior, just print a warning message.
User should prefer `-c` to explicitly select `stdout`.

Also : updated tests in Makefile to explicitly select `stdout` with `-c`.

5 years agobumped version number
Yann Collet [Fri, 19 Apr 2019 18:54:01 +0000 (11:54 -0700)]
bumped version number

to v1.9.1

5 years agoMerge pull request #684 from lz4/cxxlz4f
Yann Collet [Fri, 19 Apr 2019 18:51:29 +0000 (11:51 -0700)]
Merge pull request #684 from lz4/cxxlz4f

lz4frame: initializers compatibility with C++

5 years agolz4frame: initializers compatibility with C++
Yann Collet [Fri, 19 Apr 2019 17:23:50 +0000 (10:23 -0700)]
lz4frame: initializers compatibility with C++

fix #679, reported by @degski

5 years agoMerge pull request #683 from lz4/fuzasan
Yann Collet [Fri, 19 Apr 2019 03:57:56 +0000 (20:57 -0700)]
Merge pull request #683 from lz4/fuzasan

fixed read-after input in LZ4_decompress_safe()

5 years agoMerge pull request #682 from lz4/visual
Yann Collet [Fri, 19 Apr 2019 01:51:28 +0000 (18:51 -0700)]
Merge pull request #682 from lz4/visual

fix minor Visual warnings

5 years agofixed read-after input in LZ4_decompress_safe()
Yann Collet [Fri, 19 Apr 2019 01:50:51 +0000 (18:50 -0700)]
fixed read-after input in LZ4_decompress_safe()

5 years agoensure list of names is large enough
Yann Collet [Fri, 19 Apr 2019 00:26:01 +0000 (17:26 -0700)]
ensure list of names is large enough

5 years agofuzzer : reduced stack usage
Yann Collet [Fri, 19 Apr 2019 00:12:14 +0000 (17:12 -0700)]
fuzzer : reduced stack usage

5 years agofuzzer: reduce stack usage to please Visual static analyzer
Yann Collet [Thu, 18 Apr 2019 23:41:27 +0000 (16:41 -0700)]
fuzzer: reduce stack usage to please Visual static analyzer

5 years agoaddress a few minor Visual warnings
Yann Collet [Thu, 18 Apr 2019 23:06:02 +0000 (16:06 -0700)]
address a few minor Visual warnings

and created target cxx17build

5 years agoMerge pull request #678 from lz4/decFast
Yann Collet [Thu, 18 Apr 2019 21:29:20 +0000 (14:29 -0700)]
Merge pull request #678 from lz4/decFast

Fix out-of-bound read in LZ4_decompress_fast()

5 years agofix several minor static analyzer warnings
Yann Collet [Thu, 18 Apr 2019 19:05:50 +0000 (12:05 -0700)]
fix several minor static analyzer warnings

5 years agofuzzer: fixed scan-build leak warning
Yann Collet [Thu, 18 Apr 2019 17:38:51 +0000 (10:38 -0700)]
fuzzer: fixed scan-build leak warning

the program exit(),
so there is no need to track and dealloc every buffer.

5 years agofuzzer: fix explicit cast for C++
Yann Collet [Wed, 17 Apr 2019 22:44:14 +0000 (15:44 -0700)]
fuzzer: fix explicit cast for C++

5 years agore-enable FORCE_INLINE
Yann Collet [Wed, 17 Apr 2019 22:33:37 +0000 (15:33 -0700)]
re-enable FORCE_INLINE

was disabled for tests