platform/upstream/lz4.git
5 years agoMerge pull request #748 from amchoukir/amchoukir-doublebuffer-doc
Yann Collet [Wed, 3 Jul 2019 23:13:44 +0000 (16:13 -0700)]
Merge pull request #748 from amchoukir/amchoukir-doublebuffer-doc

Update blockStreaming_doubleBuffer.md

5 years agoUpdate blockStreaming_doubleBuffer.md
Amine Choukir [Wed, 3 Jul 2019 09:50:38 +0000 (11:50 +0200)]
Update blockStreaming_doubleBuffer.md

5 years agoMerge pull request #745 from lz4/bump_192
Yann Collet [Mon, 1 Jul 2019 18:52:08 +0000 (11:52 -0700)]
Merge pull request #745 from lz4/bump_192

bumped version number to v1.9.2

5 years agobumped version number to v1.9.2
Yann Collet [Mon, 1 Jul 2019 16:01:43 +0000 (09:01 -0700)]
bumped version number to v1.9.2

to reduce risks that future bug reports in `dev` branch report `v1.9.1` as the failing version.

5 years agoMerge pull request #739 from cmeister2/cmeister2/ossfuzz
Yann Collet [Sun, 30 Jun 2019 20:05:15 +0000 (13:05 -0700)]
Merge pull request #739 from cmeister2/cmeister2/ossfuzz

oss-fuzz: Add a fuzzing target that compiles in the oss-fuzz environment

5 years agoRemove unnecessary call to Makefile.inc
Max Dymond [Sun, 30 Jun 2019 19:16:03 +0000 (20:16 +0100)]
Remove unnecessary call to Makefile.inc

5 years agoMerge pull request #741 from lzutao/meson-deprecated-warn
Yann Collet [Sun, 30 Jun 2019 18:33:38 +0000 (11:33 -0700)]
Merge pull request #741 from lzutao/meson-deprecated-warn

meson: Fix deprecated warnings on build options

5 years agoMerge pull request #742 from lz4/fuzzasan_fixed
Yann Collet [Sun, 30 Jun 2019 16:41:56 +0000 (09:41 -0700)]
Merge pull request #742 from lz4/fuzzasan_fixed

New fuzzer test for #738

5 years agomeson: Rename options in travis config
Lzu Tao [Sat, 29 Jun 2019 13:54:39 +0000 (20:54 +0700)]
meson: Rename options in travis config

5 years agoMerge branch 'dev' into fuzzasan_fixed
Yann Collet [Sat, 29 Jun 2019 04:03:07 +0000 (21:03 -0700)]
Merge branch 'dev' into fuzzasan_fixed

5 years agotravisCI: added ASAN fuzzer tests
Yann Collet [Sat, 29 Jun 2019 03:55:47 +0000 (20:55 -0700)]
travisCI: added ASAN fuzzer tests

and fixed minor formatting warnings

5 years agomoved noisy-src decoder test into cBuffer_exact
Yann Collet [Sat, 29 Jun 2019 03:23:12 +0000 (20:23 -0700)]
moved noisy-src decoder test into cBuffer_exact

so that it can also catch any potential read out-of-bound in the input buffer
(none reported so far, just a precaution for the future).

5 years agofuzzer: added test to catch #738
Yann Collet [Sat, 29 Jun 2019 03:15:43 +0000 (20:15 -0700)]
fuzzer: added test to catch #738

5 years agomeson: Always build gen_manual on build machine
Lzu Tao [Sat, 29 Jun 2019 03:11:32 +0000 (10:11 +0700)]
meson: Always build gen_manual on build machine

As gen_manual is using as a generator, not a binary target
installed on host machine.

5 years agomeson: Fix deprecated warnings on build options
Lzu Tao [Sat, 29 Jun 2019 03:10:05 +0000 (10:10 +0700)]
meson: Fix deprecated warnings on build options

Meson now reserves `build_` prefix options.

5 years agoMerge pull request #740 from terrelln/fix2
Yann Collet [Sat, 29 Jun 2019 02:22:54 +0000 (19:22 -0700)]
Merge pull request #740 from terrelln/fix2

Fix out-of-bounds read

5 years agoMore markups for style changes
Max Dymond [Fri, 28 Jun 2019 23:23:06 +0000 (00:23 +0100)]
More markups for style changes

5 years agoMove to using C rather than C++ for compilation
Max Dymond [Fri, 28 Jun 2019 22:48:33 +0000 (23:48 +0100)]
Move to using C rather than C++ for compilation

5 years agoFix out-of-bounds read of up to 64 KB in the past
Nick Terrell [Fri, 28 Jun 2019 21:40:14 +0000 (14:40 -0700)]
Fix out-of-bounds read of up to 64 KB in the past

5 years agoWrite a simple decompress target as well
Max Dymond [Fri, 28 Jun 2019 21:19:27 +0000 (22:19 +0100)]
Write a simple decompress target as well

5 years agoCode review markups:
Max Dymond [Fri, 28 Jun 2019 19:54:46 +0000 (20:54 +0100)]
Code review markups:

- Correct use of CPPFLAGS
- Detect allocation failure
- Add a CHECK macro for failure

5 years agoAdd a fuzzing target that compiles in the oss-fuzz environment
Max Dymond [Tue, 25 Jun 2019 16:22:02 +0000 (17:22 +0100)]
Add a fuzzing target that compiles in the oss-fuzz environment

5 years agoMerge pull request #735 from lz4/versionOut
Yann Collet [Tue, 25 Jun 2019 17:32:19 +0000 (10:32 -0700)]
Merge pull request #735 from lz4/versionOut

fix #734 : --version should output to stdout

5 years agofix #734 : --version should output to stdout
Yann Collet [Mon, 24 Jun 2019 23:08:30 +0000 (16:08 -0700)]
fix #734 : --version should output to stdout

instead of stderr

5 years agoupdate simple_buffer example
Yann Collet [Thu, 6 Jun 2019 21:17:44 +0000 (14:17 -0700)]
update simple_buffer example

there were a few tiny inaccuracies, especially in error conditions.

5 years agoprecise again that LZ4 decoder needs metadata
Yann Collet [Thu, 6 Jun 2019 20:20:30 +0000 (13:20 -0700)]
precise again that LZ4 decoder needs metadata

and that such metadata must be provided / sent / saved by the application.

5 years agoMerge pull request #727 from ephiepark/dev
Yann Collet [Wed, 5 Jun 2019 21:34:28 +0000 (14:34 -0700)]
Merge pull request #727 from ephiepark/dev

CircleCI : use custom Docker Image with pre-installed dependencies

5 years agocircleci : use custom docker image with pre-installed dependencies
Ephraim Park [Tue, 4 Jun 2019 19:45:31 +0000 (12:45 -0700)]
circleci : use custom docker image with pre-installed dependencies

5 years agorestored FORCE_INLINE
Yann Collet [Tue, 4 Jun 2019 21:04:49 +0000 (14:04 -0700)]
restored FORCE_INLINE

5 years agoMerge pull request #718 from lz4/fullbench
Yann Collet [Fri, 31 May 2019 22:18:12 +0000 (15:18 -0700)]
Merge pull request #718 from lz4/fullbench

fullbench: added test scenario LZ4F_decompress_followHint

5 years agofullbench: ensure decompressionFunction and dName are initialized
Yann Collet [Fri, 31 May 2019 20:25:12 +0000 (13:25 -0700)]
fullbench: ensure decompressionFunction and dName are initialized

Visual Studio seems to miss that they are necessarily initialized in the switch() { case: }

5 years agoMerge pull request #717 from lz4/inplace
Yann Collet [Fri, 31 May 2019 19:59:38 +0000 (12:59 -0700)]
Merge pull request #717 from lz4/inplace

Added documentation and macro to support in-place compression and decompression

5 years agoMerge pull request #719 from parheliamm/dev
Yann Collet [Fri, 31 May 2019 19:23:52 +0000 (12:23 -0700)]
Merge pull request #719 from parheliamm/dev

FAST_DEC_LOOP: move offset check in specific condition.

5 years agoreplaced while(1)
Yann Collet [Fri, 31 May 2019 19:01:33 +0000 (12:01 -0700)]
replaced while(1)

by for (;;)
just to please Visual Studio C4127 .

5 years agoadded more details for in-place documentation
Yann Collet [Fri, 31 May 2019 18:56:59 +0000 (11:56 -0700)]
added more details for in-place documentation

5 years agodecompress: changed final memcpy() into memmove()
Yann Collet [Fri, 31 May 2019 18:44:37 +0000 (11:44 -0700)]
decompress: changed final memcpy() into memmove()

for compatibility with in-place decompression scenarios.

5 years agoMakefile removed CFLAGS modifier
Yann Collet [Fri, 31 May 2019 18:31:48 +0000 (11:31 -0700)]
Makefile removed CFLAGS modifier

which was removing `-O3` as a side effect

5 years agoFAST_DEC_LOOP: only did offset check in specific condition.
Chenxi Mao [Fri, 31 May 2019 00:36:13 +0000 (08:36 +0800)]
FAST_DEC_LOOP: only did offset check in specific condition.

When I did FAST_DEC_LOOP performance test, I found the
offset check is much more than v1.8.3

You will see the condition check difference via lzbench with dickens test case.
v1.8.3 34959
v.1.9.x 1055885

After investigate the code, we could see the difference.
v.1.8.3 SKIP the condition check if
if condition is true in:
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1463
AND below condition is true
https://github.com/lz4/lz4/blob/v1.8.3/lib/lz4.c#L1478\
The offset check should be invoked.

v1.9.3
The offset check code will be invoked in every loop which lead to downgrade.
So the fix would be move this check to specific condition
to avoid useless condition check.

After this change, the call number is same as v1.8.3

5 years agofullbench: added test scenario LZ4F_decompress_followHint
Yann Collet [Fri, 31 May 2019 00:29:51 +0000 (17:29 -0700)]
fullbench: added test scenario LZ4F_decompress_followHint

This emulates a streaming scenario,
where the caller follows rigorously the srcSize hints
provided as return value of LZ4F_decompress().

This is useful to show the issue in #714,
where data is uselessly copied in a tmp buffer first.

5 years agoMerge pull request #714 from dreambottle/fix-lz4f-input-buf-use
Yann Collet [Fri, 31 May 2019 00:28:59 +0000 (17:28 -0700)]
Merge pull request #714 from dreambottle/fix-lz4f-input-buf-use

Fix src temporary buffer use in lz4frame

5 years agoupdated API manual
Yann Collet [Thu, 30 May 2019 23:23:53 +0000 (16:23 -0700)]
updated API manual

5 years agoupdated LZ4_DECOMPRESS_INPLACE_MARGIN
Yann Collet [Thu, 30 May 2019 23:19:30 +0000 (16:19 -0700)]
updated LZ4_DECOMPRESS_INPLACE_MARGIN

to pass worst case scenario.
Now adds margin proportional to input size to counter local expansion.

5 years agoadded test case for in-place decompression
Yann Collet [Thu, 30 May 2019 23:17:47 +0000 (16:17 -0700)]
added test case for in-place decompression

worst case, designed to make the decoder overwrite into input

5 years agoadd more doc on in-place (de)compression
Yann Collet [Thu, 30 May 2019 16:45:21 +0000 (09:45 -0700)]
add more doc on in-place (de)compression

5 years agofuzzer: changed internal buffer size
Yann Collet [Wed, 29 May 2019 20:33:55 +0000 (13:33 -0700)]
fuzzer: changed internal buffer size

to ensure no overflow during unit tests

5 years agoone more conversion warning
Yann Collet [Wed, 29 May 2019 20:17:45 +0000 (13:17 -0700)]
one more conversion warning

5 years agosome more minor conversion warnings fixes
Yann Collet [Wed, 29 May 2019 20:14:52 +0000 (13:14 -0700)]
some more minor conversion warnings fixes

5 years agofixed minor conversion warning
Yann Collet [Wed, 29 May 2019 19:56:27 +0000 (12:56 -0700)]
fixed minor conversion warning

5 years agoensure lz4.h can be included with or without LZ4_STATIC_LINKING_ONLY in any order
Yann Collet [Wed, 29 May 2019 19:21:14 +0000 (12:21 -0700)]
ensure lz4.h can be included with or without LZ4_STATIC_LINKING_ONLY in any order

ensure correct propagation of LZ4_DISTANCE_MAX

5 years agoadded comments and macros for in-place (de)compression
Yann Collet [Wed, 29 May 2019 19:06:13 +0000 (12:06 -0700)]
added comments and macros for in-place (de)compression

5 years agoadded test case for in-place compression
Yann Collet [Wed, 29 May 2019 18:19:10 +0000 (11:19 -0700)]
added test case for in-place compression

5 years agoMerge pull request #715 from gabrielstedman/ret
Yann Collet [Mon, 27 May 2019 18:59:23 +0000 (11:59 -0700)]
Merge pull request #715 from gabrielstedman/ret

Handle file not existing case #704

5 years agoHandle file not existing case #704
gabrielstedman [Sat, 25 May 2019 18:57:04 +0000 (19:57 +0100)]
Handle file not existing case #704

5 years agofix temporary buffer use when input size hint is respected
Niko Dzhus [Fri, 24 May 2019 15:25:06 +0000 (18:25 +0300)]
fix temporary buffer use when input size hint is respected

5 years agoMerge pull request #709 from lz4/circle
Yann Collet [Fri, 17 May 2019 19:22:39 +0000 (12:22 -0700)]
Merge pull request #709 from lz4/circle

updated tests for CircleCI

5 years agoupdated tests
Yann Collet [Thu, 16 May 2019 23:46:16 +0000 (16:46 -0700)]
updated tests

- only play listTest with `make test`, not `make all` which is limited to build
- update `clangtest`, so that it's possible to disable O3 optimization, for faster processing

5 years agoMerge pull request #708 from gabrielstedman/list
Yann Collet [Thu, 16 May 2019 22:56:42 +0000 (15:56 -0700)]
Merge pull request #708 from gabrielstedman/list

Add multiframe report to --list command

5 years agoList mode improvements.
gstedman [Thu, 16 May 2019 14:12:00 +0000 (15:12 +0100)]
List mode improvements.
  Improve formatting
  Include static assert
  Use UTIL_fseek to handle potential fseek limitation
  Be explicit when refusing to read from stdin
  Properly free dctx after use
  Include valgrind tests

5 years agoAdd multiframe report to --list command
gstedman [Fri, 10 May 2019 15:54:05 +0000 (16:54 +0100)]
Add multiframe report to --list command

5 years agoMerge pull request #707 from prekageo/dev
Yann Collet [Tue, 14 May 2019 03:35:10 +0000 (20:35 -0700)]
Merge pull request #707 from prekageo/dev

enable LZ4_FAST_DEC_LOOP build macro on aarch64 by default

5 years agoenable LZ4_FAST_DEC_LOOP build macro on aarch64/GCC by default
George Prekas [Fri, 3 May 2019 16:44:56 +0000 (11:44 -0500)]
enable LZ4_FAST_DEC_LOOP build macro on aarch64/GCC by default

5 years agoMerge pull request #700 from lzutao/meson-non-failures
Yann Collet [Tue, 30 Apr 2019 06:40:42 +0000 (23:40 -0700)]
Merge pull request #700 from lzutao/meson-non-failures

contrib: Make Meson build non optional

5 years agomeson: Rely only on extracted version in lz4.h
Lzu Tao [Tue, 30 Apr 2019 05:25:40 +0000 (12:25 +0700)]
meson: Rely only on extracted version in lz4.h

So now instead of warning when failing to extract version number
from lz4.h, we error and stop the build instead.

5 years agocontrib: Make Meson build non optional
Lzu Tao [Fri, 26 Apr 2019 18:41:03 +0000 (01:41 +0700)]
contrib: Make Meson build non optional

* Update ninja version to 1.9 in CI.
* Update default project version in meson script.

5 years agoMerge pull request #699 from brendene/compressBound
Yann Collet [Fri, 26 Apr 2019 03:26:04 +0000 (20:26 -0700)]
Merge pull request #699 from brendene/compressBound

Include block checksum in worst case scenario calculation of dstCapacity

5 years agoInclude block checksum in worst case scenario calculation of dstCapacity
Brenden Eng [Fri, 26 Apr 2019 02:37:39 +0000 (22:37 -0400)]
Include block checksum in worst case scenario calculation of dstCapacity

5 years agoMerge pull request #698 from lz4/tidy
Yann Collet [Wed, 24 Apr 2019 18:50:50 +0000 (11:50 -0700)]
Merge pull request #698 from lz4/tidy

several minor style changes recommended by clang-tidy

5 years agoseveral minor style changes recommended by clang-tidy
Yann Collet [Wed, 24 Apr 2019 17:03:02 +0000 (10:03 -0700)]
several minor style changes recommended by clang-tidy

5 years agoMerge pull request #697 from ldv-alt/fixes/test-amalgamation
Yann Collet [Wed, 24 Apr 2019 00:20:03 +0000 (17:20 -0700)]
Merge pull request #697 from ldv-alt/fixes/test-amalgamation

Fix test-amalgamation

5 years agoMerge pull request #696 from lz4/mactest
Yann Collet [Tue, 23 Apr 2019 22:08:46 +0000 (15:08 -0700)]
Merge pull request #696 from lz4/mactest

Fix dynamic library compilation on Mac OS-X

5 years agotest-amalgamation: split the rule
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: split the rule

Change test-amalgamation to follow each-rule-makes-a-single-target idiom.

Fixes: a7e8d394 ("[amalgamation] add test")

5 years agotest-amalgamation: fix compilation options
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: fix compilation options

Use the same compilation options to compile lz4_all.c and other object files.

Fixes: a7e8d394 ("[amalgamation] add test")

5 years agotest-amalgamation: use a single cat command
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: use a single cat command

Use the list of prerequisites instead of listing those files manually,
this way they will never fall out of sync.

Also update the amalgamation example to use a single cat command.

Fixes: a7e8d394 ("[amalgamation] add test")
Fixes: b192c86b ("[amalgamation] lz4frame.c")

5 years agotest-amalgamation: fix the list of prerequisites
Dmitry V. Levin [Tue, 23 Apr 2019 21:18:11 +0000 (21:18 +0000)]
test-amalgamation: fix the list of prerequisites

Add $(LZ4DIR)/lz4frame.c to the list of prerequisites as the rule uses
that file.

Fixes: b192c86b ("[amalgamation] lz4frame.c")

5 years agoMerge pull request #695 from lembacon/makefile-os
Yann Collet [Tue, 23 Apr 2019 21:09:40 +0000 (14:09 -0700)]
Merge pull request #695 from lembacon/makefile-os

lib/Makefile: Fix detection of `Darwin`.

5 years agoadded library build test on Mac OS-X
Yann Collet [Tue, 23 Apr 2019 19:29:39 +0000 (12:29 -0700)]
added library build test on Mac OS-X

5 years agolib/Makefile: Fix detection of `Darwin`.
Chongyu Zhu [Tue, 23 Apr 2019 19:13:19 +0000 (03:13 +0800)]
lib/Makefile: Fix detection of `Darwin`.

5 years agoMerge pull request #693 from JPeterMugaas/master
Yann Collet [Tue, 23 Apr 2019 16:20:41 +0000 (09:20 -0700)]
Merge pull request #693 from JPeterMugaas/master

More build imrpvements

5 years agoMore build imrpvements
JPeterMugaas [Tue, 23 Apr 2019 11:44:00 +0000 (07:44 -0400)]
More build imrpvements
Moved a few other tests to Makefiles.inc.  Other things might need to go there.
Made a test for symlink appropriateness.  Windows can NOT handle them the same way Unix-like operating systems do (if at all).  This is mostly the same as the Visual C projects.
embed version info into .dll and .exes that are redistributed.

5 years agoMerge branch 'dev' of github.com:Cyan4973/lz4 into dev
Yann Collet [Tue, 23 Apr 2019 00:45:40 +0000 (17:45 -0700)]
Merge branch 'dev' of github.com:Cyan4973/lz4 into dev

5 years agoupdated NEWS for v1.9.1
Yann Collet [Tue, 23 Apr 2019 00:45:24 +0000 (17:45 -0700)]
updated NEWS for v1.9.1

5 years agoMerge pull request #691 from lz4/list
Yann Collet [Tue, 23 Apr 2019 00:43:01 +0000 (17:43 -0700)]
Merge pull request #691 from lz4/list

--list

5 years agofix: no leak when LZ4F_dctx creation fails
Yann Collet [Mon, 22 Apr 2019 23:06:22 +0000 (16:06 -0700)]
fix: no leak when LZ4F_dctx creation fails

strange, because it previous implementation, it would `exit()`,
so it should not matter ...

5 years agoMerge pull request #689 from JPeterMugaas/jpm-makefile
Yann Collet [Mon, 22 Apr 2019 22:29:50 +0000 (15:29 -0700)]
Merge pull request #689 from JPeterMugaas/jpm-makefile

Jpm makefile - as described in https://github.com/lz4/lz4/issues/688

5 years agoupdated code comments
Yann Collet [Mon, 22 Apr 2019 22:24:44 +0000 (15:24 -0700)]
updated code comments

5 years ago--list gives block type
Yann Collet [Mon, 22 Apr 2019 22:14:53 +0000 (15:14 -0700)]
--list gives block type

5 years ago--list can resume after a failed file
Yann Collet [Mon, 22 Apr 2019 22:01:20 +0000 (15:01 -0700)]
--list can resume after a failed file

5 years agoMake programs/Makefile use the includes.
JPeterMugaas [Mon, 22 Apr 2019 21:38:43 +0000 (17:38 -0400)]
Make programs/Makefile use the includes.

5 years agorefactored --list function
Yann Collet [Mon, 22 Apr 2019 21:01:19 +0000 (14:01 -0700)]
refactored --list function

better handling of special conditions,
better scoping of variables.

Also : updated man page

5 years agoFix a test for mingw
JPeterMugaas [Mon, 22 Apr 2019 20:06:04 +0000 (16:06 -0400)]
Fix a test for mingw

5 years agoTry to put some tests I made in ONE place. I also moved a test for "install" in...
JPeterMugaas [Mon, 22 Apr 2019 19:48:48 +0000 (15:48 -0400)]
Try to put some tests I made in ONE place.  I also moved a test for "install" in one place to try to isolate it.

5 years agofixed C90 compliance
Yann Collet [Mon, 22 Apr 2019 19:27:25 +0000 (12:27 -0700)]
fixed C90 compliance

re-structure code, have everything into a single section of lz4io.c

5 years agoMerge pull request #690 from gabrielstedman/dev
Yann Collet [Mon, 22 Apr 2019 19:19:28 +0000 (12:19 -0700)]
Merge pull request #690 from gabrielstedman/dev

Implement --list

5 years agoInitial commits from diff I submitted earlier
JPeterMugaas [Mon, 22 Apr 2019 17:48:59 +0000 (13:48 -0400)]
Initial commits from diff I submitted earlier

5 years agoFR #598 - Correctly initialize cfinfo & cast malloc res to (char*)
gabrielstedman [Mon, 22 Apr 2019 10:07:08 +0000 (11:07 +0100)]
FR #598 - Correctly initialize cfinfo & cast malloc res to (char*)

5 years agoFR #598 - Make fileSize unsigned long long
gabrielstedman [Mon, 22 Apr 2019 09:35:29 +0000 (10:35 +0100)]
FR #598 - Make fileSize unsigned long long

5 years agoFR #598 - Move LZ4F_compFileInfo_t def to lz4frame.h discard output_file
gabrielstedman [Mon, 22 Apr 2019 08:58:26 +0000 (09:58 +0100)]
FR #598 - Move LZ4F_compFileInfo_t def to lz4frame.h discard output_file
assert if in om_list mode

5 years agoMerge https://github.com/lz4/lz4 into dev
gabrielstedman [Mon, 22 Apr 2019 08:56:51 +0000 (09:56 +0100)]
Merge https://github.com/lz4/lz4 into dev

5 years agoFR #598 - Make LZ4IO_getCompressedFileInfo internal and reword func
gabrielstedman [Mon, 22 Apr 2019 08:00:20 +0000 (09:00 +0100)]
FR #598 - Make LZ4IO_getCompressedFileInfo internal and reword func

5 years agoMerge pull request #686 from lz4/clistdout
Yann Collet [Mon, 22 Apr 2019 03:20:41 +0000 (20:20 -0700)]
Merge pull request #686 from lz4/clistdout

cli: warning for stdout as default output