Yann Collet [Thu, 4 Apr 2019 19:47:36 +0000 (12:47 -0700)]
make `_fast*()` decoder generate a deprecation warning
updated modification
Yann Collet [Thu, 4 Apr 2019 19:24:46 +0000 (12:24 -0700)]
moved LZ4_decompress_fast*() into deprecated section
Yann Collet [Thu, 4 Apr 2019 00:15:06 +0000 (17:15 -0700)]
Merge pull request #657 from lz4/destSize
moved _destSize() into "stable API" status
Yann Collet [Wed, 3 Apr 2019 23:28:42 +0000 (16:28 -0700)]
fixed an old bug in LZ4F_flush()
which remained undetected so far,
as it requires a fairly large number of conditions to be triggered,
starting with enabling Block checksum, which is disabled by default,
and which usage is known to be extremely rare.
Yann Collet [Wed, 3 Apr 2019 21:27:21 +0000 (14:27 -0700)]
fixed doc
and bumped version number fo v1.9.0
Yann Collet [Wed, 3 Apr 2019 21:18:00 +0000 (14:18 -0700)]
moved _destSize() into "stable API" status
as requested in #642
Yann Collet [Wed, 3 Apr 2019 17:03:26 +0000 (10:03 -0700)]
Merge pull request #656 from lz4/armtest
LZ4_FAST_DEC_LOOP macros
Yann Collet [Wed, 3 Apr 2019 15:59:29 +0000 (08:59 -0700)]
minor comments and reformatting
Yann Collet [Wed, 3 Apr 2019 00:16:43 +0000 (17:16 -0700)]
fixed minor conversion warnings
Yann Collet [Tue, 2 Apr 2019 23:22:11 +0000 (16:22 -0700)]
created LZ4_FAST_DEC_LOOP build macro
Yann Collet [Tue, 2 Apr 2019 23:06:37 +0000 (16:06 -0700)]
fixed a few minor conversion warnings
Yann Collet [Sun, 3 Mar 2019 21:40:45 +0000 (13:40 -0800)]
Merge pull request #652 from vtorri/dev
Allow installation of lz4 for Windows 10 with MSYS2
Vincent Torri [Sun, 3 Mar 2019 21:06:38 +0000 (22:06 +0100)]
Allow installation of lz4 for Windows 10 with MSYS2
Yann Collet [Tue, 12 Feb 2019 00:58:53 +0000 (16:58 -0800)]
Merge pull request #645 from djwatson/optimize_decompress_generic
Optimize decompress generic
Dave Watson [Fri, 8 Feb 2019 21:57:43 +0000 (13:57 -0800)]
decompress_generic: Limit fastpath to x86
New fastpath currently shows a regression on qualcomm
arm chips. Restrict it to x86 for now
Dave Watson [Mon, 28 Jan 2019 17:36:25 +0000 (09:36 -0800)]
decompress_generic: Add fastpath for small offsets
For small offsets of size 1, 2, 4 and 8, we can set a single uint64_t,
and then use it to do a memset() variation. In particular, this makes
the somewhat-common RLE (offset 1) about 2-4x faster than the previous
implementation - we avoid not only the load blocked by store, but also
avoid the loads entirely.
Dave Watson [Sat, 26 Jan 2019 00:19:05 +0000 (16:19 -0800)]
decompress_generic: Unroll loops a bit more
Generally we want our wildcopy loops to look like the
memcpy loops from our libc, but without the final byte copy checks.
We can unroll a bit to make long copies even faster.
The only catch is that this affects the value of FASTLOOP_SAFE_DISTANCE.
Dave Watson [Sat, 26 Jan 2019 00:06:55 +0000 (16:06 -0800)]
decompress_generic: remove msan write
This store is also causing load-blocked-by-store issues, remove it.
The msan warning will have to be fixed another way if it is still an issue.
Dave Watson [Sat, 26 Jan 2019 00:01:09 +0000 (16:01 -0800)]
decompress_generic: re-add fastpath
This is the remaineder of the original 'shortcut'. If true, we can avoid
the loop in LZ4_wildCopy, and directly copy instead.
Dave Watson [Fri, 25 Jan 2019 22:47:15 +0000 (14:47 -0800)]
decompress_generic: drop partial copy check in fast loop
We've already checked that we are more than FASTLOOP_SAFE_DISTANCE
away from the end, so this branch can never be true, we will have
already jumped to the second decode loop.
Dave Watson [Fri, 25 Jan 2019 22:40:27 +0000 (14:40 -0800)]
decompress_generic: Optimize literal copies
Use LZ4_wildCopy16 for variable-length literals. For literal counts that
fit in the flag byte, copy directly. We can also omit oend checks for
roughly the same reason as the previous shortcut: We check once that both
match length and literal length fit in FASTLOOP_SAFE_DISTANCE, including
wildcopy distance.
Dave Watson [Thu, 24 Jan 2019 22:17:24 +0000 (14:17 -0800)]
decompress_generic: optimize match copy
Add an LZ4_wildCopy16, that will wildcopy, potentially smashing up
to 16 bytes, and use it for match copy. On x64, this avoids many
blocked loads due to store forwarding, similar to issue #411.
Dave Watson [Thu, 24 Jan 2019 21:42:25 +0000 (13:42 -0800)]
decompress_generic: Add a loop fastpath
Copy the main loop, and change checks such that op is always less
than oend-SAFE_DISTANCE. Currently these are added for the literal
copy length check, and for the match copy length check.
Otherwise the first loop is exactly the same as the second. Follow on
diffs will optimize the first copy loop based on this new requirement.
I also tried instead making a separate inlineable function for the copy
loop (similar to existing partialDecode flags, etc), but I think the
changes might be significant enough to warrent doubling the code, instead
pulling out common functionality to separate functions.
This is the basic transformation that will allow several following optimisations.
Dave Watson [Thu, 24 Jan 2019 21:07:57 +0000 (13:07 -0800)]
decompress_generic: Refactor variable length fields
Make a helper function to read variable lengths for literals and
match length.
Yann Collet [Thu, 7 Feb 2019 18:38:09 +0000 (10:38 -0800)]
Merge pull request #648 from aregm/fix-VS2017-solution
Build fixed by removing unavailable project
Areg Melik-Adamyan [Thu, 7 Feb 2019 04:29:31 +0000 (22:29 -0600)]
Build fixed by removing unavailable project
Yann Collet [Mon, 4 Feb 2019 22:50:28 +0000 (14:50 -0800)]
Merge pull request #646 from jbms/fix-clang-optimize-attribute-ppc64le
Eliminate optimize attribute warning with clang on PPC64LE
Jeremy Maitin-Shepard [Mon, 4 Feb 2019 20:22:56 +0000 (12:22 -0800)]
Eliminate optimize attribute warning with clang on PPC64LE
Yann Collet [Wed, 23 Jan 2019 17:03:13 +0000 (09:03 -0800)]
Merge pull request #644 from lzutao/meson-msvc-export
meson: Add -DLZ4_DLL_EXPORT=1 to build dynamic lib on Windows
Lzu Tao [Wed, 23 Jan 2019 08:40:26 +0000 (15:40 +0700)]
meson: Add -DLZ4_DLL_EXPORT=1 to build dynamic lib on Windows
Thanks @nacho for pointing it out.
Yann Collet [Fri, 11 Jan 2019 20:11:45 +0000 (12:11 -0800)]
Merge pull request #638 from lzutao/travis
Travis: Clean up .travis.yml
Yann Collet [Fri, 11 Jan 2019 20:11:35 +0000 (12:11 -0800)]
Merge pull request #639 from lzutao/meson
meson: Small improvements
Yann Collet [Fri, 11 Jan 2019 20:11:24 +0000 (12:11 -0800)]
Merge pull request #640 from tzakian/remove_io_globals
Remove a bunch of global variables that tracked settings for the IO module
Tim Zakian [Fri, 11 Jan 2019 17:49:26 +0000 (09:49 -0800)]
Add cast around malloc
Tim Zakian [Fri, 11 Jan 2019 04:40:00 +0000 (20:40 -0800)]
Add constant pointer annotations
Tim Zakian [Thu, 10 Jan 2019 23:27:47 +0000 (15:27 -0800)]
Remove a bunch of global variables that tracked settings for the IO module, and move them in to a struct
Lzu Tao [Thu, 10 Jan 2019 18:39:34 +0000 (01:39 +0700)]
travis: Prefer `apt-get` in `install` field than `addons-apt-sources`
Lzu Tao [Thu, 10 Jan 2019 18:20:16 +0000 (01:20 +0700)]
travis: Prefer `script` field than `Cmd` env
Lzu Tao [Thu, 10 Jan 2019 19:34:16 +0000 (02:34 +0700)]
meson: Favor warning if cannot find version string
Lzu Tao [Thu, 10 Jan 2019 19:33:27 +0000 (02:33 +0700)]
meson: Use libray as required argument in `pkgconfig`
Lzu Tao [Thu, 10 Jan 2019 19:32:39 +0000 (02:32 +0700)]
meson: Explicit use `meson setup` to setup a builddir
Yann Collet [Thu, 10 Jan 2019 18:56:41 +0000 (10:56 -0800)]
Merge pull request #637 from tzakian/fix_pass-through_mode
Fix pass-through mode
Tim Zakian [Thu, 10 Jan 2019 18:20:17 +0000 (10:20 -0800)]
Fix pass-through mode
Yann Collet [Thu, 10 Jan 2019 03:58:07 +0000 (19:58 -0800)]
Merge pull request #635 from tzakian/clean_call_to_LZ4HC_encodeSequence
Make effectfulness of calls to LZ4HC_encodeSequence clearer
Yann Collet [Wed, 9 Jan 2019 21:45:42 +0000 (13:45 -0800)]
fixed strict C++ compilation
Tim Zakian [Wed, 9 Jan 2019 21:42:12 +0000 (13:42 -0800)]
Make fact that certain variables that are passed into LZ4HC_encodeSequence are changed by the function call
Yann Collet [Wed, 9 Jan 2019 21:38:33 +0000 (13:38 -0800)]
fixed long sequence overflow test
Yann Collet [Wed, 9 Jan 2019 20:41:03 +0000 (12:41 -0800)]
minor explicit cast warning
Yann Collet [Wed, 9 Jan 2019 20:22:04 +0000 (12:22 -0800)]
Merge pull request #634 from lz4/longSeqTest
add a test to check long sequences (#631)
Yann Collet [Wed, 9 Jan 2019 20:21:39 +0000 (12:21 -0800)]
Merge pull request #631 from qiuyangs/dev
lz4hc.c: change (length >> 8) to (length / 255)
Yann Collet [Wed, 9 Jan 2019 20:13:17 +0000 (12:13 -0800)]
Merge pull request #633 from tzakian/make_block_size_public
Make LZ4F_getBlockSize public and public in experimental section
Yann Collet [Wed, 9 Jan 2019 20:09:52 +0000 (12:09 -0800)]
add a test to check long sequences (#631)
the test fails, as intended,
since #631 is not merged yet in this branch.
Tim Zakian [Wed, 9 Jan 2019 19:17:46 +0000 (11:17 -0800)]
Fix C90 compatibility issue
Tim Zakian [Wed, 9 Jan 2019 18:49:49 +0000 (10:49 -0800)]
Make LZ4F_getBlockSize public and publis in experimental section
Yann Collet [Wed, 9 Jan 2019 17:21:54 +0000 (09:21 -0800)]
Merge pull request #632 from rubenochiavone/fix-lz4-extesion-not-decompressing
Fix lz4 extension in input filename not causing decompression
Ruben O. Chiavone [Wed, 9 Jan 2019 04:51:40 +0000 (01:51 -0300)]
Add test to cover issue #596
Ruben O. Chiavone [Wed, 9 Jan 2019 01:56:04 +0000 (22:56 -0300)]
Fix lz4 extension in input filename not causing decompression
qiuyangs [Sun, 6 Jan 2019 08:33:53 +0000 (16:33 +0800)]
Merge pull request #1 from qiuyangs/sunqiuyang-fix-length>>8
lz4hc.c: change (length >> 8) to (length / 255)
qiuyangs [Sun, 6 Jan 2019 08:29:30 +0000 (16:29 +0800)]
lz4hc.c: change (length >> 8) to (length / 255)
Every 0xff byte in the compressed block corresponds to a length of 255 (not 256) in the input data. For long repeating sequences, using (length >> 8) may generate bad compressed blocks.
Yann Collet [Wed, 2 Jan 2019 23:02:22 +0000 (15:02 -0800)]
updated frame format
re-wording non-full blocks,
for clarity.
Yann Collet [Wed, 2 Jan 2019 22:36:12 +0000 (14:36 -0800)]
updated LZ4 block format
rewording the end of block conditions
for clarity and answering related questions.
Yann Collet [Mon, 17 Dec 2018 17:32:01 +0000 (09:32 -0800)]
Merge pull request #620 from lzutao/meson_symlink
Update meson symlink and man1 extension
Yann Collet [Fri, 14 Dec 2018 17:24:50 +0000 (09:24 -0800)]
Merge pull request #621 from lzutao/meson_getversion
meson: Remove unused sys import
Lzu Tao [Fri, 14 Dec 2018 04:12:22 +0000 (11:12 +0700)]
meson: Remove unused sys import
Lzu Tao [Thu, 13 Dec 2018 11:08:01 +0000 (18:08 +0700)]
Simplify logic by setting default value for MESON_INSTALL_DESTDIR_PREFIX
Lzu Tao [Thu, 13 Dec 2018 07:52:51 +0000 (14:52 +0700)]
meson: Update man1 extension in meson 0.49.0
Lzu Tao [Thu, 13 Dec 2018 07:27:13 +0000 (14:27 +0700)]
meson: Update InstallSymlink.py usage
Change default directory mode to 755.
Yann Collet [Wed, 12 Dec 2018 19:19:46 +0000 (11:19 -0800)]
Merge pull request #618 from dosaboy/add-snap-packaging-support
Add snap packaging support
Edward Hope-Morley [Tue, 11 Dec 2018 17:54:45 +0000 (17:54 +0000)]
Add snap packaging support
This commit adds a snapcraft.yaml file to allow lz4
to be distributed as a snap (see https://snapcraft.io/
for more info on snaps).
Building the Snap
-----------------
To build the snap simply install snapcraft and build the
snap e.g. on Ubuntu by doing:
$ sudo apt install snapcraft
$ cd contrib; snapcraft
This will build a snap that can be published [1] to
e.g. https://snapcraft.io/lz4
If you want to test the snap before publishing you
can do:
$ snap install <snapname>.snap --dangerous
And that will install it on your local host. You
will then have lz4 as a command to run.
Installing the Snap
-------------------
Once published and released in the snapstore your snap
can be installed on any platform that supports snaps
by doing:
snap install lz4
[1] https://docs.snapcraft.io/releasing-to-the-snap-store
Yann Collet [Sun, 9 Dec 2018 19:29:43 +0000 (11:29 -0800)]
Merge pull request #616 from felixhandte/lz4-attach-reject-short-dicts
Fix LZ4_compress_fast_continue() Prefix Size Check
Yann Collet [Sun, 9 Dec 2018 19:29:03 +0000 (11:29 -0800)]
Merge pull request #613 from lzutao/fix_meson_tests
meson: Fix build and test problems
Lzu Tao [Sun, 9 Dec 2018 11:57:49 +0000 (18:57 +0700)]
meson: Add README.md for newcomers [skip ci]
W. Felix Handte [Wed, 5 Dec 2018 19:24:33 +0000 (11:24 -0800)]
Fix Dict Size Test in `LZ4_compress_fast_continue()`
Dictionaries don't need to be > 4 bytes, they need to be >= 4 bytes. This test
was overly conservative.
Also removes the test in `LZ4_attach_dictionary()`.
W. Felix Handte [Tue, 4 Dec 2018 22:05:11 +0000 (14:05 -0800)]
Don't Attach Very Small Dictionaries
Fixes a mismatch in behavior between loading into the context (via
`LZ4_loadDict()`) a very small (<= 4 bytes) non-contiguous dictionary, versus
attaching it with `LZ4_attach_dictionary()`.
Before this patch, this divergence could be reproduced by running
```
make -C tests fuzzer MOREFLAGS="-m32"
tests/fuzzer -v -s1239 -t3146
```
Making sure these two paths behave exactly identically is an easy way to test
the correctness of the attach path, so it's desirable that this remain an
unpolluted, high signal test.
Lzu Tao [Tue, 4 Dec 2018 03:18:54 +0000 (10:18 +0700)]
No need to join prefix and let soversion base on version
Lzu Tao [Tue, 4 Dec 2018 03:18:01 +0000 (10:18 +0700)]
Use MESON_INSTALL_DESTDIR_PREFIX instead of DESTDIR
Lzu Tao [Mon, 3 Dec 2018 17:59:06 +0000 (00:59 +0700)]
meson: Add fallback message when not found lz4.h
Lzu Tao [Mon, 3 Dec 2018 17:56:10 +0000 (00:56 +0700)]
meson: Fix wrong path to lib/lz4.h
Lzu Tao [Mon, 3 Dec 2018 15:48:25 +0000 (22:48 +0700)]
meson: Correct support for Windows build
Lzu Tao [Sun, 2 Dec 2018 15:57:18 +0000 (22:57 +0700)]
meson: Use clang for faster build
"ninja install" automatically builds the project, no need to
use "ninja"
Lzu Tao [Mon, 3 Dec 2018 04:02:56 +0000 (11:02 +0700)]
meson: Update usage of InstallSymlink helper
Lzu Tao [Sun, 2 Dec 2018 15:42:14 +0000 (22:42 +0700)]
Use argparse instead of manually parsing [skip ci]
Lzu Tao [Sun, 2 Dec 2018 15:29:32 +0000 (22:29 +0700)]
meson: Fix soversion and generated pkg-config file [skip ci]
Lzu Tao [Sun, 2 Dec 2018 15:19:48 +0000 (22:19 +0700)]
meson: Use --werror build option instead of passing -Werror [skip ci]
Lzu Tao [Sun, 2 Dec 2018 14:36:09 +0000 (21:36 +0700)]
meson test: Update timeout and tests that use COPYING
Yann Collet [Sun, 2 Dec 2018 20:30:25 +0000 (12:30 -0800)]
Merge pull request #615 from vtorri/dev
Windows: create import library with gcc directly, remove now useless def file
Vincent Torri [Sun, 2 Dec 2018 18:42:38 +0000 (19:42 +0100)]
Merge remote-tracking branch 'upstream/dev' into dev
Vincent Torri [Sun, 2 Dec 2018 17:50:18 +0000 (18:50 +0100)]
Windows: create import library with gcc directly, remove now useless def file
Vincent Torri [Sun, 2 Dec 2018 17:46:14 +0000 (18:46 +0100)]
Merge remote-tracking branch 'upstream/master' into dev
Yann Collet [Sun, 2 Dec 2018 16:50:08 +0000 (08:50 -0800)]
Merge pull request #612 from lzutao/fix_clang_fuzzer
Fix clang warning
Lzu Tao [Sun, 2 Dec 2018 13:39:47 +0000 (20:39 +0700)]
clang: Fix -Wcomma
Lzu Tao [Sun, 2 Dec 2018 13:30:25 +0000 (20:30 +0700)]
clang: Fix -Wcast-qual
Lzu Tao [Sun, 2 Dec 2018 13:12:24 +0000 (20:12 +0700)]
Fix clang warnings: -Wformat-pedantic and -Werror=overflow
Yann Collet [Sun, 2 Dec 2018 03:35:47 +0000 (19:35 -0800)]
Merge pull request #611 from lzutao/mesonbuild
Support for Meson build and its CI tests
Lzu Tao [Sat, 1 Dec 2018 06:12:51 +0000 (13:12 +0700)]
Add Travis meson build
Lzu Tao [Sat, 1 Dec 2018 05:15:53 +0000 (12:15 +0700)]
Add meson build system
Yann Collet [Mon, 26 Nov 2018 20:14:27 +0000 (12:14 -0800)]
Merge pull request #610 from antinucleon/bootcamp
[amalgamation] lz4frame.c
Bing Xu [Mon, 26 Nov 2018 19:30:15 +0000 (11:30 -0800)]
[amalgamation] lz4frame.c
Bing Xu [Mon, 26 Nov 2018 19:12:49 +0000 (11:12 -0800)]
Merge remote-tracking branch 'upstream/master' into bootcamp
Yann Collet [Sun, 25 Nov 2018 21:38:37 +0000 (13:38 -0800)]
Merge pull request #606 from nh2/patch-1
lz4_Frame_format.md: Fix typo HexaBytes -> Exabytes