erofs-utils: lib: support GNUTYPE_LONGNAME for tarerofs

The 'L' entry is present in a header for a series of 1 or more 512-byte
tar blocks that hold just the filename for a file or directory with a
name over 100 chars.

Following that series is another header block, in the traditional form:
   A header with type '0' (regular file) or '5' (directory), followed by
   the appropriate number of data blocks with the entry data.

In the header for this series, the name will be truncated to the 1st 100
characters of the actual name.

Tested-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230717073531.43203-1-hsiangkao@linux.alibaba.com

erofs-utils: introduce tarerofs

Let's try to add a new mode "tarerofs" for mkfs.erofs.

It mainly aims at two use cases:
 - Convert a tarball (or later tarballs with a merged view) into
   a full EROFS image [--tar=f];

 - Generate an EROFS manifest image to reuse tar data [--tar=i],
   which also enables EROFS 512-byte blocks.

The second use case is mainly prepared for OCI direct mount without
OCI blob unpacking.  This also adds another `--aufs` option to
transform aufs special files into overlayfs metadata.

[ Note that `--tar=f` generates lots of temporary files for now which
  can impact performance since the original tar stream(s) may be
  non-seekable. ]

Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230714065851.70583-1-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: add libdeflate compressor support

Eric suggests a "binary search + heuristics" way by using the
current libdeflate APIs to generate fixed-sized output DEFLATE streams.

Compared to the previous built-in one, it will generate smaller images
(which is expected since the built-in one is roughly just the original
zlib replacement), yet the total compression time might be amplified a
lot especially if some larger pclusters are used by users compared to
the built-in one.

For example:
$ time mkfs.erofs -zdeflate,9 -C65536 enwik8.z enwik8
real    0m9.559s
user    0m9.453s
sys     0m0.069s

$ time mkfs.erofs -zlibdeflate,9 -C65536 enwik8.libdeflate.9.z enwik8
real    0m50.184s
user    0m50.082s
sys     0m0.074s

$ mkfs/mkfs.erofs -zlibdeflate,6 -C65536 enwik8.libdeflate.6.z enwik8
real    0m23.428s
user    0m23.329s
sys     0m0.067s

37175296 enwik8.libdeflate.6.z
37142528 enwik8.z
36835328 enwik8.libdeflate.9.z

Anyway, let's use the current APIs for users who needs smaller image
sizes for now.  Besides, EROFS also supports multiple per-file
algorithms in one image, so it can be used for specific files as well.

Suggested-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230710110251.89464-5-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: add DEFLATE algorithm support

This patch adds DEFLATE compression algorithm support to erofs-utils
compression framework.

Note that windowbits (which indicates dictionary size) is recorded in
the on-disk compression configuration.  Since some accelerators (e.g.
Intel IAA) don't have enough on-chip memory, compressed data generated
with large windowbits (e.g. > 12 for the IAA accelerator) doesn't seem
to be worked properly on those.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230710110251.89464-4-hsiangkao@linux.alibaba.com

erofs-utils: fuse,fsck: add DEFLATE algorithm support

This patch adds DEFLATE compression algorithm support to erofsfuse
by using zlib (by default) and libdeflate.  libdeflate will be used
instead of zlib if libdeflate is enabled.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230712235143.10125-1-hsiangkao@linux.alibaba.com

erofs-utils: add a built-in DEFLATE compressor

As Apple documentation written "If you require interoperability with
non-Apple devices, use COMPRESSION_ZLIB. [1]", DEFLATE is a popular
generic-purpose compression algorithm for a quite long time (many
advanced formats like zlib, gzip, zip, png are all based on that),
which is made of LZ77 as well as Huffman coding, fully documented as
RFC1951 [2] and quite easy to understand, implement.

There are several hardware on-market DEFLATE accelerators as well,
such as (s390) DFLTCC, (Intel) IAA/QAT, (HiSilicon) ZIP accelerator,
etc.  Therefore, it's useful to support DEFLATE compression in order
to use these for async I/Os and get benefits from these.

Since there is _no fixed-sized output DEFLATE compression appoach_
available in public (fitblk is somewhat ineffective) and the original
zlib is quite slowly developping, let's work out one for our use cases.
Fortunately, it's only less than 1.5kLOC with lazy matching to just
match the full zlib abilities.  Besides, near-optimal block splitting
(based on price function) doesn't support since it's no rush to us.

In the future, there might be more built-in optimizers landed to
fulfill our needs even further (especially for other popular algorithms
without native fixed-sized output support).  In addition, I'd be quite
happy to see more popular encoders to support native fixed-sized output
compression too.

[1] https://developer.apple.com/documentation/compression/compression_algorithm
[2] https://datatracker.ietf.org/doc/html/rfc1951
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230710110251.89464-2-hsiangkao@linux.alibaba.com

erofs-utils: add github issue/pull-request templates

EROFS Development is currently on the mailing list _only_.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230713111046.62793-1-hsiangkao@linux.alibaba.com

erofs-utils: add ERR_CAST macro

Add ERR_CAST macro to prepare for the upcoming tarerofs feature.

Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230711061240.23662-1-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: fix small compressed files inlining

Keep in sync with kernel commit 24331050a3e6 ("erofs: fix small
compressed files inlining") to avoid corruption due to m_llen > m_plen
for uncompressed pclusters.

Fixes: 41790d24329d ("erofs-utils: validate the extent length for uncompressed pclusters")
Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20230711032508.6892-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: switch to effective unaligned access

In order to prepare for LZ77 matchfinder.  Note that erofs_memcmp2()
is still not quite effective.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230709182511.96954-2-hsiangkao@linux.alibaba.com

erofs-utils: Provide identical functionality without libuuid

The motivation is to have standalone (statically linked) erofs binaries
for simple initrd images, that are nevertheless able to (re)create
erofs images with a given UUID.

For this reason a few of libuuid functions have implementations added
directly in erofs-utils.
A header liberofs_uuid.h provides the new functions, which are
always available. A further sideeffect is that code can be simplified
which calls into this functionality.

The uuid_unparse function replacement is always a private
implementation and split into its own file, this further restricts
the (optional) dependency on libuuid only to the erofs-mkfs tool.

Signed-off-by: Norbert Lange <nolange79@gmail.com>
Link: https://lore.kernel.org/r/20230703215803.4476-1-nolange79@gmail.com
[ Gao Xiang: a getrandom() fix since it was added in glibc 2.25. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: update my email address in AUTHORS

Update the invalid email address.

Signed-off-by: Huang Jianan <jnhuang95@gmail.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230703093531.22832-1-jnhuang95@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fsck: add support for extracting hard links

Currently hard links can't be extracted correctly, let's support it now.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Fixes: 412c8f908132 ("erofs-utils: fsck: add --extract=X support to extract to path X")
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230628022558.6198-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: fallback for copy_file_range

If tmpfs is used for /tmp and blob device is not specified, we need to
copy data between two different file systems during mkfs, which is not
supported by the copy_file_range() syscall. In this case, let's give it
a second chance by fallback to __erofs_copy_file_range().

Link: https://lore.kernel.org/r/20230625080819.44502-1-zhaoyifan@sjtu.edu.cn
Fixes: e9afc0408745 ("erofs-utils: introduce copy_file_range")
Fixes: 03cbf7b8f7f7 ("erofs-utils: mkfs: support chunk-based uncompressed files")
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Yifan Zhao <zhaoyifan@sjtu.edu.cn>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: twist calculation of shared_xattr_id

The on-disk format specifies that share xattr can be addressed by:

  xattr offset = xattr_blkaddr * block_size + 4 * shared_xattr_id

That is, the shared_xattr_id is calculated from the xattr offset
(starting from xattr_blkaddr) divided by 4.  Make this semantics
explicitly by calculating the divisor from 'sizeof(__le32)'.

It has no logic change.

Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230609090225.91890-1-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: sync up erofs_fs.h

Keep in sync with kernel erofs_fs.h of commit 6a318ccd7e08
("erofs: enable long extended attribute name prefixes").

Reviewed-by: Guo Xuenan <guoxuenan@huaweicloud.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230606170144.36902-1-hsiangkao@linux.alibaba.com

erofs-utils: fsck: add a preliminary fuzzer

Let's introduce a fuzzer for fsck.erofs by using libFuzzer [1]:
 - Clang 6.0+ installed;
 - Build with "CC=clang ./configure --enable-fuzzing";
 - Set stack size by using `ulimit -s` properly;
 - fsck/fuzz_erofsfsck -timeout=40 -max_total_time=600 CORPUS_DIR.

[1] https://www.llvm.org/docs/LibFuzzer.html
[ Gao Xiang: a dedicated test job is also set up at
   https://github.com/erofs/erofsnightly/actions/workflows/fuzzers.yml ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230605170551.273399-1-xiang@kernel.org

erofs-utils: lib: error out if de_namelen is 0

de_namelen 0 is invalid for now.

Fixes: 564adb0a852b ("erofs-utils: lib: add API to iterate dirs in EROFS")
Reviewed-by: Guo Xuenan <guoxuenan@huaweicloud.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230606075125.75125-1-hsiangkao@linux.alibaba.com

erofs-utils: fsck: fix segmentation fault for file extraction

Currently, we use fsckcfg.extract_path to record the path of file
to be extracted, when the name is too long, it will exceed the
fsckcfg.extract_path[PATH_MAX] array and segmentation fault may
occur.

Test and reproduce with the following script:
``` bash
FSCK=`which fsck.erofs`
MKFS=`which mkfs.erofs`

IN_DIR=./src
$MKFS x.img ${IN_DIR}

get_dst_dir()
{
local len=$1
local perlen=$2
local dst_dir=$(printf 'a%.0s' $(seq 1 $((perlen - 1))))
local n=$((len / ${perlen}))
local lastlen=$((len - perlen * n))
local lastdir=$(printf 'a%.0s' $(seq 1 $lastlen))
local outdir=""
for x in `seq 1 $n`
do
outdir=${outdir}/${dst_dir}
done

[[ -n $lastdir ]] && outdir=${outdir}/${lastdir}
echo ${outdir}
}

for n in `seq 4000 1 5000`
do
dst_dir=$(get_dst_dir $n 255)
echo ${#dst_dir}

OUT_DIR="./${dst_dir}"
rm -rf $(dirname $OUT_DIR) > /dev/null 2>&1
mkdir -p $OUT_DIR
$FSCK --extract=${OUT_DIR} x.img > /dev/null 2>&1
done
```

Fixes: f44043561491 ("erofs-utils: introduce fsck.erofs")
Fixes: b11f84f593f9 ("erofs-utils: fsck: convert to use erofs_iterate_dir()")
Fixes: 412c8f908132 ("erofs-utils: fsck: add --extract=X support to extract to path X")
Signed-off-by: Guo Xuenan <guoxuenan@huawei.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230605125815.2835732-1-guoxuenan@huawei.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fix EOD behavior when len is too large

Otherwise, read_count could be overflow and causes unexpected
behaviors.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230603133130.34364-2-hsiangkao@linux.alibaba.com

erofs-utils: refuse block sizes larger than EROFS_MAX_BLOCK_SIZE

The maximum supported block size is EROFS_MAX_BLOCK_SIZE.

[ Gao Xiang: shifted values can also overflow and need to be fixed. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230603133130.34364-1-hsiangkao@linux.alibaba.com

erofs-utils: fsck: verify packed_nid when checking packed inode

Since dedupe feature is also using the same feature bit as fragments.

Fixes: 017f5b402d14 ("erofs-utils: fsck: add a check to packed inode")
Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/8e87974be6533d03cff7bf6af222869e7ddba015.1685700307.git.huyue2@coolpad.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: dump: verify packed_nid when reading packed inode

Since dedupe feature is also using the same feature bit as fragments.
Meanwhile, add missing dedupe feature to feature_lists[].

Fixes: a6336feefe37 ("erofs-utils: dump: support fragments")
Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/d755ab1c4eaa634f8822b6ce663c1d1a66aae09c.1685700307.git.huyue2@coolpad.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: limit pclustersize in z_erofs_fixup_deduped_fragment()

The variable 'ctx->pclustersize' could be larger than max pclustersize.

Signed-off-by: Noboru Asai <asai@sijam.com>
Reviewed-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20230602052039.615632-1-asai@sijam.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fsck: block insane long paths when extracting images

Since some crafted EROFS filesystem images could have insane deep
hierarchy (or may form directory loops) which triggers the
PATH_MAX-sized path buffer OR stack overflow.

Actually some crafted images cannot be deemed as real corrupted
images but over-PATH_MAX paths are not something that we'd like to
support for now.

CVE: CVE-2023-33551
Closes: https://nvd.nist.gov/vuln/detail/CVE-2023-33551
Reported-by: Chaoming Yang <lometsj@live.com>
Fixes: f44043561491 ("erofs-utils: introduce fsck.erofs")
Fixes: b11f84f593f9 ("erofs-utils: fsck: convert to use erofs_iterate_dir()")
Fixes: 412c8f908132 ("erofs-utils: fsck: add --extract=X support to extract to path X")
Signeo-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230602055256.18061-1-hsiangkao@linux.alibaba.com

erofs-utils: fsck: don't allocate/read too large extents

Since some crafted EROFS filesystem images could have insane large
extents, which causes unexpected bahaviors when extracting data.

Fix it by extracting large extents with a buffer of a reasonable
maximum size limit and reading multiple times instead.

Note that only `--extract` option is impacted.

CVE: CVE-2023-33552
Closes: https://nvd.nist.gov/vuln/detail/CVE-2023-33552
Reported-by: Chaoming Yang <lometsj@live.com>
Fixes: 412c8f908132 ("erofs-utils: fsck: add --extract=X support to extract to path X")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230602030519.117071-1-hsiangkao@linux.alibaba.com

erofs-utils: fix compact format for large lcluster sizes

Use compact 4B format for large lcluster sizes if possible or fall
back to the non-compact format.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230601113416.71774-1-hsiangkao@linux.alibaba.com

erofs-utils: support detecting maximum block size

Previously PAGE_SIZE was actually unset for most cases so that it was
always hard-coded 4096.

In order to set EROFS_MAX_BLOCK_SIZE correctly, let's detect the real
page size when configuring and get rid of PAGE_SIZE.

Cc: Kelvin Zhang <zhangkelvin@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230601145359.103536-1-hsiangkao@linux.alibaba.com

erofs-utils: fsck: fix segmentfault for crafted image extract

In crafted erofs image, extract files may lead to fsck.erofs
memory access out of bounds.
Actually, there is already interception in the code, but which only
take effect in debug mode, change it to avoid that.

Signed-off-by: Guo Xuenan <guoxuenan@huawei.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Fixes: 730979c061d5 ("erofs-utils: fuse: add compressed file support")
Link: https://lore.kernel.org/r/20230531072612.2643983-3-guoxuenan@huawei.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: remove hardcoded block size shifts

This improves support for non 4K block sizes

Signed-off-by: Kelvin Zhang <zhangkelvin@google.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230530202413.2734743-1-zhangkelvin@google.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: fix the endianness of erofs_super_block

Add the missing cpu_to_le[16|32]() conversion when initializing
erofs_super_block.

Fixes: 0a94653c56b2 ("erofs-utils: introduce mkfs support")
Fixes: 116ac0a254fc ("erofs-utils: introduce shared xattr support")
Fixes: a70f35adc1b0 ("erofs-utils: introduce ondisk compression cfgs")
Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230522024848.89861-1-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fsck: fix an infinite loop of big pcluster

Actually it's outdated code compared with kernel
commit b86269f43892 ("erofs: support parsing big pcluster compact indexes")

This will cause fsck.erofs works endlessly on some crafted images.

Reported-by: Chaoming Yang <lometsj@live.com>
Fixes: 418fb683fd96 ("erofs-utils: fuse: support compact indexes for bigpcluster")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230511141520.32835-1-xiang@kernel.org

erofs-utils: fix `-Ededupe` crash without fragments enabled

Otherwise, an unexpected segfault will happen since
  EROFS_FEATURE_INCOMPAT_FRAGMENTS and
  EROFS_FEATURE_INCOMPAT_DEDUPE    share the same bit.

Fixes: 18fbf7d12e4f ("erofs-utils: build xattrs upon extra long name prefixes")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230506035306.2269-1-jefflexu@linux.alibaba.com

erofs-utils: refine blobchunk implementation

 - record device_id in erofs_blobchunk for later tarerofs;

 - fix up endianness conversion;

 - blob (hash) entries should be released in the blobchunk submodule.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230421073926.85369-1-hsiangkao@linux.alibaba.com

erofs-utils: build xattrs upon extra long name prefixes

Extra long xattr name prefixes are also considered when generating
xattr entries.

Note that the given user-specified extra long xattr name prefixes
are preferred to the pre-defined xattr name prefixes.

Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230407140902.97275-3-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: introduce on-disk format for long xattr name prefixes

Besides the predefined xattr name prefixes, introduces long xattr name
prefixes, which work similarly as the predefined name prefixes, except
that they are user specified.

It is especially useful for use cases together with overlayfs like
Composefs model, which introduces diverse xattr values with only a few
common xattr names (trusted.overlay.redirect, trusted.overlay.digest,
and maybe more in the future).  That makes the existing predefined
prefixes ineffective in both image size and runtime performance.

When a user specified long xattr name prefix is used, only the trailing
part of the xattr name apart from the long xattr name prefix will be
stored in erofs_xattr_entry.e_name.  e_name is empty if the xattr name
matches exactly as the long xattr name prefix.  All long xattr prefixes
are stored in the packed or meta inode, which depends if fragments
feature is enabled or not.

For each long xattr name prefix, the on-disk format is kept as the same
as the unique metadata format: ALIGN({__le16 len, data}, 4), where len
represents the total size of struct erofs_xattr_long_prefix, followed
by data of struct erofs_xattr_long_prefix itself.

Each erofs_xattr_long_prefix keeps predefined prefixes (base_index)
and the remaining prefix string without the trailing '\0'.

Two fields are introduced to the on-disk superblock, where
xattr_prefix_count represents the total number of the long xattr name
prefixes recorded, and xattr_prefix_start represents the start offset of
recorded name prefixes in the packed/meta inode divided by 4.

When referring to a long xattr name prefix, the highest bit (bit 7) of
erofs_xattr_entry.e_name_index is set, while the lower bits (bit 0-6)
as a whole represents the index of the referred long name prefix among
all long xattr name prefixes.

Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230407140902.97275-2-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: export packedfile APIs

Later packed_inode will be used in other scenarios other than
compressed fragments. Let's separate packedfile APIs for future use.

Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230407140902.97275-1-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: xattr: skip xattrs with unidentified "system." prefix

Skip xattrs with unidentified "system." prefix to avoid ENODATA error.
Such as building AOSP on NFSv4 servers.

Signed-off-by: Weizhao Ouyang <o451686892@gmail.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230420092739.75464-1-o451686892@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: man: dump.erofs: wording/formatting touchups

Some things that gave me pause or were weirdly formatted.

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/c556cbafa7124d5ab5d20979be068ba36a125ed6.1681041325.git.nabijaczleweli@nabijaczleweli.xyz
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: man: fsck.erofs: wording/formatting touchups

Some things that gave me pause or were weirdly formatted.

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/nrjslxj7x6fufnvg4qavwm6zy5gues42wbjf23fgxpihsxyrrc@ddkzmz2usmyl
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: man: mkfs.erofs: wording/formatting touchups

Some things that gave me pause or were weirdly formatted.

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/8490d5f3cbfdd11ee2690b3e642f7cd70ac9f582.1681041325.git.nabijaczleweli@nabijaczleweli.xyz
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: drop dead code in -z parsing

-z is specified as "z:".

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/df945a64dbab234faec152a2fca75747f724d0c2.1681041325.git.nabijaczleweli@nabijaczleweli.xyz
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: rb_tree: fix broken rb_iter_init() prototype

In file included from rb_tree.c:34:
./rb_tree.h:96:17: warning: a function declaration without a prototype
is deprecated in all versions of C and is treated as a zero-parameter
prototype in C2x, conflicting with a subsequent definition
[-Wdeprecated-non-prototype]
struct rb_iter *rb_iter_init            ();
                ^
rb_tree.c:422:1: note: conflicting prototype is here
rb_iter_init (struct rb_iter *self) {
^

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/d80484200b3ba60127ff3b92e0c7660a2e8726bf.1681041325.git.nabijaczleweli@nabijaczleweli.xyz
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: xattr: avoid using inode_xattr_node for shared xattrs

Let's introduce next_shared_xattr instead to chain shared xattrs,
and it will also be used for ranged shared xattrs.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230407133805.60975-3-hsiangkao@linux.alibaba.com

erofs-utils: xattr: avoid global variable shared_xattrs_size

Let's calculate shared_xattrs_size lazily instead.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230407133805.60975-2-hsiangkao@linux.alibaba.com

erofs-utils: get rid of erofs_buf_write_bhops

`nbh->off - bh->off` in erofs_bh_flush_generic_write() is
problematic due to erofs_bdrop(bh, false).

Let's avoid generic erofs_buf_write_bhops instead.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230407140856.100602-1-hsiangkao@linux.alibaba.com

erofs-utils: xattr: declare prefix_len as u8

The on-disk erofs_xattr_entry.e_name_len is declared as u8, which
implies the maximum length of the xattr name.

Let's also declare xattr_prefix.prefix_len as u8 to keep sync with the
on-disk structure.

Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230404080224.77577-2-jefflexu@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fix nid lookup for packed inode

If inode->nid is less than 0, it should be unassigned as well.

Reported-by: Jingbo Xu <jefflexu@linux.alibaba.com>
Fixes: 21d84349e79a ("erofs-utils: rearrange on-disk metadata")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230403083819.47453-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix EOD read when calculate sb checksum

Since images could be very small for smaller block sizes.

Fixes: a4fb8ea13ac7 ("erofs-utils: support arbitrary block sizes")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230328035456.89831-2-hsiangkao@linux.alibaba.com

erofs-utils: fix missing tail blocks for directories

Currently directory blocks will be allocated after inode metadata
space is reserved, but miss to fix tail blocks.

Fixes: 21d84349e79a ("erofs-utils: rearrange on-disk metadata")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230328035456.89831-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: justify post-EOD read behavior

In the past, errors will be returned when reading post-EOD data.
Let's align this with the generic EOD behavior (zero-filling) instead.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230327025737.62488-1-hsiangkao@linux.alibaba.com

erofs-utils: separate directory data from file data

It'd better to split directory data out although directory data is
still not lazy written since it tends to put directory data next to
the corresponding inode metadata.

Laterly, aligned directory data could be written in a batch way so
that inodes can be more compact.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230324082749.1915-1-hsiangkao@linux.alibaba.com

erofs-utils: fix up nlink for d_type unsupported fses

i_mode should be used instead for some old random fs.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230319151417.20432-1-hsiangkao@linux.alibaba.com

erofs-utils: rearrange on-disk metadata

 - Use BFS instead of DFS;

 - Regular data is separated from metadata and dir data.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230317085045.16263-1-hsiangkao@linux.alibaba.com

erofs-utils: support arbitrary block sizes

 - Add a new command option for mkfs.erofs;

 - fuse/dump/fsck supports arbitrary block sizes (uncompressed files).

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230314062121.115020-4-hsiangkao@linux.alibaba.com

erofs-utils: drop hard-coded block size

Now block sizes can be less than PAGE_SIZE.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230314074251.80425-1-hsiangkao@linux.alibaba.com

erofs-utils: add per-sb block size

Will be used for subpage blocksize support.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230314062121.115020-2-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: validate chunk/pcluster sizes in the end

Laterly, erofs-utils will support sub-page block sizes and
an arbitrary block size can be given at any position of the
command line.

Therefore, chunk/pcluster sizes needs to be validated in the end.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230314062121.115020-1-hsiangkao@linux.alibaba.com

erofs-utils: fix PERFORMANCE.md typos

Trivial updates.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230315082523.10314-1-hsiangkao@linux.alibaba.com

erofs-utils: release 1.6

Signed-off-by: Gao Xiang <xiang@kernel.org>

erofs-utils: add missing help for multiple algorithms

Users can try this with the detailed instructions.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230311041245.27006-1-hsiangkao@linux.alibaba.com

erofs-utils: refine README

Add some words about userspace fragment cache as a TODO.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230310091601.97930-2-hsiangkao@linux.alibaba.com

erofs-utils: fix liblzma extreme compression levels

100 ~ 109 are now valid for LZMA extreme compression.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230310091601.97930-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix errors when building xattrs

EOPNOTSUPP could be reported by llistxattr() and mkfs.erofs could
fail out due to the following error:

<E> erofs: llistxattr to get the size of names for xxxxx failed
<E> erofs:  Could not format the device : [Error 95] Operation not supported

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230309112630.74230-3-hsiangkao@linux.alibaba.com

erofs-utils: improve documentation for upcoming 1.6

Refine README as well as add docs/INSTALL.md and docs/PERFORMANCE.md.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230309112630.74230-2-hsiangkao@linux.alibaba.com

erofs-utils: optimize dedupe matching

Match in words instead of byte granularity.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230309112630.74230-1-hsiangkao@linux.alibaba.com

erofs-utils: dedupe more in the packed inode if possible

In addition to just compare the in-memory data.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230304195812.120063-5-hsiangkao@linux.alibaba.com

erofs-utils: handle mmap failure when packing a whole file

Directly read/write if mmap fails.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230304195812.120063-4-hsiangkao@linux.alibaba.com

erofs-utils: avoid using a static srcpath

To avoid unnecessary memory overhead.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230304195812.120063-3-hsiangkao@linux.alibaba.com

erofs-utils: switch sha256 algorithm directly from LibTomCrypt

LibTomCrypt was released into public domain so that we don't need
to worry about license issues against Apache-2.0 later.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230304195812.120063-2-hsiangkao@linux.alibaba.com

erofs-utils: fix delta[1] out-of-bound of compact indexes

delta[1] should not exceed (1 << lclusterbits) - 1, usually 4095
blocks, which means a single compressed extent which isn't less
than 16MiB could trigger this issue (it very rare happens).

Fixes: 76b822726ff8 ("erofs-utils: introduce compacted compression indexes")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230304195812.120063-1-hsiangkao@linux.alibaba.com

erofs-utils: validate the extent length for uncompressed pclusters

Keep in sync with the kernel commit c505feba4c0d ("erofs: validate the
extent length for uncompressed pclusters"), so that we can catch the
issue as well in fuse.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20230303080743.25713-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: add `-Eall-fragments` option

It's almost the same as `-Efragments` option, except that will
explicitly pack the whole data into the special inode instead.

Tested-by: Yue Hu <huyue2@coolpad.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230228185459.117762-3-hsiangkao@linux.alibaba.com

erofs-utils: use compressed pclusters to mark fragments

The decoded lengths of uncompressed pclusters should be
strictly no more than encoded lengths.

Fixes: 9fa9b017f773 ("erofs-utils: mkfs: support fragments")
Reviewed-by: Yue Hu <huyue2@coolpad.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230228185459.117762-2-hsiangkao@linux.alibaba.com

erofs-utils: get rid of useless nr_dup

Also refine the longest detection.

Fixes: 990c7e383795 ("erofs-utils: mkfs: support fragment deduplication")
Reviewed-by: Yue Hu <huyue2@coolpad.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230228185459.117762-1-hsiangkao@linux.alibaba.com

erofs-utils: don't warn ztailpacking feature anymore

The ztailpacking feature has been merged for a year, it has been mostly
stable now. Let's drop such warning now.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20230227095311.13033-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: manpage: add -Ededupe option

Complete the manpage for global compressed data deduplication feature.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20230225142707.28408-1-zbestahu@163.com
[ Gao Xiang: refine manpage description. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: manpage: add fragments extended option

Complete the manpage for fragments feature.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20230225125133.26165-1-zbestahu@163.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: support randomizing algorithms in debugging mode

It's used for multiple algorithms selftest.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230222191148.112677-3-hsiangkao@linux.alibaba.com

erofs-utils: support multiple algorithms in a single image

Some binaries in an image may be just for archival purposes only
so that the runtime performance is not the top priority at all.

Therefore, it'd better to use another algorithm with higher
compression ratios and (even) pack the whole file into the packed
inode entirely (will enable this way later.)

In order to use alternative algorithms, just specify two or more
compressing configurations together separated by ':' like below:
   -zlzma:lz4hc,12:lzma,9 -C32768

Although mkfs still choose the first one by default, you could try
to write a compress-hints file like below:
   4096  1 .*\.so$
   32768 2 .*\.txt$
   4096    sbin/.*$
   16384 0 .*

So that ".so" files will use "lz4hc,12" compression with 4k pclusters,
".txt" files will use "lzma,9" compression with 32k pclusters, files
under "/sbin" will use the default "lzma" compression with 4k plusters
and other files will use "lzma" compression with 16k pclusters.

Note that the largest pcluster size should be specified with the "-C"
option, otherwise all larger pclusters will be limited.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230222191148.112677-2-hsiangkao@linux.alibaba.com

erofs-utils: ignore comments and blank lines in compress hints

Identify lines starting with '#' as a comment.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230222191148.112677-1-hsiangkao@linux.alibaba.com

erofs-utils: get rid of PAGE_SIZE leftovers

commit c47df5aa2d16 ("erofs-utils: fuse: introduce xattr support")
introduced two new references.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230222190814.102585-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: don't skip `lost+found`

It's somewhat strange to skip the `lost+found` directory.

Fixes: a17497f0844a ("erofs-utils: introduce inode operations")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230209033524.38234-1-hsiangkao@linux.alibaba.com

erofs-utils: support chunk-based sparse files

Scan holes for chunk-based inodes if either --chunksize=# or
-Ededupe (without compression) is specified so that sparse files
can be made, which has already been supported since Linux 5.15.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230131051454.47719-1-hsiangkao@linux.alibaba.com

erofs-utils: fix chunk-based image handling without real data

Otherwise it will report:
 <I> erofs: total metadata: 982 blocks
 <E> erofs:  Could not format the device : [Error 5] Input/output error

Fixes: 03cbf7b8f7f7 ("erofs-utils: mkfs: support chunk-based uncompressed files")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230131094808.67525-1-hsiangkao@linux.alibaba.com

erofs-utils: fuse: fix warnings on MacOS

Reported by the latest nightly build:
https://github.com/erofs/erofsnightly/actions/runs/4050082667/jobs/6969376259

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230131084805.2494-1-hsiangkao@linux.alibaba.com

erofs-utils: fsck: add a check to packed inode

Add a check to packed inode for fsck.erofs.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/ce29198d32f284c19589877c4c9e4e1588ad77b3.1673401718.git.huyue2@coolpad.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fsck: cleanup erofs_verify_inode_data()

Diretly call {z_}erofs_read_one_data() to avoid duplicated code.
Accordingly, fragment and partial-referenced plusters are also supported
after this change.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/115e61fc9c2d34cab6d3dd78383ac57c94a491fc.1673401718.git.huyue2@coolpad.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: export parts of erofs_pread()

Export parts of erofs_pread() to avoid duplicated code in
erofs_verify_inode_data(). Let's make two helpers for this.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/ff560da9c798b2ca1f1a663a000501486d865487.1673401718.git.huyue2@coolpad.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: dump: support fragments

Add compressed fragments support for dump feature.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20230105021343.23419-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: dump: cleanup update_file_size_statatics()

The statistic update of occupied_size is the same as original size.
Also, correct naming to update_file_size_statistics().

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/0e4797ef5f22ac3c2134aa4b005c489c233d2eec.1671443064.git.huyue2@coolpad.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: dump: avoid printing `Extent size` field

inode.extent_isize is only meaningful during mkfs for now.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20230104073019.90530-1-hsiangkao@linux.alibaba.com

erofs-utils: fuse: change to use erofs_pread to read fragment

Packed inode may be uncompressed as well due to no space.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20221224093851.10095-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: do not deduplicate compressed data for packed inode

Packed inode is composed of fragments which have already been
deduplicated before.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20221209035337.26998-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fix fragmentoff overflow for large packed inode

The return value of ftell() is a long int, use ftello{64} instead. Also,
need to return at once if it fails. Meanwhile, use lseek64 for large
file position as well to avoid this error if we have it.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20221213091746.16683-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: support fragment deduplication

Previously, there's no fragment deduplication when this feature is
introduced.  Let's support it now.

Fragments are deduplicated before compression, so that duplicated
parts will not be written into the packed inode again.

Dataset:               linux 5.10 + 5.10.50 + 5.10.100
Compression algorithm: lz4hc,12

Additional options:    -T0 --force-uid=1000 --force-gid=1000
              (in order to force 32-byte inodes to match squashfs)

      4k  pcluster + fragment + dedupe  397168640
      8k  pcluster + fragment + dedupe  364224512
     16k  pcluster + fragment + dedupe  341921792
     32k  pcluster + fragment + dedupe  328298496
     64k  pcluster + fragment + dedupe  324694016
    128k  pcluster + fragment + dedupe  323674112
    256k  pcluster + fragment + dedupe  322011136

squashfs-tools 4.5.1 test results (which uses level 12 by default
for lz4hc):
     16k  block                         428785664
     32k  block                         382894080
     64k  block                         350179328
    128k  block                         327073792
    128k  block + noI                   334327808
    256k  block                         315441152
    256k  block + noI                   322707456
      1m  block                         307425280
      1m  block + noI                   314712064

If the compressed block size is large, the image size of squashfs
will benefit from its simple _unseekable_ data block indexes:

  location = block_start
  for i = 0; i < index; i++
      location += block_sizes[i] & 0x00FFFFFF

and its fragment lookup table.  EROFS can improve such cases later.

Suggested-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20221208084859.24190-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fsck: support interlaced uncompressed pcluster

Support uncompressed data layout with on-disk interlaced offset in
compression mode for fsck.erofs.

Signed-off-by: Yue Hu <huyue2@coolpad.com>
Link: https://lore.kernel.org/r/20221221084842.28196-1-zbestahu@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: internal.h: Make LFS mandatory for all usecases

erofs depends on the consistent use of a 64bit offset
type, force downstreams to use transparent LFS (_FILE_OFFSET_BITS=64),
so that it becomes impossible for them to use 32bit interfaces.

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Link: https://lore.kernel.org/r/20221215085842.130804-3-raj.khem@gmail.com
[ Gao Xiang: subject title fixup. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: replace [l]stat64 by equivalent [l]stat

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Link: https://lore.kernel.org/r/20221215085842.130804-2-raj.khem@gmail.com
[ Gao Xiang: subject title fixup and code rebase. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: configure: Use 64bit off_t

Pass -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 via CFLAGS
this enabled large file support on 32bit architectures

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Link: https://lore.kernel.org/r/20221215085842.130804-1-raj.khem@gmail.com
[ Gao Xiang: subject title fixup. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>