erofs-utils: mkfs: support `-Efragdedupe=inode`

If the entire inode can be deduplicated against an existing fragment,
simply reuse it.

Multi-threading can still be applied for `-Efragdedupe=inode` with
the current codebase:

Fedora Linux 39 (Workstation Edition) LiveCD results:
 -zlzma,level=6,dictsize=131072 -C65536 -Eall-fragments

   `-E^fragdedupe`         2,003,587,072 bytes (1911 MiB)
   `-Efragdedupe=inode`    1,970,577,408 bytes (1880 MiB)

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20250103024011.198163-1-hsiangkao@linux.alibaba.com

erofs-utils: release 1.8.4

Signed-off-by: Gao Xiang <xiang@kernel.org>

erofs-utils: fix -Ededupe crash without fragments enabled

The root cause is the same as commit d9baceba7026 ("erofs-utils:
fix -Ededupe crash without fragments enabled").

In fact, the fragment manager should be reworked in the next major
erofs-utils version to avoid those dirty hacks.

Fixes: fc880e31b7c7 ("erofs-utils: mkfs: minor cleanup & rearrangement")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20250102113418.3246744-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix `-Eall-fragments` for multi-threaded compression

`may_packing` in __z_erofs_compress_one() is still bypassed when
multi-threaded compression is enabled, which is unexpected.

Furthermore, multi-threaded `-Eall-fragments,ztailpacking` can
sometimes corrupt images.  Let's fix it.

Fixes: 882ad1c3157f ("erofs-utils: mkfs: fix `-Eall-fragments` for multi-threaded compression")
Fixes: 10c1590c0920 ("erofs-utils: enable multi-threaded support for `-Eall-fragments`")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20250102024225.2433419-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: allow disabling fragment deduplication

Currently, although fragment inode compression is already
multi-threaded, the data parts of inodes prior to their own fragments
are still single-threaded if fragment deduplication is on.  This can
greatly slow down `-Eall-fragments` image building at least for
the current mkfs codebase.

Let's add an extended option `-E^fragdedupe` to explicitly disable it.

After this commit, the Fedora Kiwi builds I'm testing can be reduced
from 1148s (3,096,842,240 bytes, 2.9G) to 137s (2,969,956,352 bytes,
2.8G) with `-Eall-fragments,^fragdedupe -C524288 -z lzma,level=6,
dictsize=524288` on Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz with
32 cores.

Cc: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241223094031.1534175-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: speed up uncompressed data handling

Currently, it only writes one uncompressed block (typically 4 KiB) and
then tries to recompress the remaining part if the entire data was
compressed and proven to be incompressible.

This is wasteful of CPU resources and slow, as such incompressible data
could be skipped entirely.

The LZMA build time for large pclusters (e.g., 512K, 1M) can be greatly
reduced if there is a significant amount of uncompressed data.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241220143859.643175-2-hsiangkao@linux.alibaba.com

erofs-utils: lib: get rid of `len` in z_erofs_compress_one()

More than one extent could be emitted in __z_erofs_compress_one()
later.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241220143859.643175-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: tar: fix LIBARCHIVE.xattr URL-encoded keys

As tar(5) [1] mentioned: `The key value is URL-encoded: All non-ASCII
characters and the two special characters "=" and "%" are encoded as
"%" followed by two uppercase hexadecimal digits.`

Fix it now.

[1] https://man.freebsd.org/cgi/man.cgi?tar(5)

Fixes: c0063a73b01b ("erofs-utils: lib: support importing xattrs from tarerofs")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241226045808.95101-3-hsiangkao@linux.alibaba.com

erofs-utils: lib: tar: fix LIBARCHIVE.xattr base64 decoding

Base64 is described in RFC1521 [1], except that LIBARCHIVE.xattr base64
may not have padding at the end of the data using the '=' character.

[1] https://datatracker.ietf.org/doc/html/rfc1521#section-5.2

Fixes: c0063a73b01b ("erofs-utils: lib: support importing xattrs from tarerofs")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241226045808.95101-2-hsiangkao@linux.alibaba.com

erofs-utils: lib: tar: ignore useless fields of PAX extended headers

Since some unknown writer just leaves zero-filled mtime, e.g.
registry.k8s.io/pause:3.6  --platform windows

Layer sha256: bc8517709e9cfff223cb034ff5be8fcbfa5409de286cdac9ae1b8878ebea6b84

Fixes: 95d315fd7958 ("erofs-utils: introduce tarerofs")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241226045808.95101-1-hsiangkao@linux.alibaba.com

erofs-utils: use external xxhash library if possible

It's expected to be faster than the internal one.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241220144139.648210-1-hsiangkao@linux.alibaba.com

erofs-utils: release 1.8.3

Signed-off-by: Gao Xiang <xiang@kernel.org>

erofs-utils: lib: correct erofsfuse build script

Some of the symbols required by erofsfuse are provided by liberofs.
When option 'enable-static-fuse' is set, all these object file should be
exported to liberofsfuse.a

Signed-off-by: ComixHe <heyuming@deepin.org>
Link: https://lore.kernel.org/r/8725A28257A20420+20241213063250.314786-1-heyuming@deepin.org
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: add --hard-dereference option

Add option --hard-dereference to dereference hardlinks when
creating an image. Instead of reusing the inode, hardlinks are added
as separate inodes. This is useful for reproducible builds, when the
rootfs is space-optimized using hardlinks on some machines, but not on
others.

Co-authored-by: Leonard Cohnen <leonard.cohnen@gmail.com>
Signed-off-by: Paul Meyer <katexochen0@gmail.com>
Link: https://lore.kernel.org/r/20241212165550.58756-1-katexochen0@gmail.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: add `-U <clear|random>` support

To match `mke2fs`.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241211092704.4008111-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: get rid of pthread_cancel() for workqueue

Since bionic (Android's libc) does not have pthread_cancel(),
call erofs_destroy_workqueue() when initialization fails.

Reviewed-by: Yifan Zhao <zhaoyifan@sjtu.edu.cn>
Cc: Kelvin Zhang <zhangkelvin@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241211025009.3393476-1-hsiangkao@linux.alibaba.com

erofs-utils: use pkg-config for lz4 configuration

Also obsolete those `LZ4_HC_STATIC_LINKING_ONLY` versions since it's
too complicated to be maintained.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241205030846.70656-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: add missing dependencies

Add the missing ${libdeflate_CFLAGS} and ${libzstd_CFLAGS} to avoid
compilation errors.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241205020915.966196-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: make output stable

The iteration order of opendir/readdir depends on filesystem
implementation. Initializng inode->i_ino[0] in the loop causes the
output unstable even though the entries are sorted in
erofs_prepare_dir_file().

In this change,  inode->i_ino[0] is initialized in
erofs_prepare_inode_buffer() instead to make the output stable. (not
affected by readdir())

  # should generate the same output

Test: mkfs.erofs ... inputdir(ext3)
Test: mkfs.erofs ... inputdir(tmpfs)
Signed-off-by: Jooyung Han <jooyung@google.com>
Link: https://lore.kernel.org/r/20241203074531.3728133-1-jooyung@google.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: fix user-after-free in xattr.c

Currently, put_xattritem() does not remove the `item`
from the hash table after freeing it, which may lead
to a user-after-free issue.

This patch fixes it.

Fixes: 47d6895a5ff9 ("erofs-utils: introduce inline xattr support")
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20241202024019.85901-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: rebuild: set the appropriate `dev` field for dirs

Currently, setting a default `dev` value (i.e., 0) for directories
during parsing tar files can lead to the following error:

<E> erofs: bogus i_mode (0) @ nid 0
<E> erofs: failed to read inode @ 0

Consider the following incremental build scenario, where the path
"dir1/dir2" is currently being parsed in tarerofs_parse_tar() and
the directory "dir1" has never appeared before.
`erofs_rebuild_get_dentry()` will call erofs_rebuild_mkdir() to
allocate a new inode for "dir1", with its `dev` field set to 0 by
default.

During the dump tree phase, since `dir1->dev` matches `sbi->dev`
(both are 0), erofs_rebuild_load_basedir() will be called to read
the contents of directory "dir1" from the disk. However, since
there is no information for the new directory "dir1" on the disk,
the above error occurs.

This patch resolves the above issue by setting the appropriate value
for the directory's `dev` field during the tar file parsing phase.

Fixes: f64d9d02576b ("erofs-utils: introduce incremental builds")
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241127111346.49290-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: avoid silent corruption caused by `c_root_xattr_isize`

When `c_root_xattr_isize` is too large, `i_xattr_icount` will overflow,
resulting in silent corruption of the filesystem image. This patch performs
checks in advance and reports errors.

Fixes: 8f93c2f83962 ("erofs-utils: mkfs: support inline xattr reservation for rootdirs")
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241127092825.4105724-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: clean up z_erofs_load_full_lcluster()

Let's keep in sync with kernel commit d69189428d50 ("erofs: clean up
z_erofs_load_full_lcluster()").

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241121023517.581040-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: clean up zmap.c

Let's keep in sync with kernel commit 8241fdd3cdfe ("erofs: clean up
zmap.c").

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241119085427.1672789-1-hsiangkao@linux.alibaba.com

erofs-utils: fix `Not a directory` error for incremental builds

If an incremental layer contains a directory but the same path in
the base layer is a non-directory, it will fail unexpectedly.

Fix it now.

Reported-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Co-developped-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Fixes: f64d9d02576b ("erofs-utils: introduce incremental builds")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241122075659.2869515-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: capture errors from {mkfs,rebuild}_handle_inode()

Currently, the error code returned by erofs_{mkfs,rebuild}_handle_inode()
in erofs_mkfs_dump_tree() may be ignored. This patch introduces `err2` to
capture errors from {mkfs,rebuild}_handle_inode().

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241112020830.849210-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: rearrange struct erofs_configure

Move the fields controlled by the NDEBUG macro to the end
to maintain a consistent layout for preceding variables.

It addresses cases where a third-party application does not define
NDEBUG while erofs-utils does.  Ideally, third-party applications
should use the same macros as erofs-utils to get a unique
`struct erofs_configure`. However, since NDEBUG enables unnecessary
assertions, restructuring the layout resolves such inconsistencies.

Fixes: ad6c80dc168d ("erofs-utils: lib: add erofs_get_configure()")
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241111095715.3814956-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: Fix input offset counting in headerball mode

When using --tar=headerball, most files included in the headerball are
not included in the EROFS image. mkfs.erofs typically exits prematurely,
having processed non-USTAR blocks as USTAR and believing they are
end-of-archive markers. (Other failure modes are probably also possible
if the input stream doesn't look like end-of-archive markers at the
locations that are being read.)

This is because we lost correct count of bytes that are read from the
input stream when in headerball (or ddtaridx) modes. We were assuming that
in these modes no data would be read following the ustar block, but in
case of things like PAX headers, lots more data may be read without
incrementing tar->offset.

This corrects by always incrementing the offset counter, and then
decrementing it again in the one case where headerballs differ -
regular file data blocks are not present.

Signed-off-by: Mike Baynton <mike@mbaynton.com>
Link: https://lore.kernel.org/r/20241111164819.560567-1-mike@mbaynton.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: avoid allocating large arrays on the stack

The default pthread stack size of bionic is 1M. Use malloc to avoid
stack overflow.

Signed-off-by: Jianan Huang <huangjianan@xiaomi.com>
Link: https://lore.kernel.org/r/20241025015246.649209-1-huangjianan@xiaomi.com
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: report leftovers for partially filled blocks

3rd-party applications may need the exact zeroed size for each
partial filesystem block to populate remote data.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241018062417.2297930-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix unexpected errors for chunk-based images

mkfs.erofs may fail because copy_file_range() returns the number of
bytes which could be less than the length originally requested.

Fixes: 03cbf7b8f7f7 ("erofs-utils: mkfs: support chunk-based uncompressed files")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241016074849.2862282-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix `-Eall-fragments` for multi-threaded compression

If `-Eall-fragments` is specified when multi-threaded compression is
enabled, it should only apply to the packed inode instead of other
real inodes for now.

Fixes: 10c1590c0920 ("erofs-utils: enable multi-threaded support for `-Eall-fragments`")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241012035213.3729725-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: Explicitly include <pthread.h> where used

compress.c and inode.c use pthread functions but do not explicitly include <pthread.h>.
This causes build failures with the Android build system,
which throws "error: implicit declaration of function pthread_*".

Signed-off-by: Satoshi Niwa <niwa@google.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241008060819.2442945-1-niwa@google.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: release 1.8.2

Signed-off-by: Gao Xiang <xiang@kernel.org>

erofs-utils: lib: fix compressed packed inodes

Commit 2fdbd28 fixed uncompressed packed inodes by not always writing
compressed data, but it broke compressed packed inodes because now
uncompressed file data is always written after the compressed data.

The new error handling always rewinds with lseek and falls through to
write_uncompressed_file_from_fd, regardless of whether the compressed
data was written successfully (ret = 0) or not (ret = -ENOSPC). This
can result in corrupted files.

Fix it by simplifying the error handling to better match the old code.

Fixes: 2fdbd28 ("erofs-utils: lib: fix uncompressed packed inode")
Co-authored-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Danny Lin <danny@orbstack.dev>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240923051759.7563-2-danny@orbstack.dev
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: add `--sort=none`

Currently, `--tar=f` writes file data twice due to unseekable streams
and EROFS data ordering requirements.  Some use cases may need to avoid
unnecessary data writes for performance and do NOT require a strict
data ordering.

It adds `--sort=none` to address this.  The image is still reproducible;
it simply means no specific file data ordering is implied.

Currently, It comes into effect if `-E^inline_data` is specified and no
compression is applied.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240923074929.2445674-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: get rid of outdated subpage compression warning

It has been supported since Linux 6.8.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240919105519.3656878-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix a regression where rebuild mode does not work

It will fail unexpectedly if any sub-image lacks an extra device (blob).

Fixes: 7550a30c332c ("erofs-utils: enable incremental builds")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240919021635.2922387-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix off-by-one issue with invalid device ID

The device ID should be no less than `1 + number of blobs`. In other
words, it should be greater than `number of blobs`.

Fixes: 89dfe997c2ee ("erofs-utils: lib: fix global-buffer-overflow due to invalid device")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240917091115.3920734-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix sorting shared xattrs

The length of xattr_item->kvbuf is calculated by EROFS_XATTR_KVSIZE,
and the key part has a trailing '\0' before the value part. When qsort
compares two xattr_items, the key-value length should be calculated by
EROFS_XATTR_KVSIZE, and use memcmp instead of strncmp to avoid key-value
string being cut by '\0'.

Fixes: 5df285cf405d ("erofs-utils: lib: refactor extended attribute name prefixes")
Signed-off-by: Sheng Yong <shengyong@oppo.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240913143542.3265071-1-shengyong@oppo.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fsck: introduce exporting xattrs

Currently `fsck --extract` does not support exporting
extended attributes. This patch adds the `--xattrs` option
to dump extended attributes and the `--no-xattrs` option to
omit them (the default behavior).

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240913072100.577753-2-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: expose erofs_xattr_prefix_matches()

Prepare for the feature of exporting extended attributes for
`fsck.erofs`.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240913072100.577753-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: fix incorrect nblocks in block list for chunked inodes

Currently, the number of physical blocks (nblocks) for the last chunk
written to the block list file is incorrectly recorded as the inode
chunksize.

This patch writes the actual number of physical blocks for the inode in
the last chunk to the block list file.

Fixes: 7b46f7a0160a ("erofs-utils: lib: merge consecutive chunks if possible")
Fixes: b6749839e710 ("erofs-utils: generate preallocated extents for tarerofs")
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240911085531.2133723-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: use another way to check power-of-2

Otherwise, Coverity will complain with "Operands don't affect result
(CONSTANT_EXPRESSION_RESULT)", which I don't think is an issue.

Coverity-id: 508261
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240910075256.3423180-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix inaccurate assertion of hardlinks in rebuild mode

`erofs_parent_inode(inode) == dir` is only true for non-hardlink
inodes and directories.

Only debug builds are impacted.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240909111305.1850391-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: tar: allow pax headers with empty names

Usually, `exthdr.name`s are set as "%d/PaxHeaders.%p/%f" [1].

However, since both `GNU tar` and `bsdtar` can process empty
`exthdr.name`s, let's allow them too.

[1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html

Reported-by: Colin Walters <walters@verbum.org>
Fixes: 95d315fd7958 ("erofs-utils: introduce tarerofs")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240909093354.1712460-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix an undefined behavior of memcpy

Call trace:
 - erofs_mkfs_build_tree
  - erofs_mkfs_go(sbi, ~0, NULL, 0);

inode.c:1395:20: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior inode.c:1395:20 in

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240906083651.341555-1-hsiangkao@linux.alibaba.com

erofs-utils: fix invalid argument type in erofs_err()

Fix several issues found by Coverity regarding "Invalid type in argument
for printf format specifier".

Coverity-id: 502374, 502367, 502362, 502348, 502342, 502341,
     502340, 502358
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240830100354.2093735-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: don't include <lzma.h> and <zlib.h> in external headers

Applications don't need internal header dependencies.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240830065542.94908-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: actually skip the unidentified xattrs

parse_one_xattr() will return null if it detects unidentified xattr.
In such cases we need to skip this xattr which was our intention than
try to add it in erofs_xattr_add() which results in null pointer
dereference.

Fixes: 3037f8958f3b ("erofs-utils: skip all unidentified xattrs from local paths")
Signed-off-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240820210123.2684886-2-dhavale@google.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: mkfs: fix an indefinite wait race

Coverity reports: In erofs_mkfs_flushjobs, a thread waits for
a thread-shared condition that may have already been satisfied,
causing a hang.

It might indeed happen if the dfops worker runs with the specific
timing.  Let's try to fix it and see if the report is cleared.

Coverity-id: 502330
Fixes: 37e5abcd8720 ("erofs-utils: mkfs: assign root NID in the main thread")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240821034326.2464146-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix potential overflow issue

Change 8U to 8ULL to avoid arithmetic multiplication overflow.

Coverity-id: 502353
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240813121023.781122-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: adjust volume label maximum length to the kernel implementation

The erofs implementation of kernel has limitation of the volume label length.
The volume label data size of super block is 16 bytes.
The kernel implementation requires to null terminate inside a that 16 bytes.

Logs:
  $ ./mkfs/mkfs.erofs test16.erofs -L 0123456789abcdef test/
  $ mount -o loop ./test16.erofs ./mnt/
  $ dmesg
  [26477.019283] erofs: (device loop0): erofs_read_superblock: bad volume name without NIL terminator

  $ ./mkfs/mkfs.erofs test15.erofs -L 0123456789abcde test/
  $ mount -o loop ./test15.erofs ./mnt/
  $ dmesg
  [26500.516871] erofs: (device loop0): mounted with root inode @ nid 36.

This patch adjusts volume label maximum length to the kernel implementation.

Signed-off-by: Naoto Yamaguchi <naoto.yamaguchi@aisin.co.jp>
Link: https://lore.kernel.org/r/20240816031601.45848-1-naoto.yamaguchi@aisin.co.jp
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: use $EROFS_UTILS_VERSION, if set, as the version

This lets downstreams embed the unpolluted downstream version;
when building the Debian package (from gbp) the resulting binary yields
  $ mkfs.erofs
  <E> erofs: missing argument: FILE
  mkfs.erofs 1.8.1-fead89d91-dirty
  Try 'mkfs.erofs --help' for more information.

Now, d/rules can
  export EROFS_UTILS_VERSION := $(shell IFS="$$IFS()" read -r _ v _ < debian/changelog; echo "$$v")
yielding
  $ mkfs.erofs
  <E> erofs: missing argument: FILE
  mkfs.erofs 1.8.1-1
  Try 'mkfs.erofs --help' for more information.

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Link: https://lore.kernel.org/r/gho2b67qax222ewv5xb5cjkkgjgzftr3pyecl536g6jshcfexa@tarta.nabijaczleweli.xyz
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: exclude: #include PATH_MAX workaround

Fixes build on the hurd.

Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Link: https://lore.kernel.org/r/xs4azw3vs7oryqnpkvzsl6qbmma6p646igoklia2fextt6pdiw@tarta.nabijaczleweli.xyz
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: release 1.8.1

Signed-off-by: Gao Xiang <xiang@kernel.org>

erofs-utils: lib: fix heap-buffer-overflow on read

Wrap up into kite_mf_hc3_skip() for now.

Fixes: 861037f4fc15 ("erofs-utils: add a built-in DEFLATE compressor")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240809161536.3961647-1-hsiangkao@linux.alibaba.com

erofs-utils: fuse: fix partial decompression for libdeflate

Actually, libdeflate doesn't support partial decompression; therefore,
fix it by reallocating larger decompression buffers.

Although a better approach would be to obtain the exact decompressed
length instead for libdeflate decompressor, which requires more changes,
a quick fix is needed.

Fixes: 29b9e7140162 ("erofs-utils: fuse,fsck: add DEFLATE algorithm support")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240809105636.3641536-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix truncated uncompressed files

Some uncompressed files which are more than 4GiB can be truncated
incorrectly.

Fixes: 358177730598 ("erofs-utils: optimize write_uncompressed_file_from_fd()")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240809033747.3178912-1-hsiangkao@linux.alibaba.com

erofs-utils: release 1.8

Signed-off-by: Gao Xiang <xiang@kernel.org>

erofs-utils: lib: fix global-buffer-overflow due to invalid device

Fuzzer generates an image with crafted chunks of some invalid device.
Also refine the printed message of EOD.

Closes: https://github.com/erofs/erofsnightly/actions/runs/10172576269/job/28135408276
Closes: https://github.com/erofs/erofs-utils/issues/11
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240808160343.2544426-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: add `--mkfs-time` option

Some users need a fixed build time in the superblock for reproducible
builds rather than a fixed timestamp everywhere.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240808094522.2161075-1-hsiangkao@linux.alibaba.com

erofs-utils: update README for the upcoming 1.8

Add descriptions to multi-threaded compression and reproducible builds.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240808050818.1822583-1-hsiangkao@linux.alibaba.com

erofs-utils: enable multi-threaded support for `-Eall-fragments`

Since `-Eall-fragments` packs the whole data into the special inode,
it's possible to use the multi-threaded compression for this.

Some users may be interested in `-Eall-fragments` for extreme
compression anyway.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240807085413.717066-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix potential overflow issue

Coverity-id: 502377
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240805032510.2637488-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: disallow new algorithms on incremental builds

On-disk compression configurations are not rewritten on incremental
builds, therefore there is no way to add new algorithms in this mode.

Clean builds should be used instead.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240806000859.66658-1-xiang@kernel.org

erofs-utils: mkfs: fix uninitialized nblocks

Coverity-id: 502376
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240805073953.2864289-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fix invalid argument type in erofs_err()

Coverity-id: 502378
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240805024408.2598464-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: fix missing argument to erofs_err()

Fixes: cdd34def670b ("erofs-utils: mkfs: introduce rebuild mode")
Coverity-id: 502380
Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240805022826.2581887-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: fix potential memory leak in erofs_export_xattr_ibody()

Although it won't happen in reality except for bugs.

Fixes: 8f93c2f83962 ("erofs-utils: mkfs: support inline xattr reservation for rootdirs")
Coverity-id: 507395
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240802062316.2368403-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix fd leak on failure in erofs_dev_open()

Coverity-id: 502356
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240802015527.2113797-3-hsiangkao@linux.alibaba.com

erofs-utils: lib: fix out-of-bounds in erofs_io_xcopy()

Coverity-id: 502334
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240802015527.2113797-2-hsiangkao@linux.alibaba.com

erofs-utils: fsck: fix fd leak on failure in erofs_extract_file()

Ignore the return values as other close()s instead.

Coverity-id: 502331
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240802015527.2113797-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: uuid: fix compilation error if __NR_getrandom doesn't exist

Let's use rand() for insecure randomness without getrandom() support
on very old kernels to resolve issues as below:

uuid.c: In function ‘s_getrandom’:
uuid.c:44:32: error: ‘__NR_getrandom’ undeclared (first use in this function); did you mean ‘s_getrandom’?
   44 |   ssize_t r = (ssize_t)syscall(__NR_getrandom, out, size, flags);
      |                                ^~~~~~~~~~~~~~
      |                                s_getrandom

I'm not sure who cares since most users just use `--with-uuid` instead.

Fixes: 5de439566bc5 ("erofs-utils: Provide identical functionality without libuuid")
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240729112524.930460-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: support inline xattr reservation for rootdirs

Due to the current on-disk limitation (16-bit on-disk root_nid), on-disk
root inodes must be updated in place for now.

If rootdir xattr sizes are expanded during incremental updates, there
may be insufficient space to keep additional extended attributes.

To work around this, let's add a mkfs option `--root-xattr-isize=#` to
specify the minimum inline xattr size of root directories in advance.

Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240729075027.712339-2-hsiangkao@linux.alibaba.com

erofs-utils: lib: allow xattr e_name_index as 0

Since it's implicitly supported since the initial EROFS kernel version.

It is particularly useful as an on-disk inode xattr placeholder for
on-disk core inode in-place updates, especially for root inodes because
"root_nid" recorded in the on-disk superblock is limited to 16 bits
for now.

Fixes: 1e429b74bff8 ("erofs-utils: lib: fix potential out-of-bound in xattr_entrylist()")
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240729075027.712339-1-hsiangkao@linux.alibaba.com

erofs-utils: misc: Fix potential memory leak in realloc failure path

As realloc returns NULL on failure, the original value will be
overwritten if it is used as lvalue. Fix this by using a temporary
variable to hold the return value and exit with -ENOMEM in case of
failure. This patch fixes 2 of the realloc blocks with similar fix.

Signed-off-by: Sandeep Dhavale <dhavale@google.com>
Link: https://lore.kernel.org/r/20240718202204.1224620-1-dhavale@google.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: manpage: add more description for --extract option

Especially, extract files to a specific directory.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240715033829.2338056-1-hsiangkao@linux.alibaba.com

erofs-utils: lib: tar: fix garbage ns timestamps

Some "#if" directives were used incorrectly.

Fixes: 95d315fd7958 ("erofs-utils: introduce tarerofs")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240714044119.1119717-1-hsiangkao@linux.alibaba.com

erofs-utils: mkfs: fix -U option

`-U <UUID>` option cannot work properly now.

Fixes: 7550a30c332c ("erofs-utils: enable incremental builds")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240712093808.2986196-2-hsiangkao@linux.alibaba.com

erofs-utils: fix reproducible builds for multi-threaded libdeflate

`last_uncompressed_size` should be reset on the basis of segments.

Fixes: 830b27bc2334 ("erofs-utils: mkfs: introduce inner-file multi-threaded compression")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240712093808.2986196-1-hsiangkao@linux.alibaba.com

erofs-utils: add per-sbi buffer support

It updates all relevant function definitions and callers to get rid of
the global g_sbi, which will be used for external use.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240710082906.203180-2-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib/cache.c: replace &g_sbi with sbi

Prepare for the upcoming per-sbi buffers.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240710082906.203180-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: tar: support ddtaridx format informally

`ddtaridx` is a customized tar meta-only format implemented in
Alibaba's OverlayBD project [1].

Please don't use it externally if you have no idea of this except for
the OverlayBD project.  It will be removed if some better way exists.

[1] https://github.com/containerd/overlaybd

Cc: Yifan Yuan <tuji.yyf@alibaba-inc.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240713064028.4134602-1-hsiangkao@linux.alibaba.com

erofs-utils: fix bitops fls_long()

`__builtin_clz` is for unsigned int, while it is now applied to
unsigned long. This fixes it by using `__builtin_clzl`.

`roundup_pow_of_two()` in `erofs_init_devices()` could give wrong
results although the current compiler optimization level "-O2"
covers it up.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240709073819.3061805-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: rename the global sbi to g_sbi

Rename the global `sbi` to `g_sbi` to prepare for
the upcoming per-sbi buffer management.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240704050259.520618-2-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: get rid of global sbi in lib/inode.c

Get rid of the global sbi when EROFS_MT_ENABLED is defined.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240704050259.520618-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: change function definition of erofs_blocklist_open()

Modify erofs_blocklist_open() to accept a file pointer instead of
a file path, making it suitable for external use in liberofs.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240703030327.3280503-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: rebuild: only update dev/i_ino[1] pairs for directories

Since the underlying dev/i_ino[1] pairs are only useful for merged
sub-directories, don't bother with other types of inodes.

Otherwise, the original i_ino[1] could be overwritten unexpectedly,
which impacts resvsp mode at least..

Fixes: f64d9d02576b ("erofs-utils: introduce incremental builds")
Reported-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240702083144.2120808-1-hsiangkao@linux.alibaba.com

erofs-utils: fix "non-trivial designated initializers not supported"

This partially reverts commit 79f6e168d94c ("erofs-utils: improve
compatibility and reduce header conflicts") since some C++ compiler
will complain:

include/erofs_fs.h: In function 'void erofs_check_ondisk_layout_definitions()':
include/erofs_fs.h:460:2: sorry, unimplemented: non-trivial designated initializers not supported

Let's just bypass this compile-time check for the C++ language since
only external programs may be written in C++.

Fixes: 79f6e168d94c ("erofs-utils: improve compatibility and reduce header conflicts")
Cc: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240627031343.3424030-1-hsiangkao@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: add erofs_get_configure()

This adds `erofs_get_configure()` to get the global configuration
`cfg`.  It allows external entities to change the global configuration
through this helper, thereby controlling the EROFS mkfs process.

It is just a temporary helper for liberofs and it will be deprecated
in the future.  Don't rely on it too much.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240627022741.3912785-1-hongzhen@linux.alibaba.com
[ Gao Xiang: minor commit message update. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: lib: add erofs_{rebuild_make_root,enable_sb_chksum}

Move erofs_sb_csum_set() and erofs_mkfs_alloc_root() into liberofs
for external use.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240625062458.1514209-1-hongzhen@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: improve compatibility and reduce header conflicts

Adjust initializers of union in erofs-utils to ensure compatibility
with various compilers. The original C99 designated initializer style
was not supported in other compilers (e.g., C++11), leading to build
failures.  Additionally, change the codebase to minimize potential
conflicts with headers from other projects.

Signed-off-by: Hongzhen Luo <hongzhen@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240624063217.170251-1-hongzhen@linux.alibaba.com
[ Gao Xiang: minor commit message update. ]
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: introduce `payload` field in `struct erofs_vfile`

Allow customized `vfile` with non-NULL `ops` utilizing `payload`
for additional information.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240624115923.4090196-2-hsiangkao@linux.alibaba.com

erofs-utils: derive i_srcpath for erofs_rebuild_mkdir()

Also add missing erofs_iput() on errors.

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240624115923.4090196-1-hsiangkao@linux.alibaba.com

erofs-utils: optimize write_uncompressed_file_from_fd()

Utilize copy offloading to speed up copying data from the source
filesystem to the target EROFS filesystem.

This method improves build speed by approximately 9% (tested with
Linux 5.4.140 source code dataset).

Reported-by: Daan De Meyer <daan.j.demeyer@gmail.com>
Closes: https://lore.kernel.org/r/CAO8sHcmZZORnrJXA=QzmGkYNkNWn7M+amAK_DZ19-WL4kLUvpw@mail.gmail.com
Link: https://lore.kernel.org/r/20240623115932.2696312-1-hsiangkao@linux.alibaba.com
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>

erofs-utils: skip all unidentified xattrs from local paths

Just warn out but continue.  Don't over-complicate for now.

Reported-by: Gael Donval <gael.donval@manchester.ac.uk>
Closes: https://lore.kernel.org/r/4abed942399fb29933f0fa85cc55d3d795ae8bcd.camel@manchester.ac.uk
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240618111700.267702-1-hsiangkao@linux.alibaba.com

erofs-utils: enable mapfile for `--tar=f`

The data offsets in the tar streams can always be looked up now:
    mkfs.erofs --tar=f,MAPFILE IMAGE TARBALL

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240618082414.47876-9-hsiangkao@linux.alibaba.com

erofs-utils: enable incremental builds

`--incremental=<data|rvsp>` are now supported for tarerofs but
only `--incremental=rvsp` works for the rebuild mode.

For example:
$ mkfs.erofs --tar=f --gzip --aufs --clean=data foo.erofs f0.tgz
$ mkfs.erofs --tar=f --gzip --aufs --incremental=data foo.erofs f1.tgz
...
$ mkfs.erofs --tar=f --gzip --aufs --incremental=data foo.erofs fn-1.tgz

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240618082414.47876-8-hsiangkao@linux.alibaba.com

erofs-utils: fix incremental builds for tarerofs index mode

The blob data area should be considered in the total block number to
prevent overlap during incremental builds.

Fixes: b6749839e710 ("erofs-utils: generate preallocated extents for tarerofs")
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240618082414.47876-7-hsiangkao@linux.alibaba.com