erofs-utils: mkfs: allow disabling fragment deduplication
authorGao Xiang <hsiangkao@linux.alibaba.com>
Mon, 23 Dec 2024 09:40:31 +0000 (17:40 +0800)
committerGao Xiang <hsiangkao@linux.alibaba.com>
Thu, 26 Dec 2024 05:23:35 +0000 (13:23 +0800)
commit5aa368528be1b06f919cf520885b67ca6e643703
tree493209204df84ef12b4eea8fb3079a158ae976ff
parent341d23a878a295b42ddd61a3361f8f1dccb9ed67
erofs-utils: mkfs: allow disabling fragment deduplication

Currently, although fragment inode compression is already
multi-threaded, the data parts of inodes prior to their own fragments
are still single-threaded if fragment deduplication is on.  This can
greatly slow down `-Eall-fragments` image building at least for
the current mkfs codebase.

Let's add an extended option `-E^fragdedupe` to explicitly disable it.

After this commit, the Fedora Kiwi builds I'm testing can be reduced
from 1148s (3,096,842,240 bytes, 2.9G) to 137s (2,969,956,352 bytes,
2.8G) with `-Eall-fragments,^fragdedupe -C524288 -z lzma,level=6,
dictsize=524288` on Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz with
32 cores.

Cc: Neal Gompa <ngompa13@gmail.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20241223094031.1534175-1-hsiangkao@linux.alibaba.com
include/erofs/config.h
lib/compress.c
mkfs/main.c