erofs-utils: mkfs: fix inefficient fragment deduplication
authorGao Xiang <hsiangkao@linux.alibaba.com>
Tue, 21 Jan 2025 14:34:00 +0000 (22:34 +0800)
committerGao Xiang <hsiangkao@linux.alibaba.com>
Tue, 21 Jan 2025 14:39:37 +0000 (22:39 +0800)
commitc962911673666ed3ab663522a1f310664d91c77a
treef56d704b37f1efa4ca8562347c5e5abb3a4d5ccb
parent4cab97e3af2119800d8ad59babdabc6487a0ef6c
erofs-utils: mkfs: fix inefficient fragment deduplication

Currently, long fragment comparisons could cause suboptimal results,
leading to final image sizes still larger than expected:
 _________________________________________________________________________
|______ Testset _____|_______ Vanilla _________|_________ After __________| Command Line
|  CoreOS [1]        |   802107392 (765 MiB)   |   687501312 (656 MiB)    | -zlzma,6 -Eall-fragments,fragdedupe=inode -C131072
|____________________|__ 771715072 (736 MiB) __|__ 658485248 (628 MiB)  __| -zlzma,6 -Eall-fragments,fragdedupe=inode -C1048576
|  Fedora KIWI [2]   |_ 2584076288 (2465 MiB) _|_ 2550837248 (2433 MiB) __| -zlzma,6 -Eall-fragments,fragdedupe=inode -C1048576
|____________________|_ 2843598848 (2712 MiB) _|_ 2810359808 (2681 MiB) __| (Fedora-KDE-Desktop-Live-Rawhide.0.x86_64.iso)

Almost all images that use `-Eall-fragments` could benefit from this.

[1] https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/41.20241215.3.0/x86_64/fedora-coreos-41.20241215.3.0-live.x86_64.iso
[2] https://pagure.io/fedora-kiwi-descriptions.git

Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20250121143400.1857269-1-hsiangkao@linux.alibaba.com
lib/compress.c
lib/fragments.c