erofs-utils: introduce fragment cache
authorGao Xiang <hsiangkao@linux.alibaba.com>
Mon, 20 Jan 2025 02:35:38 +0000 (10:35 +0800)
committerGao Xiang <hsiangkao@linux.alibaba.com>
Thu, 6 Feb 2025 05:01:10 +0000 (13:01 +0800)
commitf511cfbbc0da3eb54d8da2d6751865a54a1916e0
treef9e719ecf5479e5683d2939d596af58ee77c2ed4
parentbeb8129002dfd53324f6bb4f0c4dbbda2cdce61a
erofs-utils: introduce fragment cache

Difference from the previous Yiyan's version [1], it just uses
a tmpfile to keep all decompressed data for fragments.

Dataset: linux 5.4.140
mkfs.erofs command line:
mkfs.erofs -zlzma -C131072 -T0 -Eall-fragments,fragdedupe=inode foo.erofs <dir>
Test command line:
hyperfine -p "echo 3 > /proc/sys/vm/drop_caches; sleep 1" "fsck/fsck.erofs --extract foo.erofs"

Vanilla:
  Time (mean ± σ):     362.309 s ±  0.406 s   [User: 360.298 s, System: 0.956 s]

After:
  Time (mean ± σ):     20.880 s ±  0.026 s    [User: 19.751 s, System: 1.058 s]

An in-memory LRU cache could also be implemented later to meet
different needs.

[1] https://lore.kernel.org/r/20231023071528.1912105-1-lyy0627@sjtu.edu.cn

Cc: Li Yiyan <lyy0627@sjtu.edu.cn>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20250120023538.3156606-1-hsiangkao@linux.alibaba.com
fsck/main.c
fuse/main.c
include/erofs/fragments.h
lib/data.c
lib/fragments.c