erofs-utils: mkfs: introduce global compressed data deduplication
authorZiyang Zhang <ZiyangZhang@linux.alibaba.com>
Mon, 26 Sep 2022 15:25:11 +0000 (23:25 +0800)
committerGao Xiang <hsiangkao@linux.alibaba.com>
Tue, 4 Oct 2022 17:32:09 +0000 (01:32 +0800)
commitf3f9a2ce313727a16dd44d5f6cb37882247bdeea
tree706c56198e4373d7e583eb350b8914c9cd052901
parent7c76868e97022e5d7a6841f77de98229f39cd454
erofs-utils: mkfs: introduce global compressed data deduplication

This patch introduces global compressed data deduplication to
reuse potential prefixes for each pcluster.

It also uses rolling hashing and tries to shorten the previous
compressed extent in order to explore more possibilities for
data deduplication.

Co-developed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Ziyang Zhang <ZiyangZhang@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20220926152511.94832-9-hsiangkao@linux.alibaba.com
include/erofs/config.h
include/erofs/dedupe.h [new file with mode: 0644]
lib/Makefile.am
lib/compress.c
lib/dedupe.c [new file with mode: 0644]
lib/rolling_hash.h [new file with mode: 0644]
mkfs/main.c