xfs: create a big array data structure
authorDarrick J. Wong <djwong@kernel.org>
Thu, 10 Aug 2023 14:48:04 +0000 (07:48 -0700)
committerDarrick J. Wong <djwong@kernel.org>
Thu, 10 Aug 2023 14:48:04 +0000 (07:48 -0700)
commit3934e8ebb7cc6e5f1ade35d586ed3eb79b88eb95
tree0322e0d24d10c1249754b3dd7c34c6df292c762a
parent014ad53732d2bac34d21a251f3622a4da516e21b
xfs: create a big array data structure

Create a simple 'big array' data structure for storage of fixed-size
metadata records that will be used to reconstruct a btree index.  For
repair operations, the most important operations are append, iterate,
and sort.

Earlier implementations of the big array used linked lists and suffered
from severe problems -- pinning all records in kernel memory was not a
good idea and frequently lead to OOM situations; random access was very
inefficient; and record overhead for the lists was unacceptably high at
40-60%.

Therefore, the big memory array relies on the 'xfile' abstraction, which
creates a memfd file and stores the records in page cache pages.  Since
the memfd is created in tmpfs, the memory pages can be pushed out to
disk if necessary and we have a built-in usage limit of 50% of physical
memory.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Kent Overstreet <kent.overstreet@linux.dev>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
fs/xfs/Kconfig
fs/xfs/Makefile
fs/xfs/scrub/trace.c
fs/xfs/scrub/trace.h
fs/xfs/scrub/xfarray.c [new file with mode: 0644]
fs/xfs/scrub/xfarray.h [new file with mode: 0644]
fs/xfs/scrub/xfile.c [new file with mode: 0644]
fs/xfs/scrub/xfile.h [new file with mode: 0644]