riscv: mm: Pre-allocate PGD entries for vmalloc/modules area
authorBjörn Töpel <bjorn@rivosinc.com>
Wed, 31 May 2023 09:38:17 +0000 (11:38 +0200)
committerPalmer Dabbelt <palmer@rivosinc.com>
Tue, 20 Jun 2023 00:58:01 +0000 (17:58 -0700)
commit7d3332be011e4ed061c1403b30b5e54ebccb4fa2
tree5ea66abc64191a329eeb7ed89ae4826c7c2b1a83
parent16252e018a30486eedcfec81fc313445cac25bea
riscv: mm: Pre-allocate PGD entries for vmalloc/modules area

The RISC-V port requires that kernel PGD entries are to be
synchronized between MMs. This is done via the vmalloc_fault()
function, that simply copies the PGD entries from init_mm to the
faulting one.

Historically, faulting in PGD entries have been a source for both bugs
[1], and poor performance.

One way to get rid of vmalloc faults is by pre-allocating the PGD
entries. Pre-allocating the entries potientially wastes 64 * 4K (65 on
SV39). The pre-allocation function is pulled from Jörg Rödel's x86
work, with the addition of 3-level page tables (PMD allocations).

The pmd_alloc() function needs the ptlock cache to be initialized
(when split page locks is enabled), so the pre-allocation is done in a
RISC-V specific pgtable_cache_init() implementation.

Pre-allocate the kernel PGD entries for the vmalloc/modules area, but
only for 64b platforms.

Link: https://lore.kernel.org/lkml/20200508144043.13893-1-joro@8bytes.org/
Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Link: https://lore.kernel.org/r/20230531093817.665799-1-bjorn@kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
arch/riscv/mm/fault.c
arch/riscv/mm/init.c