riscv: mm: Implement pmdp_collapse_flush for THP
authorMayuresh Chitale <mchitale@ventanamicro.com>
Mon, 30 Jan 2023 07:48:15 +0000 (13:18 +0530)
committerPalmer Dabbelt <palmer@rivosinc.com>
Thu, 2 Feb 2023 04:52:09 +0000 (20:52 -0800)
commitf0293cd1f4fcc4fbdcd65a5a7b3b318a6d471f78
treef92b31b5384510bef44f0c3aeffb513fca2f962f
parent2f394c0e7d1129a35156e492bc8f445fb20f43ac
riscv: mm: Implement pmdp_collapse_flush for THP

When THP is enabled, 4K pages are collapsed into a single huge
page using the generic pmdp_collapse_flush() which will further
use flush_tlb_range() to shoot-down stale TLB entries. Unfortunately,
the generic pmdp_collapse_flush() only invalidates cached leaf PTEs
using address specific SFENCEs which results in repetitive (or
unpredictable) page faults on RISC-V implementations which cache
non-leaf PTEs.

Provide a RISC-V specific pmdp_collapse_flush() which ensures both
cached leaf and non-leaf PTEs are invalidated by using non-address
specific SFENCEs as recommended by the RISC-V privileged specification.

Fixes: e88b333142e4 ("riscv: mm: add THP support on 64-bit")
Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
Link: https://lore.kernel.org/r/20230130074815.1694055-1-mchitale@ventanamicro.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
arch/riscv/include/asm/pgtable.h
arch/riscv/mm/pgtable.c