kasan: speed up mte_set_mem_tag_range
authorEvgenii Stepanov <eugenis@google.com>
Fri, 21 May 2021 01:00:23 +0000 (18:00 -0700)
committerWill Deacon <will@kernel.org>
Tue, 25 May 2021 18:21:58 +0000 (19:21 +0100)
commit3d0cca0b02ac98eac9157b26cf3951997db68b37
tree63faa9f036f3d926c0dc915cae59ecda399852dc
parentc4681547bcce777daf576925a966ffa824edd09d
kasan: speed up mte_set_mem_tag_range

Use DC GVA / DC GZVA to speed up KASan memory tagging in HW tags mode.

The first cacheline is always tagged using STG/STZG even if the address is
cacheline-aligned, as benchmarks show it is faster than a conditional
branch.

Signed-off-by: Evgenii Stepanov <eugenis@google.com>
Co-developed-by: Peter Collingbourne <pcc@google.com>
Signed-off-by: Peter Collingbourne <pcc@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Link: https://lore.kernel.org/r/20210521010023.3244784-1-eugenis@google.com
Signed-off-by: Will Deacon <will@kernel.org>
arch/arm64/include/asm/mte-kasan.h