bpf, arm64: Support more atomic operations
authorHou Tao <houtao1@huawei.com>
Thu, 17 Feb 2022 07:22:31 +0000 (15:22 +0800)
committerDaniel Borkmann <daniel@iogearbox.net>
Mon, 28 Feb 2022 15:27:22 +0000 (16:27 +0100)
commit1902472b4fa97dba1fd10a204c6b231d6a560081
tree7ba4d807807c26dd6876d65b83fedafc377d45b6
parent79e7ce2e519ef5b3a5cacfd0e1d8ed456b89ff9f
bpf, arm64: Support more atomic operations

Atomics for eBPF patch series adds support for atomic[64]_fetch_add,
atomic[64]_[fetch_]{and,or,xor} and atomic[64]_{xchg|cmpxchg}, but it
only adds support for x86-64, so support these atomic operations for
arm64 as well.

Basically the implementation procedure is almost mechanical translation
of code snippets in atomic_ll_sc.h & atomic_lse.h & cmpxchg.h located
under arch/arm64/include/asm.

When LSE atomic is unavailable, an extra temporary register is needed for
(BPF_ADD | BPF_FETCH) to save the value of src register, instead of adding
TMP_REG_4 just use BPF_REG_AX instead. Also make emit_lse_atomic() as an
empty inline function when CONFIG_ARM64_LSE_ATOMICS is disabled.

For cpus_have_cap(ARM64_HAS_LSE_ATOMICS) case and no-LSE-ATOMICS case, the
following three tests: "./test_verifier", "./test_progs -t atomic" and
"insmod ./test_bpf.ko" are exercised and passed.

Signed-off-by: Hou Tao <houtao1@huawei.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20220217072232.1186625-4-houtao1@huawei.com
arch/arm64/net/bpf_jit.h
arch/arm64/net/bpf_jit_comp.c