bpf: Use NOP_ATOMIC5 instead of emit_nops(&prog, 5) for BPF_TRAMP_F_CALL_ORIG
authorStanislav Fomichev <sdf@google.com>
Sat, 20 Mar 2021 00:00:01 +0000 (17:00 -0700)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Wed, 7 Apr 2021 13:00:14 +0000 (15:00 +0200)
commit7973a0dad073ce94bc13edd30224114e5c6aacb7
tree0657c69d799c5afad05309eed6be06de60a133d4
parent12b5f9dae41027f895b144ea01741d687bb31051
bpf: Use NOP_ATOMIC5 instead of emit_nops(&prog, 5) for BPF_TRAMP_F_CALL_ORIG

commit b9082970478009b778aa9b22d5561eef35b53b63 upstream.

__bpf_arch_text_poke does rewrite only for atomic nop5, emit_nops(xxx, 5)
emits non-atomic one which breaks fentry/fexit with k8 atomics:

P6_NOP5 == P6_NOP5_ATOMIC (0f1f440000 == 0f1f440000)
K8_NOP5 != K8_NOP5_ATOMIC (6666906690 != 6666666690)

Can be reproduced by doing "ideal_nops = k8_nops" in "arch_init_ideal_nops()
and running fexit_bpf2bpf selftest.

Fixes: e21aa341785c ("bpf: Fix fexit trampoline.")
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20210320000001.915366-1-sdf@google.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
arch/x86/net/bpf_jit_comp.c