bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy
authorHou Tao <houtao1@huawei.com>
Thu, 1 Sep 2022 06:19:35 +0000 (14:19 +0800)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Wed, 26 Oct 2022 10:34:41 +0000 (12:34 +0200)
commitf91e25cfa5538e930ec8f91aab1168e6c94755ef
tree99d7efdd2ae703015ef2318a7ddb2090085bf916
parent0e13425104903970a5ede853082d3bbb4edec6f3
bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy

[ Upstream commit 197827a05e13808c60f52632e9887eede63f1c16 ]

Now migrate_disable() does not disable preemption and under some
architectures (e.g. arm64) __this_cpu_{inc|dec|inc_return} are neither
preemption-safe nor IRQ-safe, so for fully preemptible kernel concurrent
lookups or updates on the same task local storage and on the same CPU
may make bpf_task_storage_busy be imbalanced, and
bpf_task_storage_trylock() on the specific cpu will always fail.

Fixing it by using this_cpu_{inc|dec|inc_return} when manipulating
bpf_task_storage_busy.

Fixes: bc235cdb423a ("bpf: Prevent deadlock from recursive bpf_task_storage_[get|delete]")
Signed-off-by: Hou Tao <houtao1@huawei.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/r/20220901061938.3789460-2-houtao@huaweicloud.com
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
kernel/bpf/bpf_local_storage.c
kernel/bpf/bpf_task_storage.c