bpf: Run sleepable programs with migration disabled
authorAlexei Starovoitov <ast@kernel.org>
Wed, 10 Feb 2021 03:36:27 +0000 (19:36 -0800)
committerDaniel Borkmann <daniel@iogearbox.net>
Thu, 11 Feb 2021 15:18:55 +0000 (16:18 +0100)
In older non-RT kernels migrate_disable() was the same as preempt_disable().
Since commit 74d862b682f5 ("sched: Make migrate_disable/enable() independent of RT")
migrate_disable() is real and doesn't prevent sleeping.

Running sleepable programs with migration disabled allows to add support for
program stats and per-cpu maps later.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: KP Singh <kpsingh@kernel.org>
Link: https://lore.kernel.org/bpf/20210210033634.62081-3-alexei.starovoitov@gmail.com
kernel/bpf/trampoline.c

index 5be3beeedd74f36c6c897080e960e7ce4edab3cf..89fc849ba271b258b415e42d9dd4865165277af4 100644 (file)
@@ -425,11 +425,13 @@ void notrace __bpf_prog_exit(struct bpf_prog *prog, u64 start)
 void notrace __bpf_prog_enter_sleepable(void)
 {
        rcu_read_lock_trace();
+       migrate_disable();
        might_fault();
 }
 
 void notrace __bpf_prog_exit_sleepable(void)
 {
+       migrate_enable();
        rcu_read_unlock_trace();
 }