sched: Improve wake_up_all_idle_cpus() take #2

As reported by syzbot and experienced by Pavel, using cpus_read_lock()
in wake_up_all_idle_cpus() generates lock inversion (against mmap_sem
and possibly others).

Instead, shrink the preempt disable region by iterating all CPUs and
checking the online status for each individual CPU while having
preemption disabled.

Fixes: 8850cb663b5c ("sched: Simplify wake_up_*idle*()")
Reported-by: syzbot+d5b23b18d2f4feae8a67@syzkaller.appspotmail.com
Reported-by: Pavel Machek <pavel@ucw.cz>
Reported-by: Qian Cai <quic_qiancai@quicinc.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Qian Cai <quic_qiancai@quicinc.com>

diff --git a/kernel/smp.c b/kernel/smp.c
index ad0b68a3a3d36e3b6adeeaa8aa27c8589dc7db2c..01a7c1706a58b1df8724f8afc49b4a3a46919bcd 100644 (file)
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -1170,14 +1170,12 @@ void wake_up_all_idle_cpus(void)
 {
        int cpu;
 
-       cpus_read_lock();
-       for_each_online_cpu(cpu) {
-               if (cpu == raw_smp_processor_id())
-                       continue;
-
-               wake_up_if_idle(cpu);
+       for_each_possible_cpu(cpu) {
+               preempt_disable();
+               if (cpu != smp_processor_id() && cpu_online(cpu))
+                       wake_up_if_idle(cpu);
+               preempt_enable();
        }
-       cpus_read_unlock();
 }
 EXPORT_SYMBOL_GPL(wake_up_all_idle_cpus);