sched: Fix migrate_disable() vs set_cpus_allowed_ptr()
authorPeter Zijlstra <peterz@infradead.org>
Fri, 18 Sep 2020 15:24:31 +0000 (17:24 +0200)
committerPeter Zijlstra <peterz@infradead.org>
Tue, 10 Nov 2020 17:39:00 +0000 (18:39 +0100)
commit6d337eab041d56bb8f0e7794f39906c21054c512
treeeafcdc2dd8f2850ffd87d151363d275c66dde01b
parentaf449901b84c98cbd84a0113223ba3bcfcb12a26
sched: Fix migrate_disable() vs set_cpus_allowed_ptr()

Concurrent migrate_disable() and set_cpus_allowed_ptr() has
interesting features. We rely on set_cpus_allowed_ptr() to not return
until the task runs inside the provided mask. This expectation is
exported to userspace.

This means that any set_cpus_allowed_ptr() caller must wait until
migrate_enable() allows migrations.

At the same time, we don't want migrate_enable() to schedule, due to
patterns like:

preempt_disable();
migrate_disable();
...
migrate_enable();
preempt_enable();

And:

raw_spin_lock(&B);
spin_unlock(&A);

this means that when migrate_enable() must restore the affinity
mask, it cannot wait for completion thereof. Luck will have it that
that is exactly the case where there is a pending
set_cpus_allowed_ptr(), so let that provide storage for the async stop
machine.

Much thanks to Valentin who used TLA+ most effective and found lots of
'interesting' cases.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Link: https://lkml.kernel.org/r/20201023102346.921768277@infradead.org
include/linux/sched.h
kernel/sched/core.c