sched: Fix race in task_call_func()
authorPeter Zijlstra <peterz@infradead.org>
Wed, 26 Oct 2022 11:43:00 +0000 (13:43 +0200)
committerPeter Zijlstra <peterz@infradead.org>
Mon, 14 Nov 2022 08:58:32 +0000 (09:58 +0100)
commit91dabf33ae5df271da63e87ad7833e5fdb4a44b9
tree86a1d62dca4c1172682dce3c8af7a5028dd34a00
parent448dca8c88755b768552e19bd1618be34ef6d1ff
sched: Fix race in task_call_func()

There is a very narrow race between schedule() and task_call_func().

  CPU0 CPU1

  __schedule()
    rq_lock();
    prev_state = READ_ONCE(prev->__state);
    if (... && prev_state) {
      deactivate_tasl(rq, prev, ...)
        prev->on_rq = 0;

task_call_func()
  raw_spin_lock_irqsave(p->pi_lock);
  state = READ_ONCE(p->__state);
  smp_rmb();
  if (... || p->on_rq) // false!!!
    rq = __task_rq_lock()

  ret = func();

    next = pick_next_task();
    rq = context_switch(prev, next)
      prepare_lock_switch()
        spin_release(&__rq_lockp(rq)->dep_map...)

So while the task is on it's way out, it still holds rq->lock for a
little while, and right then task_call_func() comes in and figures it
doesn't need rq->lock anymore (because the task is already dequeued --
but still running there) and then the __set_task_frozen() thing observes
it's holding rq->lock and yells murder.

Avoid this by waiting for p->on_cpu to get cleared, which guarantees
the task is fully finished on the old CPU.

( While arguably the fixes tag is 'wrong' -- none of the previous
  task_call_func() users appears to care for this case. )

Fixes: f5d39b020809 ("freezer,sched: Rewrite core freezer logic")
Reported-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://lkml.kernel.org/r/Y1kdRNNfUeAU+FNl@hirez.programming.kicks-ass.net
kernel/sched/core.c