rcu: exp: Protect all sync_rcu_preempt_exp_done() with rcu_node lock
authorBoqun Feng <boqun.feng@gmail.com>
Fri, 9 Mar 2018 01:14:51 +0000 (09:14 +0800)
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Tue, 15 May 2018 17:26:07 +0000 (10:26 -0700)
commit55ebfce0605a4dce35467dfb1dc1b3ad26fe9f86
tree18e8badda41172f8c3527929fcc8e0e776a3c22b
parent7be8c56f8f8a58af92f8791c5a09d48e342d7101
rcu: exp: Protect all sync_rcu_preempt_exp_done() with rcu_node lock

Currently some callsites of sync_rcu_preempt_exp_done() are not called
with the corresponding rcu_node's ->lock held, which could introduces
bugs as per Paul:

o CPU 0 in sync_rcu_preempt_exp_done() reads ->exp_tasks and
sees that it is NULL.

o CPU 1 blocks within an RCU read-side critical section, so
it enqueues the task and points ->exp_tasks at it and
clears CPU 1's bit in ->expmask.

o All other CPUs clear their bits in ->expmask.

o CPU 0 reads ->expmask, sees that it is zero, so incorrectly
concludes that all quiescent states have completed, despite
the fact that ->exp_tasks is non-NULL.

To fix this, sync_rcu_preempt_exp_unlocked() is introduced to replace
lockless callsites of sync_rcu_preempt_exp_done().

Further, a lockdep annotation is added into sync_rcu_preempt_exp_done()
to prevent mis-use in the future.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
kernel/rcu/tree_exp.h