rcu/nocb: Unify timers
authorFrederic Weisbecker <frederic@kernel.org>
Tue, 23 Feb 2021 00:10:11 +0000 (01:10 +0100)
committerPaul E. McKenney <paulmck@kernel.org>
Wed, 12 May 2021 19:10:23 +0000 (12:10 -0700)
commite75bcd48e2c4026b1f3feda916a2327b1744d664
tree1891eba686d079afc2ed4370282e2d58cb624b97
parent870905169da8bfae0570df013efe860d33251b0f
rcu/nocb: Unify timers

Now that ->nocb_timer and ->nocb_bypass_timer have become quite similar,
this commit merges them together.  A new RCU_NOCB_WAKE_BYPASS wake level
is introduced.  As a result, timers perform all kinds of deferred wake
ups but other deferred wakeup callsites only handle non-bypass wakeups
in order not to wake up rcuo too early.

The timer also unconditionally executes a full barrier so as to order
timer_pending() and callback enqueue although the path performing
RCU_NOCB_WAKE_FORCE that makes use of it is debatable. It should also
test against the rdp leader instead of the current rdp.

This unconditional full barrier shouldn't bring visible overhead since
these timers almost never fire.

Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
include/trace/events/rcu.h
kernel/rcu/tree.h
kernel/rcu/tree_plugin.h