rcu: Speed up calling of RCU tasks callbacks
authorSteven Rostedt (VMware) <rostedt@goodmis.org>
Thu, 24 May 2018 22:49:46 +0000 (18:49 -0400)
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Thu, 12 Jul 2018 22:39:21 +0000 (15:39 -0700)
commitc03be752d39dc64dcfda0ac8ce87fb10b1ee5621
tree7fa3b8a9bb7dd09558e6d549de49fcecee81d262
parent0d805a70a652a6eef8d0283e5183879e7acb85ad
rcu: Speed up calling of RCU tasks callbacks

Joel Fernandes found that the synchronize_rcu_tasks() was taking a
significant amount of time. He demonstrated it with the following test:

 # cd /sys/kernel/tracing
 # while [ 1 ]; do x=1; done &
 # echo '__schedule_bug:traceon' > set_ftrace_filter
 # time echo '!__schedule_bug:traceon' > set_ftrace_filter;

real 0m1.064s
user 0m0.000s
sys 0m0.004s

Where it takes a little over a second to perform the synchronize,
because there's a loop that waits 1 second at a time for tasks to get
through their quiescent points when there's a task that must be waited
for.

After discussion we came up with a simple way to wait for holdouts but
increase the time for each iteration of the loop but no more than a
full second.

With the new patch we have:

 # time echo '!__schedule_bug:traceon' > set_ftrace_filter;

real 0m0.131s
user 0m0.000s
sys 0m0.004s

Which drops it down to 13% of what the original wait time was.

Link: http://lkml.kernel.org/r/20180523063815.198302-2-joel@joelfernandes.org
Reported-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Suggested-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
kernel/rcu/update.c