rtmutex: Wake up the waiters lockless while dropping the read lock.
authorThomas Gleixner <tglx@linutronix.de>
Tue, 28 Sep 2021 15:00:06 +0000 (17:00 +0200)
committerPeter Zijlstra <peterz@infradead.org>
Fri, 1 Oct 2021 11:57:52 +0000 (13:57 +0200)
commit9321f8152d9a764208c3f0dad49e0c55f293b7ab
treeda2f8007e85e6ff75483bf78d5cc1cb388594cae
parent8fe46535e10dbfebad68ad9f2f8260e49f5852c9
rtmutex: Wake up the waiters lockless while dropping the read lock.

The rw_semaphore and rwlock_t implementation both wake the waiter while
holding the rt_mutex_base::wait_lock acquired.
This can be optimized by waking the waiter lockless outside of the
locked section to avoid a needless contention on the
rt_mutex_base::wait_lock lock.

Extend rt_mutex_wake_q_add() to also accept task and state and use it in
__rwbase_read_unlock().

Suggested-by: Davidlohr Bueso <dave@stgolabs.net>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210928150006.597310-3-bigeasy@linutronix.de
kernel/locking/rtmutex.c
kernel/locking/rwbase_rt.c