sbitmap: Try each queue to wake up at least one waiter
authorGabriel Krisman Bertazi <krisman@suse.de>
Tue, 15 Nov 2022 22:45:53 +0000 (17:45 -0500)
committerJens Axboe <axboe@kernel.dk>
Wed, 16 Nov 2022 18:33:03 +0000 (11:33 -0700)
commit26edb30dd1c0c9be11fa676b4f330ada7b794ba6
tree3f7946826afd282eda99b0396540d77b39e90b63
parentee7dc86b6d3e3b86c2c487f713eda657850de238
sbitmap: Try each queue to wake up at least one waiter

Jan reported the new algorithm as merged might be problematic if the
queue being awaken becomes empty between the waitqueue_active inside
sbq_wake_ptr check and the wake up.  If that happens, wake_up_nr will
not wake up any waiter and we loose too many wake ups.  In order to
guarantee progress, we need to wake up at least one waiter here, if
there are any.  This now requires trying to wake up from every queue.

Instead of walking through all the queues with sbq_wake_ptr, this call
moves the wake up inside that function.  In a previous version of the
patch, I found that updating wake_index several times when walking
through queues had a measurable overhead.  This ensures we only update
it once, at the end.

Fixes: 4f8126bb2308 ("sbitmap: Use single per-bitmap counting to wake up queued tags")
Reported-by: Jan Kara <jack@suse.cz>
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20221115224553.23594-4-krisman@suse.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
lib/sbitmap.c