From 649bb75d19c93f5459f450191953dff4825fda3e Mon Sep 17 00:00:00 2001 From: Almog Khaikin Date: Mon, 21 Mar 2022 11:00:59 +0200 Subject: [PATCH] io_uring: fix memory ordering when SQPOLL thread goes to sleep Without a full memory barrier between the store to the flags and the load of the SQ tail the two operations can be reordered and this can lead to a situation where the SQPOLL thread goes to sleep while the application writes to the SQ tail and doesn't see the wakeup flag. This memory barrier pairs with a full memory barrier in the application between its store to the SQ tail and its load of the flags. Signed-off-by: Almog Khaikin Link: https://lore.kernel.org/r/20220321090059.46313-1-almogkh@gmail.com Signed-off-by: Jens Axboe --- fs/io_uring.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/io_uring.c b/fs/io_uring.c index 1a65d78..48f4540 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -8042,6 +8042,13 @@ static int io_sq_thread(void *data) needs_sched = false; break; } + + /* + * Ensure the store of the wakeup flag is not + * reordered with the load of the SQ tail + */ + smp_mb(); + if (io_sqring_entries(ctx)) { needs_sched = false; break; -- 2.7.4