From: Pavel Begunkov Date: Thu, 6 Apr 2023 13:20:13 +0000 (+0100) Subject: io_uring: refactor __io_cq_unlock_post_flush() X-Git-Tag: v6.6.17~4893^2~35 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=c66ae3ec38f946edb1776d25c1c8cd63803b8ec3;p=platform%2Fkernel%2Flinux-rpi.git io_uring: refactor __io_cq_unlock_post_flush() Separate ->task_complete path in __io_cq_unlock_post_flush(). Signed-off-by: Pavel Begunkov Link: https://lore.kernel.org/r/baa9b8d822f024e4ee01c40209dbbe38d9c8c11d.1680782017.git.asml.silence@gmail.com Signed-off-by: Jens Axboe --- diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 8a327a8..0ea50c4 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -627,21 +627,23 @@ static inline void __io_cq_unlock_post(struct io_ring_ctx *ctx) io_cqring_wake(ctx); } -static inline void __io_cq_unlock_post_flush(struct io_ring_ctx *ctx) +static void __io_cq_unlock_post_flush(struct io_ring_ctx *ctx) __releases(ctx->completion_lock) { io_commit_cqring(ctx); - __io_cq_unlock(ctx); - io_commit_cqring_flush(ctx); - /* - * As ->task_complete implies that the ring is single tasked, cq_wait - * may only be waited on by the current in io_cqring_wait(), but since - * it will re-check the wakeup conditions once we return we can safely - * skip waking it up. - */ - if (!(ctx->flags & IORING_SETUP_DEFER_TASKRUN)) + if (ctx->task_complete) { + /* + * ->task_complete implies that only current might be waiting + * for CQEs, and obviously, we currently don't. No one is + * waiting, wakeups are futile, skip them. + */ + io_commit_cqring_flush(ctx); + } else { + __io_cq_unlock(ctx); + io_commit_cqring_flush(ctx); io_cqring_wake(ctx); + } } void io_cq_unlock_post(struct io_ring_ctx *ctx)