io_uring: skip futile iopoll iterations

The only way to get out of io_iopoll_getevents() and continue iterating
is to have empty iopoll_list, otherwise the main loop would just exit.
So, instead of the unlock on 8th time heuristic, do that based on
iopoll_list.

Also, as no one can add new requests to iopoll_list while
io_iopoll_check() hold uring_lock, it's useless to spin with the list
empty, return in that case.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/5b8ebe84f5fff7ffa1f708952dfef7fc78b668e2.1618278933.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 3632b5a4f13f0250cb69c09306024366cb0d6d46..ffcb3ec4de95b02a0d36e095f3111567fc5c3e64 100644 (file)
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2385,7 +2385,7 @@ static void io_iopoll_try_reap_events(struct io_ring_ctx *ctx)
 static int io_iopoll_check(struct io_ring_ctx *ctx, long min)
 {
        unsigned int nr_events = 0;
-       int iters = 0, ret = 0;
+       int ret = 0;
 
        /*
         * We disallow the app entering submit/complete with polling, but we
@@ -2414,10 +2414,13 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, long min)
                 * forever, while the workqueue is stuck trying to acquire the
                 * very same mutex.
                 */
-               if (!(++iters & 7)) {
+               if (list_empty(&ctx->iopoll_list)) {
                        mutex_unlock(&ctx->uring_lock);
                        io_run_task_work();
                        mutex_lock(&ctx->uring_lock);
+
+                       if (list_empty(&ctx->iopoll_list))
+                               break;
                }
 
                ret = io_iopoll_getevents(ctx, &nr_events, min);