io_uring/kbuf: fix not advancing READV kbuf ring

When we don't recycle a selected ring buffer we should advance the head
of the ring, so don't just skip io_kbuf_recycle() for IORING_OP_READV
but adjust the ring.

Fixes: 934447a603b22 ("io_uring: do not recycle buffer in READV")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Dylan Yudaken <dylany@fb.com>
Link: https://lore.kernel.org/r/a6d85e2611471bcb5d5dcd63a8342077ddc2d73d.1662480490.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>

diff --git a/io_uring/kbuf.h b/io_uring/kbuf.h
index d6af208..746fbf3 100644 (file)
--- a/io_uring/kbuf.h
+++ b/io_uring/kbuf.h
@@ -91,9 +91,13 @@ static inline void io_kbuf_recycle(struct io_kiocb *req, unsigned issue_flags)
         * buffer data. However if that buffer is recycled the original request
         * data stored in addr is lost. Therefore forbid recycling for now.
         */
-       if (req->opcode == IORING_OP_READV)
+       if (req->opcode == IORING_OP_READV) {
+               if ((req->flags & REQ_F_BUFFER_RING) && req->buf_list) {
+                       req->buf_list->head++;
+                       req->buf_list = NULL;
+               }
                return;
-
+       }
        if (req->flags & REQ_F_BUFFER_SELECTED)
                io_kbuf_recycle_legacy(req, issue_flags);
        if (req->flags & REQ_F_BUFFER_RING)