io_uring: optimise io_free_batch_list
authorPavel Begunkov <asml.silence@gmail.com>
Mon, 21 Mar 2022 22:02:22 +0000 (22:02 +0000)
committerJens Axboe <axboe@kernel.dk>
Sun, 24 Apr 2022 23:34:16 +0000 (17:34 -0600)
commita538be5be328229d4da3343d4d6514bb4d5c3d5d
tree786c3125d1fa4d04af507c3346fd28b939c3e982
parent7819a1f6ac0393f1f7861dfece6ffd5ef010f0f9
io_uring: optimise io_free_batch_list

We do several req->flags checks in the fast path of
io_free_batch_list(). One explicit check of REQ_F_REFCOUNT, and two
other hidden in io_queue_next() and io_dismantle_req(). Moreover, there
is a io_req_put_rsrc_locked() call in between, so there is no hope
req->flags will be preserved in registers.

All those flags if not a slow path than definitely a slower path, so
put them all under a single flags mask check and save several mem
reloads and ifs.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/0fb493f73f2009aea395c570c2932fecaa4e1244.1647897811.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c