io_uring: inline hot path of __io_queue_sqe()
authorPavel Begunkov <asml.silence@gmail.com>
Fri, 24 Sep 2021 20:59:59 +0000 (21:59 +0100)
committerJens Axboe <axboe@kernel.dk>
Tue, 19 Oct 2021 11:49:53 +0000 (05:49 -0600)
commitd475a9a6226c86b7febe3863b900b820a0e6b71c
tree548d831c57764d973e28471ad572bd65ca54b115
parent4652fe3f10e57abc3fc6ac11c431b2ef39f78c03
io_uring: inline hot path of __io_queue_sqe()

Extract slow paths from __io_queue_sqe() into a function and inline the
hot path. With that we have everything completely inlined on the
submission path up until io_issue_sqe().

-> io_submit_sqes()
  -> io_submit_sqe() (inlined)
    -> io_queue_sqe() (inlined)
       -> __io_queue_sqe() (inlined)
         -> io_issue_sqe()

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/f1606864d95d7f26dc28c7eec3dc6ed6ec32618a.1632516769.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c