io_uring/rw: add write support for IOCB_DIO_CALLER_COMP
authorJens Axboe <axboe@kernel.dk>
Sat, 8 Jul 2023 16:03:52 +0000 (10:03 -0600)
committerJens Axboe <axboe@kernel.dk>
Tue, 1 Aug 2023 23:32:45 +0000 (17:32 -0600)
commit099ada2c87260e5c52fbdad4ca40c165fc194a2e
treee48cea61e8cea2124071f839679814a6667a0ecc
parent9cf3516c29e6dba95d36d7d86c03b06495107859
io_uring/rw: add write support for IOCB_DIO_CALLER_COMP

If the filesystem dio handler understands IOCB_DIO_CALLER_COMP, we'll
get a kiocb->ki_complete() callback with kiocb->dio_complete set. In
that case, rather than complete the IO directly through task_work, queue
up an intermediate task_work handler that first processes this callback
and then immediately completes the request.

For XFS, this avoids a punt through a workqueue, which is a lot less
efficient and adds latency to lower queue depth (or sync) O_DIRECT
writes.

Only do this for non-polled IO, as polled IO doesn't need this kind
of deferral as it always completes within the task itself. This then
avoids a check for deferral in the polled IO completion handler.

Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Dave Chinner <dchinner@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_uring/rw.c