dm: support bio polling
authorMing Lei <ming.lei@redhat.com>
Sat, 5 Mar 2022 02:08:04 +0000 (21:08 -0500)
committerMike Snitzer <snitzer@redhat.com>
Wed, 9 Mar 2022 17:21:56 +0000 (12:21 -0500)
commitb99fdcdc36369d2a3a26bf44736d420eb629d8b7
treee0f40de03f4b9264dba2de7201898a4782fe37c3
parent69fe0f29892077f14b56e2a479b6bcf533209d53
dm: support bio polling

Support bio polling (REQ_POLLED) in the following approach:

1) only support io polling on normal READ/WRITE, and other abnormal IOs
still fallback to IRQ mode, so the target io (and DM's clone bio) is
exactly inside the dm io.

2) hold one refcnt on io->io_count after submitting this dm bio with
REQ_POLLED

3) support dm native bio splitting, any dm io instance associated with
current bio will be added into one list which head is bio->bi_private
which will be recovered before ending this bio

4) implement .poll_bio() callback, call bio_poll() on the single target
bio inside the dm io which is retrieved via bio->bi_bio_drv_data; call
dm_io_dec_pending() after the target io is done in .poll_bio()

5) enable QUEUE_FLAG_POLL if all underlying queues enable QUEUE_FLAG_POLL,
which is based on Jeffle's previous patch.

These changes are good for a 30-35% IOPS improvement for polled IO.

For detailed test results please see (Jens, thanks for testing!):
https://listman.redhat.com/archives/dm-devel/2022-March/049868.html
or https://marc.info/?l=linux-block&m=164684246214700&w=2

Tested-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
drivers/md/dm-core.h
drivers/md/dm-table.c
drivers/md/dm.c