io_uring: add memory barrier to synchronize io_kiocb's result and iopoll_completed
authorXiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
Mon, 15 Jun 2020 18:06:38 +0000 (02:06 +0800)
committerJens Axboe <axboe@kernel.dk>
Wed, 17 Jun 2020 18:49:09 +0000 (12:49 -0600)
commitbbde017a32b32d2fa8d5fddca25fade20132abf8
tree90d492437553c906b55d271e2effc71058e2c9e2
parent2d7d67920e5c8e0854df23ca77da2dd5880ce5dd
io_uring: add memory barrier to synchronize io_kiocb's result and iopoll_completed

In io_complete_rw_iopoll(), stores to io_kiocb's result and iopoll
completed are two independent store operations, to ensure that once
iopoll_completed is ture and then req->result must been perceived by
the cpu executing io_do_iopoll(), proper memory barrier should be used.

And in io_do_iopoll(), we check whether req->result is EAGAIN, if it is,
we'll need to issue this io request using io-wq again. In order to just
issue a single smp_rmb() on the completion side, move the re-submit work
to io_iopoll_complete().

Cc: stable@vger.kernel.org
Signed-off-by: Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
[axboe: don't set ->iopoll_completed for -EAGAIN retry]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c