block: optimise for_each_bvec() advance
authorPavel Begunkov <asml.silence@gmail.com>
Tue, 24 Nov 2020 17:58:12 +0000 (17:58 +0000)
committerJens Axboe <axboe@kernel.dk>
Wed, 2 Dec 2020 16:46:55 +0000 (09:46 -0700)
commit6b6667aa4d1e0866f00b62d35a9be3875c7551f8
treec495f7838c57e1865a421173e4f40ac442e797d0
parent977115c0f664e016a6b2774d4f97116ade23d732
block: optimise for_each_bvec() advance

Because of how for_each_bvec() works it never advances across multiple
entries at a time, so bvec_iter_advance() is an overkill. Add
specialised bvec_iter_advance_single() that is faster. It also handles
zero-len bvecs, so can kill bvec_iter_skip_zero_bvec().

   text    data     bss     dec     hex filename
before:
  23977     805       0   24782    60ce lib/iov_iter.o
before, bvec_iter_advance() w/o WARN_ONCE()
  22886     600       0   23486    5bbe ./lib/iov_iter.o
after:
  21862     600       0   22462    57be lib/iov_iter.o

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
include/linux/bvec.h