mm/filemap.c: avoid always dirtying mapping->flags on O_DIRECT
authorJens Axboe <axboe@fb.com>
Thu, 22 May 2014 18:54:16 +0000 (11:54 -0700)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Thu, 9 Oct 2014 19:21:29 +0000 (12:21 -0700)
commit6591981a30a8c752b1c537211f7fe4b9530fea41
tree01ae6e14bec1832a0eabb14b838b79b3937ee7b7
parent29c2a88157819d1e68ffea8b7d80117b332c8efe
mm/filemap.c: avoid always dirtying mapping->flags on O_DIRECT

commit 7fcbbaf18392f0b17c95e2f033c8ccf87eecde1d upstream.

In some testing I ran today (some fio jobs that spread over two nodes),
we end up spending 40% of the time in filemap_check_errors().  That
smells fishy.  Looking further, this is basically what happens:

blkdev_aio_read()
    generic_file_aio_read()
        filemap_write_and_wait_range()
            if (!mapping->nr_pages)
                filemap_check_errors()

and filemap_check_errors() always attempts two test_and_clear_bit() on
the mapping flags, thus dirtying it for every single invocation.  The
patch below tests each of these bits before clearing them, avoiding this
issue.  In my test case (4-socket box), performance went from 1.7M IOPS
to 4.0M IOPS.

Signed-off-by: Jens Axboe <axboe@fb.com>
Acked-by: Jeff Moyer <jmoyer@redhat.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
mm/filemap.c