md/raid1: Avoid lock contention from wake_up()
authorJack Wang <jinpu.wang@ionos.com>
Wed, 5 Jul 2023 11:32:27 +0000 (13:32 +0200)
committerSong Liu <song@kernel.org>
Thu, 27 Jul 2023 07:13:30 +0000 (00:13 -0700)
commit21bd9a68fef47c4f0e951be9a6fac9745cee1bab
treed69ed66090a034c81c0cd33713f1aa44f735db1d
parente24ed04389f9619e0aaef615a8948633c182a8b0
md/raid1: Avoid lock contention from wake_up()

wake_up is called unconditionally in a few paths such as make_request(),
which cause lock contention under high concurrency workload like below
    raid1_end_write_request
     wake_up
      __wake_up_common_lock
       spin_lock_irqsave

Improve performance by only call wake_up() if waitqueue is not empty

Fio test script:

[global]
name=random reads and writes
ioengine=libaio
direct=1
readwrite=randrw
rwmixread=70
iodepth=64
buffered=0
filename=/dev/md0
size=1G
runtime=30
time_based
randrepeat=0
norandommap
refill_buffers
ramp_time=10
bs=4k
numjobs=400
group_reporting=1
[job1]

Test result with 2 ramdisk in raid1 on a Intel Broadwell 56 cores server.

Before this patch       With this patch
READ BW=4621MB/s BW=7337MB/s
WRITE BW=1980MB/s BW=3144MB/s

The patch is inspired by Yu Kuai's change for raid10:
https://lore.kernel.org/r/20230621105728.1268542-1-yukuai1@huaweicloud.com

Cc: Yu Kuai <yukuai3@huawei.com>
Signed-off-by: Jack Wang <jinpu.wang@ionos.com>
Reviewed-by: Yu Kuai <yukuai3@huawei.com>
Link: https://lore.kernel.org/r/20230705113227.148494-1-jinpu.wang@ionos.com
Signed-off-by: Song Liu <song@kernel.org>
drivers/md/raid1.c