powerpc: POWER7 optimised memcpy using VMX and enhanced prefetch
authorAnton Blanchard <anton@samba.org>
Wed, 30 May 2012 20:22:09 +0000 (20:22 +0000)
committerBenjamin Herrenschmidt <benh@kernel.crashing.org>
Tue, 3 Jul 2012 04:14:46 +0000 (14:14 +1000)
commitb3f271e86e5a440713716bb222e1aa1227994c50
treea215f44357cf20c9f2b342a7bdb840bb85ca4cb1
parentbce4b4bd91efab9dca693ac37c8ddf88103280d8
powerpc: POWER7 optimised memcpy using VMX and enhanced prefetch

Implement a POWER7 optimised memcpy using VMX and enhanced prefetch
instructions.

This is a copy of the POWER7 optimised copy_to_user/copy_from_user
loop. Detailed implementation and performance details can be found in
commit a66086b8197d (powerpc: POWER7 optimised
copy_to_user/copy_from_user using VMX).

I noticed memcpy issues when profiling a RAID6 workload:

.memcpy
.async_memcpy
.async_copy_data
.__raid_run_ops
.handle_stripe
.raid5d
.md_thread

I created a simplified testcase by building a RAID6 array with 4 1GB
ramdisks (booting with brd.rd_size=1048576):

# mdadm -CR -e 1.2 /dev/md0 --level=6 -n4 /dev/ram[0-3]

I then timed how long it took to write to the entire array:

# dd if=/dev/zero of=/dev/md0 bs=1M

Before: 892 MB/s
After:  999 MB/s

A 12% improvement.

Signed-off-by: Anton Blanchard <anton@samba.org>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
arch/powerpc/lib/Makefile
arch/powerpc/lib/memcpy_64.S
arch/powerpc/lib/memcpy_power7.S [new file with mode: 0644]