net-zerocopy: Copy straggler unaligned data for TCP Rx. zerocopy.
authorArjun Roy <arjunroy@google.com>
Wed, 2 Dec 2020 22:53:42 +0000 (14:53 -0800)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Fri, 26 Nov 2021 09:39:14 +0000 (10:39 +0100)
commit5f7aadf03f98d641b32837f32a401e543d6e5a69
tree66dc16234b702829246a883fd1951b455c6c04ee
parent8da80ec6d4f7c90e0ea5b9edb37c211adc3e9a01
net-zerocopy: Copy straggler unaligned data for TCP Rx. zerocopy.

[ Upstream commit 18fb76ed53865c1b5d5f0157b1b825704590beb5 ]

When TCP receive zerocopy does not successfully map the entire
requested space, it outputs a 'hint' that the caller should recvmsg().

Augment zerocopy to accept a user buffer that it tries to copy this
hint into - if it is possible to copy the entire hint, it will do so.
This elides a recvmsg() call for received traffic that isn't exactly
page-aligned in size.

This was tested with RPC-style traffic of arbitrary sizes. Normally,
each received message required at least one getsockopt() call, and one
recvmsg() call for the remaining unaligned data.

With this change, almost all of the recvmsg() calls are eliminated,
leading to a savings of about 25%-50% in number of system calls
for RPC-style workloads.

Signed-off-by: Arjun Roy <arjunroy@google.com>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
include/uapi/linux/tcp.h
net/ipv4/tcp.c