net: use indirect call wrappers for skb_copy_datagram_iter()
authorEric Dumazet <edumazet@google.com>
Wed, 25 Mar 2020 02:23:21 +0000 (19:23 -0700)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Sat, 2 May 2020 06:49:00 +0000 (08:49 +0200)
commitf1610480602a475323f9c335540d47b3a95c62fd
treea61b7aa1c985ac43bc9c7f6d22e7cd3775a9beb4
parentec9cf8afcd641263bac73a608440a2b22e026a0c
net: use indirect call wrappers for skb_copy_datagram_iter()

commit 29f3490ba9d2399d3d1b20c4aa74592d92bd4e11 upstream.

TCP recvmsg() calls skb_copy_datagram_iter(), which
calls an indirect function (cb pointing to simple_copy_to_iter())
for every MSS (fragment) present in the skb.

CONFIG_RETPOLINE=y forces a very expensive operation
that we can avoid thanks to indirect call wrappers.

This patch gives a 13% increase of performance on
a single flow, if the bottleneck is the thread reading
the TCP socket.

Fixes: 950fcaecd5cc ("datagram: consolidate datagram copy to iter helpers")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
net/core/datagram.c