tcp: add one skb cache for tx
authorEric Dumazet <edumazet@google.com>
Fri, 22 Mar 2019 15:56:39 +0000 (08:56 -0700)
committerDavid S. Miller <davem@davemloft.net>
Sun, 24 Mar 2019 01:57:38 +0000 (21:57 -0400)
commit472c2e07eef045145bc1493cc94a01c87140780a
treee1980ba54472857684241be387d8f25e561be901
parentdc05360fee660a9dbe59824b3f7896534210432b
tcp: add one skb cache for tx

On hosts with a lot of cores, RPC workloads suffer from heavy contention on slab spinlocks.

    20.69%  [kernel]       [k] queued_spin_lock_slowpath
     5.64%  [kernel]       [k] _raw_spin_lock
     3.83%  [kernel]       [k] syscall_return_via_sysret
     3.48%  [kernel]       [k] __entry_text_start
     1.76%  [kernel]       [k] __netif_receive_skb_core
     1.64%  [kernel]       [k] __fget

For each sendmsg(), we allocate one skb, and free it at the time ACK packet comes.

In many cases, ACK packets are handled by another cpus, and this unfortunately
incurs heavy costs for slab layer.

This patch uses an extra pointer in socket structure, so that we try to reuse
the same skb and avoid these expensive costs.

We cache at most one skb per socket so this should be safe as far as
memory pressure is concerned.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Acked-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
include/net/sock.h
net/ipv4/tcp.c