xsk: Fix race in SKB mode transmit with shared cq
authorMagnus Karlsson <magnus.karlsson@intel.com>
Fri, 18 Dec 2020 13:45:24 +0000 (14:45 +0100)
committerDaniel Borkmann <daniel@iogearbox.net>
Fri, 18 Dec 2020 15:10:21 +0000 (16:10 +0100)
commitf09ced4053bc0a2094a12b60b646114c966ef4c6
treeda2345ffe99bd31cf1db279a97bbdfe4bac268f8
parent8bee683384087a6275c9183a483435225f7bb209
xsk: Fix race in SKB mode transmit with shared cq

Fix a race when multiple sockets are simultaneously calling sendto()
when the completion ring is shared in the SKB case. This is the case
when you share the same netdev and queue id through the
XDP_SHARED_UMEM bind flag. The problem is that multiple processes can
be in xsk_generic_xmit() and call the backpressure mechanism in
xskq_prod_reserve(xs->pool->cq). As this is a shared resource in this
specific scenario, a race might occur since the rings are
single-producer single-consumer.

Fix this by moving the tx_completion_lock from the socket to the pool
as the pool is shared between the sockets that share the completion
ring. (The pool is not shared when this is not the case.) And then
protect the accesses to xskq_prod_reserve() with this lock. The
tx_completion_lock is renamed cq_lock to better reflect that it
protects accesses to the potentially shared completion ring.

Fixes: 35fcde7f8deb ("xsk: support for Tx")
Reported-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Björn Töpel <bjorn.topel@intel.com>
Link: https://lore.kernel.org/bpf/20201218134525.13119-2-magnus.karlsson@gmail.com
include/net/xdp_sock.h
include/net/xsk_buff_pool.h
net/xdp/xsk.c
net/xdp/xsk_buff_pool.c