svcrdma: Relieve contention on sc_send_lock.
authorChuck Lever <chuck.lever@oracle.com>
Tue, 9 Feb 2021 15:32:20 +0000 (10:32 -0500)
committerChuck Lever <chuck.lever@oracle.com>
Tue, 17 Aug 2021 15:47:53 +0000 (11:47 -0400)
commitb6c2bfea096ba22583f1071c10ce0745804b9b95
treed6a74441d55852b6335b4fe935300550d2bb993a
parent6c8c84f525100a1cade5698320b4abe43062e159
svcrdma: Relieve contention on sc_send_lock.

/proc/lock_stat indicates the the sc_send_lock is heavily
contended when the server is under load from a single client.

To address this, convert the send_ctxt free list to an llist.
Returning an item to the send_ctxt cache is now waitless, which
reduces the instruction path length in the single-threaded Send
handler (svc_rdma_wc_send).

The goal is to enable the ib_comp_wq worker to handle a higher
RPC/RDMA Send completion rate given the same CPU resources. This
change reduces CPU utilization of Send completion by 2-3% on my
server.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Reviewed-By: Tom Talpey <tom@talpey.com>
include/linux/sunrpc/svc_rdma.h
net/sunrpc/xprtrdma/svc_rdma_sendto.c
net/sunrpc/xprtrdma/svc_rdma_transport.c