xprtrdma: Fix cwnd update ordering
authorChuck Lever <chuck.lever@oracle.com>
Mon, 19 Apr 2021 18:02:41 +0000 (14:02 -0400)
committerTrond Myklebust <trond.myklebust@hammerspace.com>
Mon, 26 Apr 2021 13:24:05 +0000 (09:24 -0400)
commit35d8b10a25884050bb3b0149b62c3818ec59f77c
treed5d3dd7bca87beb2f9a957c73d2a3bdc2696a1d1
parent9e3ca33b62d4878f6ae39776abb6deebb37db597
xprtrdma: Fix cwnd update ordering

After a reconnect, the reply handler is opening the cwnd (and thus
enabling more RPC Calls to be sent) /before/ rpcrdma_post_recvs()
can post enough Receive WRs to receive their replies. This causes an
RNR and the new connection is lost immediately.

The race is most clearly exposed when KASAN and disconnect injection
are enabled. This slows down rpcrdma_rep_create() enough to allow
the send side to post a bunch of RPC Calls before the Receive
completion handler can invoke ib_post_recv().

Fixes: 2ae50ad68cd7 ("xprtrdma: Close window between waking RPC senders and posting Receives")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
net/sunrpc/xprtrdma/rpc_rdma.c
net/sunrpc/xprtrdma/verbs.c
net/sunrpc/xprtrdma/xprt_rdma.h