rxrpc: Fix race between incoming ACK parser and retransmitter
authorDavid Howells <dhowells@redhat.com>
Thu, 11 Jun 2020 20:57:00 +0000 (21:57 +0100)
committerDavid S. Miller <davem@davemloft.net>
Fri, 12 Jun 2020 01:18:22 +0000 (18:18 -0700)
There's a race between the retransmission code and the received ACK parser.
The problem is that the retransmission loop has to drop the lock under
which it is iterating through the transmission buffer in order to transmit
a packet, but whilst the lock is dropped, the ACK parser can crank the Tx
window round and discard the packets from the buffer.

The retransmission code then updated the annotations for the wrong packet
and a later retransmission thought it had to retransmit a packet that
wasn't there, leading to a NULL pointer dereference.

Fix this by:

 (1) Moving the annotation change to before we drop the lock prior to
     transmission.  This means we can't vary the annotation depending on
     the outcome of the transmission, but that's fine - we'll retransmit
     again later if it failed now.

 (2) Skipping the packet if the skb pointer is NULL.

The following oops was seen:

BUG: kernel NULL pointer dereference, address: 000000000000002d
Workqueue: krxrpcd rxrpc_process_call
RIP: 0010:rxrpc_get_skb+0x14/0x8a
...
Call Trace:
 rxrpc_resend+0x331/0x41e
 ? get_vtime_delta+0x13/0x20
 rxrpc_process_call+0x3c0/0x4ac
 process_one_work+0x18f/0x27f
 worker_thread+0x1a3/0x247
 ? create_worker+0x17d/0x17d
 kthread+0xe6/0xeb
 ? kthread_delayed_work_timer_fn+0x83/0x83
 ret_from_fork+0x1f/0x30

Fixes: 248f219cb8bc ("rxrpc: Rewrite the data and ack handling code")
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
net/rxrpc/call_event.c

index 61a51c2..aa1c8ee 100644 (file)
@@ -248,7 +248,18 @@ static void rxrpc_resend(struct rxrpc_call *call, unsigned long now_j)
                if (anno_type != RXRPC_TX_ANNO_RETRANS)
                        continue;
 
+               /* We need to reset the retransmission state, but we need to do
+                * so before we drop the lock as a new ACK/NAK may come in and
+                * confuse things
+                */
+               annotation &= ~RXRPC_TX_ANNO_MASK;
+               annotation |= RXRPC_TX_ANNO_RESENT;
+               call->rxtx_annotations[ix] = annotation;
+
                skb = call->rxtx_buffer[ix];
+               if (!skb)
+                       continue;
+
                rxrpc_get_skb(skb, rxrpc_skb_got);
                spin_unlock_bh(&call->lock);
 
@@ -262,24 +273,6 @@ static void rxrpc_resend(struct rxrpc_call *call, unsigned long now_j)
 
                rxrpc_free_skb(skb, rxrpc_skb_freed);
                spin_lock_bh(&call->lock);
-
-               /* We need to clear the retransmit state, but there are two
-                * things we need to be aware of: A new ACK/NAK might have been
-                * received and the packet might have been hard-ACK'd (in which
-                * case it will no longer be in the buffer).
-                */
-               if (after(seq, call->tx_hard_ack)) {
-                       annotation = call->rxtx_annotations[ix];
-                       anno_type = annotation & RXRPC_TX_ANNO_MASK;
-                       if (anno_type == RXRPC_TX_ANNO_RETRANS ||
-                           anno_type == RXRPC_TX_ANNO_NAK) {
-                               annotation &= ~RXRPC_TX_ANNO_MASK;
-                               annotation |= RXRPC_TX_ANNO_UNACK;
-                       }
-                       annotation |= RXRPC_TX_ANNO_RESENT;
-                       call->rxtx_annotations[ix] = annotation;
-               }
-
                if (after(call->tx_hard_ack, seq))
                        seq = call->tx_hard_ack;
        }