IB/rxe: Advance the consumer pointer before posting the CQE

A simple userspace application might poll the CQ, find a completion,
and then attempt to post a new WQE to the SQ. A spurious error can
occur if the userspace application detects a full SQ in the instant
before the kernel is able to advance the SQ consumer pointer.

This is noticeable when using single-entry SQs with ibv_rc_pingpong
if lots of kernel and userspace library debugging is enabled.

Signed-off-by: Andrew Boyer <andrew.boyer@dell.com>
Reviewed-by: Yonatan Cohen <yonatanc@mellanox.com>
Signed-off-by: Doug Ledford <dledford@redhat.com>

diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c
index 6c5e29d..d46c49b 100644 (file)
--- a/drivers/infiniband/sw/rxe/rxe_comp.c
+++ b/drivers/infiniband/sw/rxe/rxe_comp.c
@@ -420,11 +420,12 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
            (wqe->wr.send_flags & IB_SEND_SIGNALED) ||
            (qp->req.state == QP_STATE_ERROR)) {
                make_send_cqe(qp, wqe, &cqe);
+               advance_consumer(qp->sq.queue);
                rxe_cq_post(qp->scq, &cqe, 0);
+       } else {
+               advance_consumer(qp->sq.queue);
        }
 
-       advance_consumer(qp->sq.queue);
-
        /*
         * we completed something so let req run again
         * if it is trying to fence