lib/irq_poll: Support schedules in non-interrupt contexts

Do not assume irq_poll_sched() is called from an interrupt context only.
So use raise_softirq_irqoff() instead of __raise_softirq_irqoff() so it
will kick the ksoftirqd if the schedule is from a non-interrupt context.

This is required for RDMA drivers, like soft iwarp, that generate cq
completion notifications in a workqueue or kthread context.  Without this
change, siw completion notifications to the ULP can take several hundred
usecs, depending on the system load.

Signed-off-by: Steve Wise <swise@opengridcomputing.com>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>

diff --git a/lib/irq_poll.c b/lib/irq_poll.c
index 86a7099..2f17b48 100644 (file)
--- a/lib/irq_poll.c
+++ b/lib/irq_poll.c
@@ -35,7 +35,7 @@ void irq_poll_sched(struct irq_poll *iop)
 
        local_irq_save(flags);
        list_add_tail(&iop->list, this_cpu_ptr(&blk_cpu_iopoll));
-       __raise_softirq_irqoff(IRQ_POLL_SOFTIRQ);
+       raise_softirq_irqoff(IRQ_POLL_SOFTIRQ);
        local_irq_restore(flags);
 }
 EXPORT_SYMBOL(irq_poll_sched);