scsi: lpfc: Fix interrupt assignments when multiple vectors are supported on same CPU

With certain platforms its possible pci_alloc_irq_vectors() may affinitize
irq vectors to multiple (all?) CPUs. The driver is currently assuming
exclusivity and vectors being doled out to different CPUs and is assigning
primary ownership of each vector to the first CPU in the mask.  The code
doesn't bother to check if the CPU already owns a vector and will
unconditionally overwrite the CPU to vector mapping. This causes the
relationships between eq's and cq's to get confused and gets worse when
CPUs start to offline. The net results are interrupts are skipped resulting
in mailbox timeouts and there are oopses in CPU offling flows.

Fix this changing up the primary vector assignment. Now assign the eq to a
CPU only if it is the CPU in the mask that does not have a prior
assignment. And once the primary ownership is assigned, break from the
loop. For CPUs that may have been set before but not the primary owner, the
lpfc_cpu_affinity_check() routine will balance the CPU to eq assignment.

Link: https://lore.kernel.org/r/20200706204230.130363-1-jsmart2021@gmail.com
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <jsmart2021@gmail.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 33d334a..4ba8202 100644 (file)
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -11522,9 +11522,9 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba)
        char *name;
        const struct cpumask *aff_mask = NULL;
        unsigned int cpu = 0, cpu_cnt = 0, cpu_select = nr_cpu_ids;
+       struct lpfc_vector_map_info *cpup;
        struct lpfc_hba_eq_hdl *eqhdl;
        const struct cpumask *maskp;
-       bool first;
        unsigned int flags = PCI_IRQ_MSIX;
 
        /* Set up MSI-X multi-message vectors */
@@ -11597,18 +11597,28 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba)
                } else {
                        maskp = pci_irq_get_affinity(phba->pcidev, index);
 
-                       first = true;
                        /* Loop through all CPUs associated with vector index */
                        for_each_cpu_and(cpu, maskp, cpu_present_mask) {
+                               cpup = &phba->sli4_hba.cpu_map[cpu];
+
                                /* If this is the first CPU thats assigned to
                                 * this vector, set LPFC_CPU_FIRST_IRQ.
+                                *
+                                * With certain platforms its possible that irq
+                                * vectors are affinitized to all the cpu's.
+                                * This can result in each cpu_map.eq to be set
+                                * to the last vector, resulting in overwrite
+                                * of all the previous cpu_map.eq.  Ensure that
+                                * each vector receives a place in cpu_map.
+                                * Later call to lpfc_cpu_affinity_check will
+                                * ensure we are nicely balanced out.
                                 */
+                               if (cpup->eq != LPFC_VECTOR_MAP_EMPTY)
+                                       continue;
                                lpfc_assign_eq_map_info(phba, index,
-                                                       first ?
-                                                       LPFC_CPU_FIRST_IRQ : 0,
+                                                       LPFC_CPU_FIRST_IRQ,
                                                        cpu);
-                               if (first)
-                                       first = false;
+                               break;
                        }
                }
        }