powerpc/powernv: Fix stale iommu table base after VFIO

The powernv platform uses @dma_iommu_ops for non-bypass DMA. These ops
need an iommu_table pointer which is stored in
dev->archdata.iommu_table_base. It is initialized during
pcibios_setup_device() which handles boot time devices. However when a
device is taken from the system in order to pass it through, the
default IOMMU table is destroyed but the pointer in a device is not
updated; also when a device is returned back to the system, a new
table pointer is not stored in dev->archdata.iommu_table_base either.
So when a just returned device tries using IOMMU, it crashes on
accessing stale iommu_table or its members.

This calls set_iommu_table_base() when the default window is created.
Note it used to be there before but was wrongly removed (see "fixes").
It did not appear before as these days most devices simply use bypass.

This adds set_iommu_table_base(NULL) when a device is taken from the
system to make it clear that IOMMU DMA cannot be used past that point.

Fixes: c4e9d3c1e65a ("powerpc/powernv/pseries: Rework device adding to IOMMU groups")
Cc: stable@vger.kernel.org # v5.0+
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>

diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 6b0caa2..dc4a3c5 100644 (file)
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -2462,6 +2462,14 @@ static long pnv_pci_ioda2_setup_default_config(struct pnv_ioda_pe *pe)
        if (!pnv_iommu_bypass_disabled)
                pnv_pci_ioda2_set_bypass(pe, true);
 
+       /*
+        * Set table base for the case of IOMMU DMA use. Usually this is done
+        * from dma_dev_setup() which is not called when a device is returned
+        * from VFIO so do it here.
+        */
+       if (pe->pdev)
+               set_iommu_table_base(&pe->pdev->dev, tbl);
+
        return 0;
 }
 
@@ -2549,6 +2557,8 @@ static void pnv_ioda2_take_ownership(struct iommu_table_group *table_group)
        pnv_pci_ioda2_unset_window(&pe->table_group, 0);
        if (pe->pbus)
                pnv_ioda_setup_bus_dma(pe, pe->pbus);
+       else if (pe->pdev)
+               set_iommu_table_base(&pe->pdev->dev, NULL);
        iommu_tce_table_put(tbl);
 }