PCI: Avoid generating invalid ThunderX2 DMA aliases

On Cavium ThunderX2 arm64 SoCs (formerly known as Broadcom Vulcan), the PCI
topology is slightly unusual.  For a multi-node system, it looks like:

    00:00.0 PCI bridge to [bus 01-1e]
    01:0a.0 PCI-to-PCIe bridge to [bus 02-04]
    02:00.0 PCIe Root Port bridge to [bus 03-04] (XLATE_ROOT)
    03:00.0 PCIe Endpoint

pci_for_each_dma_alias() assumes IOMMU translation is done at the root of
the PCI hierarchy.  It generates 03:00.0, 01:0a.0, and 00:00.0 as DMA
aliases for 03:00.0 because buses 01 and 00 are non-PCIe buses that don't
carry the Requester ID.

Because the ThunderX2 IOMMU is at 02:00.0, the Requester IDs 01:0a.0 and
00:00.0 are never valid for the endpoint.  This quirk stops alias
generation at the XLATE_ROOT bridge so we won't generate 01:0a.0 or
00:00.0.

The current IOMMU code only maps the last alias (this is a separate bug in
itself).  Prior to this quirk, we only created IOMMU mappings for the
invalid Requester ID 00:00:0, which never matched any DMA transactions.

With this quirk, we create IOMMU mappings for a valid Requester ID, which
fixes devices with no aliases but leaves devices with aliases still broken.

The last alias for the endpoint is also used by the ARM GICv3 MSI-X code.
Without this quirk, the GIC Interrupt Translation Tables are setup with the
invalid Requester ID, and the MSI-X generated by the device fails to be
translated and routed.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=195447
Signed-off-by: Jayachandran C <jnair@caviumnetworks.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Acked-by: David Daney <david.daney@cavium.com>

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 178242d0d9dfcc306830e3d932f27df312010ebc..74344db4ed4aea5c7541c518367a4a35ad62fba1 100644 (file)
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -3956,6 +3956,20 @@ static void quirk_mic_x200_dma_alias(struct pci_dev *pdev)
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2260, quirk_mic_x200_dma_alias);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x2264, quirk_mic_x200_dma_alias);
 
+/*
+ * The IOMMU and interrupt controller on Broadcom Vulcan/Cavium ThunderX2 are
+ * associated not at the root bus, but at a bridge below. This quirk avoids
+ * generating invalid DMA aliases.
+ */
+static void quirk_bridge_cavm_thrx2_pcie_root(struct pci_dev *pdev)
+{
+       pdev->dev_flags |= PCI_DEV_FLAGS_BRIDGE_XLATE_ROOT;
+}
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9000,
+                               quirk_bridge_cavm_thrx2_pcie_root);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9084,
+                               quirk_bridge_cavm_thrx2_pcie_root);
+
 /*
  * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero)
  * class code.  Fix it.