[+cc Joerg, Will, Robin]
On Mon, Jan 09, 2023 at 04:35:25PM +0800, guo.ziliang@xxxxxxxxxx wrote:
bridge base/limit(memory behind in lspci info, outbound pcie address/size)
region is used to route outbound mem read/write transaction to ep. This
base/limit region also may filter out inbound transactions which will
result in inbound(eg: dma) transaction fail.
For example, if bridge base/limit is [0x20000000, 0x203fffff], system ram
is [0x20000000, 0x27ffffff]. The inbound mapping is usually 1:1 equal
mapping. When allocated system ram for inbound tansaction is 0x20004000
(any in bridge base/limit), this inbound transactions will be filter out.
AER may report 'UnsupReq' on inbound mem read/write transactions if address
is in this base/limit region, but not all pcie AER enabled or work well. We
warn it also in bridge pci address setting phase.
Signed-off-by: Chen Lin <chen.lin5@xxxxxxxxxx>
This would need the 0-day warnings cleaned up, of course.
---
drivers/pci/setup-bus.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index b4096598dbcb..1a9f527d2317 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -608,6 +608,24 @@ static void pci_setup_bridge_io(struct pci_dev *bridge)
pci_write_config_dword(bridge, PCI_IO_BASE_UPPER16, io_upper16);
}
+static void check_bridge_region_overlaps_systemram(struct pci_dev *bridge,
+ struct pci_bus_region *region)
+{
+ int is_ram;
+
+ /*
+ * bridge base/limit(memory behind) region may filter out inbound
+ * transactions which will result in inbound(eg: dma) fail of ep.
+ * AER may report it if enabled, we warn it also.
+ */
+ is_ram = region_intersects(region->start, region->end - region->start + 1,
+ IORESOURCE_SYSTEM_RAM, IORES_DESC_NONE);
+ if (is_ram == REGION_INTERSECTS) {
+ pci_warn(bridge, "%#012llx..%#012llx bridge base/limit region overlaps with system ram, may result in inbound fail\n",
+ region->start, region->end);
This compares PCI bus addresses (from struct pci_bus_region) with CPU
physical addresses (the struct resources used by region_intersects()).
But I don't think you can do that directly because an IOMMU might map
those PCI bus addresses to something different before a DMA gets to
system RAM.
I see that you say "The inbound mapping is usually 1:1 equal mapping"
above, so maybe I'm missing something. Maybe the IOMMU folks will
clue me in.
+ }
+}
+
static void pci_setup_bridge_mmio(struct pci_dev *bridge)
{
struct resource *res;
@@ -621,6 +639,7 @@ static void pci_setup_bridge_mmio(struct pci_dev *bridge)
l = (region.start >> 16) & 0xfff0;
l |= region.end & 0xfff00000;
pci_info(bridge, " bridge window %pR\n", res);
+ check_bridge_region_overlaps_systemram(bridge, ®ion);
} else {
l = 0x0000fff0;
}
@@ -652,6 +671,7 @@ static void pci_setup_bridge_mmio_pref(struct pci_dev *bridge)
lu = upper_32_bits(region.end);
}
pci_info(bridge, " bridge window %pR\n", res);
+ check_bridge_region_overlaps_systemram(bridge, ®ion);
} else {
l = 0x0000fff0;
}
--
2.15.2