Re: [PATCH 2/6] PCI/MSI: add hooks to populate the msi_domain field

[Jiang Liu]

On 2014/12/9 20:12, Marc Zyngier wrote:
> Yijing,
> 
> On 09/12/14 11:57, Yijing Wang wrote:
>>>>> +void __weak pcibios_set_phb_msi_domain(struct pci_bus *bus)
>>>>> +{
>>>>> +}
>>>>> +
>>>>> +static void pci_set_bus_msi_domain(struct pci_bus *bus)
>>>>> +{
>>>>> +	struct pci_dev *bridge = bus->self;
>>>>> +
>>>>> +	if (!bridge)
>>>>> +		pcibios_set_phb_msi_domain(bus);
>>>>> +	else
>>>>> +		dev_set_msi_domain(&bus->dev, dev_get_msi_domain(&bridge->dev));
>>>>> +}
>>>>
>>>>
>>>> Hi Marc, we can not assume pci devices under same phb share the same msi irq domain,
>>>> now in x86, pci devices under the same phb may associate different msi irq domain.
>>
>> Hi Marc,
>>
>>>
>>> Well, this is not supposed to be a perfect solution yet, but instead a
>>> basis for discussion. What I'd like to find out is:
>>>
>>> - What is the minimum granularity for associating a device with its MSI
>>> domain in existing platforms?
>>
>> PCI device, after Gerry's msi irq domain patchset which now in linux-next,
>> in x86, we will find msi irq domain by pci_dev.
> 
> Are you *really* associating the MSI domain on a per pci-device basis?
> That is, you have devices on the same PCI bus talking to different MSI hw?
Hi Marc,
	This is a little wild:(
	On x86 platform with Intel VT-d(not the case for AMD-v),
interrupt remapping is tight to DMA remapping (IOMMU) unit.
For most common cases, IOMMU unit manages PCI bus and its sub-hierarchy.
But it may also manage a specific PCI device. This is typically used to
provide QoS for audio device by using dedicated IOMMU unit to avoid
resource contention on DMA remapping tables. BIOS uses ACPI table to
report PCI bus/device to IOMMU unit mapping relationship. (To be honest,
I have no really experience with such a hardware platform yet, just for
theoretical analysis)
	On the other hand, we now support hierarchy irqdomain. So to
support per-PCI IOMMU unit case, we need maintain irqdomain at PCI
device level.
	This piece of code from your [4/6] is flexible enough, which
retrieves msi_domain from PCI device, then fallback to PCI bus,
then fallback to platform specific method.
	domain = dev_get_msi_domain(&dev->dev);
	if (!domain && dev->bus->msi)
 		domain = dev->bus->msi->domain;
 	if (!domain)
 		domain = arch_get_pci_msi_domain(dev);
Thanks!
Gerry
> 
>> I generally agree your first patch which associate basic device with msi irq domain.
>>
>>> - What topology data structures do you use to find out what MSI
>>> controller a device should be matched with?
>>
>> Now only arm and arm64 use msi controller to setup/teardown msi irqs,
>> in arm, now msi controller saved in pci_sys_data, and for arm64, it seems
>> to be saved in pci_bus. For a more common method to find msi controller/irq domain,
>> I prefer pci_dev/device.
> 
> Forget about msi_controller, the whole goal of this series is to make it
> obsolete. On your x86 platform, what how do you identify which MSI
> domain should be associated with a given PCI device? Surely you must
> have a set of data structures or ACPI tables which give you that
> information.
> 
>>> - What in-tree platform already has this requirements?
>>
>> As mentioned above, x86 does.
> 
> Let me rephrase that in a non-ambiguous manner: can you point me to a
> file implementing this in mainline?
> 
> Thanks,
> 
> 	M.
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/