Re: [PATCH v2 0/5] PCI: dwc: improve msi handling

[Marc Zyngier]

On 2020-09-29 19:02, Jon Hunter wrote:
> On 29/09/2020 18:25, Marc Zyngier wrote:
> > On 2020-09-29 14:22, Jon Hunter wrote:
> > > Hi Jisheng,
> > >
> > > On 29/09/2020 11:48, Jisheng Zhang wrote:
> > > > Hi Jon,
> > > >
> > > > On Fri, 25 Sep 2020 09:53:45 +0100 Jon Hunter wrote:
> > > >
> > > > > On 24/09/2020 12:05, Jisheng Zhang wrote:
> > > > > > Improve the msi code:
> > > > > > 1. Add proper error handling.
> > > > > > 2. Move dw_pcie_msi_init() from each users to designware host to 
> > > > > > solve
> > > > > > msi page leakage in resume path.
> > > > >
> > > > > Apologies if this is slightly off topic, but I have been meaning to 
> > > > > ask
> > > > > about MSIs and PCI. On Tegra194 which uses the DWC PCI driver,
> > > > > whenever we
> > > > > hotplug CPUs we see the following warnings ...
> > > > >
> > > > >  [      79.068351] WARNING KERN IRQ70: set affinity failed(-22).
> > > > >  [      79.068362] WARNING KERN IRQ71: set affinity failed(-22).
> > > >
> > > > I tried to reproduce this issue on Synaptics SoC, but can't 
> > > > reproduce
> > > > it.
> > > > Per my understanding of the code in kernel/irq/cpuhotplug.c, this
> > > > warning
> > > > happened when we migrate irqs away from the offline cpu, this 
> > > > implicitly
> > > > implies that before this point the irq has bind to the offline cpu,
> > > > but how
> > > > could this happen given current dw_pci_msi_set_affinity() 
> > > > implementation
> > > > always return -EINVAL
> > >
> > > By default the smp_affinity should be set so that all CPUs can be
> > > interrupted ...
> > >
> > > $ cat /proc/irq/70/smp_affinity
> > > 0xff
> > >
> > > In my case there are 8 CPUs and so 0xff implies that the interrupt 
> > > can
> > > be triggered on any of the 8 CPUs.
> > >
> > > Do you see the set_affinity callback being called for the DWC irqchip 
> > > in
> > > migrate_one_irq()?
> >
> > The problem is common to all MSI implementations that end up muxing
> > all the end-point MSIs into a single interrupt. With these systems,
> > you cannot set the affinity of individual MSIs (they don't target a
> > CPU, they target another interrupt... braindead). Only the mux
> > interrupt can have its affinity changed.
> >
> > So returning -EINVAL is the right thing to do.
>
> Right, so if that is the case, then surely there should be some way to
> avoid these warnings because they are not relevant?

I don't think there is a way to do this, because the core code
doesn't (and cannot) know the exact interrupt topology.

The only alternative would be to change the affinity of the mux
interrupt when a MSI affinity changes, but that tends to break
userspace (irqbalance, for example).

        M.
--
Jazz is not dead. It just smells funny...