RE: [PATCH RFC v2 02/18] irq/dev-msi: Add support for a new DEV_MSI irq domain

[Dey, Megha]

Hi Jason,

> -----Original Message-----
> From: Jason Gunthorpe <jgg@xxxxxxxxxxxx>
> Sent: Wednesday, August 5, 2020 5:19 PM
> To: Dey, Megha <megha.dey@xxxxxxxxx>
> Cc: Marc Zyngier <maz@xxxxxxxxxx>; Jiang, Dave <dave.jiang@xxxxxxxxx>;
> vkoul@xxxxxxxxxx; bhelgaas@xxxxxxxxxx; rafael@xxxxxxxxxx;
> gregkh@xxxxxxxxxxxxxxxxxxx; tglx@xxxxxxxxxxxxx; hpa@xxxxxxxxx;
> alex.williamson@xxxxxxxxxx; Pan, Jacob jun <jacob.jun.pan@xxxxxxxxx>; Raj,
> Ashok <ashok.raj@xxxxxxxxx>; Liu, Yi L <yi.l.liu@xxxxxxxxx>; Lu, Baolu
> <baolu.lu@xxxxxxxxx>; Tian, Kevin <kevin.tian@xxxxxxxxx>; Kumar, Sanjay K
> <sanjay.k.kumar@xxxxxxxxx>; Luck, Tony <tony.luck@xxxxxxxxx>; Lin, Jing
> <jing.lin@xxxxxxxxx>; Williams, Dan J <dan.j.williams@xxxxxxxxx>;
> kwankhede@xxxxxxxxxx; eric.auger@xxxxxxxxxx; parav@xxxxxxxxxxxx;
> Hansen, Dave <dave.hansen@xxxxxxxxx>; netanelg@xxxxxxxxxxxx;
> shahafs@xxxxxxxxxxxx; yan.y.zhao@xxxxxxxxxxxxxxx; pbonzini@xxxxxxxxxx;
> Ortiz, Samuel <samuel.ortiz@xxxxxxxxx>; Hossain, Mona
> <mona.hossain@xxxxxxxxx>; dmaengine@xxxxxxxxxxxxxxx; linux-
> kernel@xxxxxxxxxxxxxxx; x86@xxxxxxxxxx; linux-pci@xxxxxxxxxxxxxxx;
> kvm@xxxxxxxxxxxxxxx
> Subject: Re: [PATCH RFC v2 02/18] irq/dev-msi: Add support for a new DEV_MSI
> irq domain
> 
> On Thu, Aug 06, 2020 at 12:13:24AM +0000, Dey, Megha wrote:
> > > Well, I had suggested to pass in the parent struct device, but it
> > > could certainly use an irq_domain instead:
> > >
> > >   platform_msi_assign_domain(dev,
> > > device_to_iommu(p_dev)->ir_domain);
> > >
> > > Or
> > >
> > >   platform_msi_assign_domain(dev, pdev->msi_domain)
> > >
> > > ?
> > >
> > > Any maybe the natural expression is to add a version of
> > > platform_msi_create_device_domain() that accepts a parent
> > > irq_domain() and if the device doesn't already have a msi_domain
> > > then it creates one. Might be too tricky to manage lifetime of the new
> irq_domain though..
> > >
> > > It feels cleaner to me if everything related to this is contained in
> > > the platform_msi and the driver using it. Not sure it makes sense to
> > > involve the iommu?
> >
> > Well yeah something like this can be done, but what is the missing
> > piece is where the IRQ domain actually gets created, i.e where this
> > new version of platform_msi_create_device_domain() is called. That is
> > the only piece that is currently done in the IOMMU driver only for DSA
> > mdev. Not that all devices need to do it this way.. do you have
> > suggestions as to where you want to call this function?
> 
> Oops, I was thinking of platform_msi_domain_alloc_irqs() not
> create_device_domain()
> 
> ie call it in the device driver that wishes to consume the extra MSIs.
> 
> Is there a harm if each device driver creates a new irq_domain for its use?

Well, the only harm is if we want to reuse the irq domain.

As of today, we only have DSA mdev which uses the dev-msi domain. In the IRQ domain hierarchy,
We will have this:

Vector-> intel-ir->dev-msi 

So tmrw if we have a new device, which would also want to have the intel-ir as the parent and use the same domain ops, we will simply be creating a copy of this IRQ domain, which may not be very fruitful.

But apart from that, I don't think there are any issues..

What do you think is the best approach here?
> 
> Jason