Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent

[Robin Murphy]

On 2022-09-08 10:51, Radovanovic, Aleksandar wrote:
> [AMD Official Use Only - General]
>
>
>
> > -----Original Message-----
> > From: Marc Zyngier <maz@xxxxxxxxxx>
> > Sent: 08 September 2022 09:08
> > To: Radovanovic, Aleksandar <aleksandar.radovanovic@xxxxxxx>
> > Cc: Jason Gunthorpe <jgg@xxxxxxxxxx>; Gupta, Nipun
> > <Nipun.Gupta@xxxxxxx>; robh+dt@xxxxxxxxxx;
> > krzysztof.kozlowski+dt@xxxxxxxxxx; gregkh@xxxxxxxxxxxxxxxxxxx;
> > rafael@xxxxxxxxxx; eric.auger@xxxxxxxxxx; alex.williamson@xxxxxxxxxx;
> > cohuck@xxxxxxxxxx; Gupta, Puneet (DCG-ENG)
> > <puneet.gupta@xxxxxxx>; song.bao.hua@xxxxxxxxxxxxx;
> > mchehab+huawei@xxxxxxxxxx; f.fainelli@xxxxxxxxx;
> > jeffrey.l.hugo@xxxxxxxxx; saravanak@xxxxxxxxxx;
> > Michael.Srba@xxxxxxxxx; mani@xxxxxxxxxx; yishaih@xxxxxxxxxx;
> > robin.murphy@xxxxxxx; will@xxxxxxxxxx; joro@xxxxxxxxxx;
> > masahiroy@xxxxxxxxxx; ndesaulniers@xxxxxxxxxx; linux-arm-
> > kernel@xxxxxxxxxxxxxxxxxxx; linux-kbuild@xxxxxxxxxxxxxxx; linux-
> > kernel@xxxxxxxxxxxxxxx; devicetree@xxxxxxxxxxxxxxx; kvm@xxxxxxxxxxxxxxx;
> > okaya@xxxxxxxxxx; Anand, Harpreet <harpreet.anand@xxxxxxx>; Agarwal,
> > Nikhil <nikhil.agarwal@xxxxxxx>; Simek, Michal <michal.simek@xxxxxxx>;
> > git (AMD-Xilinx) <git@xxxxxxx>
> > Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its
> > domain as parent
> >
> > [CAUTION: External Email]
> >   
> > OK, so you definitely need a mapping, but it cannot be a translation, and it
> > needs to be in all the possible address spaces. OMG.
>
> Could you elaborate why it needs to be in all the possible address spaces? I'm in no way familiar with kernel IOVA allocation, so not sure I understand this requirement. Note that each CDX device will have its own unique StreamID (in general case, equal to DeviceID sent to the GIC), so, from a SMMU perspective, the mapping can be specific to that device. As long as that IOVA is not allocated to any DMA region for _that_ device, things should be OK? But, I appreciate it might not be that simple from a kernel perspective.

That's the point - any device could could have its own mapping, 
therefore that hole has to be punched in *every* mapping that any of 
those devices could use, so that MSI writes don't unexpectedly fault, or 
corrupt memory if that address is free to be used to map a DMA buffer. 
At least the HiSilicon PCI quirk is functionally similar (for slightly 
different underlying reasons) so there's already precedent and an 
example that you can follow to a reasonable degree.

Robin.