Re: [PATCH v3 0/3] arm64: realm: Fix DMA address for devices

[Suzuki K Poulose]

On 04/03/2025 13:40, Marek Szyprowski wrote:
> Hi,
>
> On 03.03.2025 12:35, Suzuki K Poulose wrote:
> > On 27/02/2025 14:41, Suzuki K Poulose wrote:
> > > Linux can be run as a Confidential Guest in Arm CCA from Linux v6.13.
> > > The address
> > > space (GPA or IPA) of a Realm VM is split into two halves, with
> > > private bottom
> > > half and shared top half. In Linux we treat the "top" bit of the IPA
> > > space as
> > > an attribute, to indicate whether it is shared or not (MSB == 1
> > > implies shared).
> > > Stage2 (GPA to PA) translations used by the CPU accesses, cover the
> > > full IPA space,
> > > and are managed by RMM. The "top" bit as attribute is only a software
> > > construct.
> > >
> > > At present any device passed through to a Realm is treated as
> > > untrusted and the
> > > Realm uses bounce buffering for any DMA, using the "decrypted"
> > > (shared) DMA
> > > buffers (i.e., IPA with top bit set). In Linux, we only send the
> > > "DMA" address
> > > masking the "top" bit. In Arm CCA, SMMU for untrusted devices are
> > > managed by the
> > > non-secure Host and thus it can be confusing for the host/device when
> > > an unmasked
> > > address is provided. Given there could be other hypervisors than
> > > Linux/KVM
> > > running Arm CCA guests, the Realm Guest must adhere to a single
> > > convention for
> > > the DMA address. This gets further complicated when we add support
> > > for trusted
> > > devices, which can DMA into the full Realm memory space, once
> > > accepted. Thus,
> > > a DMA masked address (with "top" bit lost) will prevent a trusted
> > > device from
> > > accessing a shared buffer.
> > >
> > > To resolve this Arm has decided to standardise the DMA address used
> > > by the Realm
> > > to include the full IPA address bits (including the "top" bit, which
> > > Linux uses
> > > as an attribute). This implies, any DMA to a shared buffer must have
> > > the top bit
> > > of the IPA space set.
> > >
> > > There is already a provision to do this in phys_to_dma* and
> > > dma_to_phys(), but
> > > that is specific to AMD SME and is quite the opposite of what we need
> > > for Arm CCA.
> > > i.e., For Arm CCA we need to set the bit for "decrypted" DMA and
> > > clear the bit
> > > for "encrypted".
> > >
> > > This series converts the existing __sme_* helpers to a bit more
> > > generalised versions :
> > > dma_addr_decrypted() and dma_encrypted(). Also, while converting a
> > > DMA address back
> > > to CPU physical address requires clearing off any
> > > "encryption/decryption" bits.
> > > I have named this "dma_addr_canonical()". (The other options are :
> > >     * dma_addr_clear_encryption - Well, not just for encryption, but
> > > we clear decryption
> > >       too, so not ideal.
> > >     * dma_addr_normal
> > >     * dma_addr_clear
> > >     * dma_addr_default
> > >
> > > This also implies that the VMMs must take care to :
> > >
> > >    1. Create the S2-SMMU mappings for VFIO at the "unprotected" alias.
> > >    2. Always mask the "top" bit off any IPA it receives from the Realm
> > > for DMA.
> > >       KVM already does that today and no changes are required.
> > >
> > > A kvmtool branch with the changes above is available here [1]. There
> > > are two
> > > patches [2] & [3], that are really required on top of the Arm CCA
> > > support.
> > >
> > > Ideally it would be good to get this backported to v6.13 stable
> > > kernel releases
> > > to make sure that they are compliant with this change.
> >
> > Please could you take a look at this series and let us know your
> > thoughts ? If you are happy with the changes, are you happy to pull
> > this through the DMA MAP tree ? The relevant bits have been reviewed/
> > acked by people (arm64 and AMD bits).
>
> The changes look fine. However I won't be able to setup new dma-mapping
> git tree this week because I got really sick has to stay in bed. :/ If
> You don't want such delay, please merge it via ARM64 tree. Here is my:

Sorry to hear that. Hope you feel better soon.

> Acked-by: Marek Szyprowski <m.szyprowski@xxxxxxxxxxx>

Thanks Marek.

Btw, this series fixes the "Realm Guest" support for Linux, which was 
merged in v6.13. To be precise, this should have :

Fixes: 42be24a4178f ("arm64: Enable memory encrypt for Realms")

Will/Catalin,

Please let me know if you would like me to send the series with
all the Acks, Reviews and mainly the Fixes tag added ?

Kind regards
Suzuki

> Best regards