Re: [PATCH v16 00/45] arm64: Support for Arm CCA in KVM

[Gavin Shan]

Hi Alper and Suzuki,

On 8/12/26 4:04 PM, Suzuki K Poulose wrote:
> Hi Alper, Gavin
>
> On 12/08/2026 04:25, Alper Gun wrote:
> > On Tue, Aug 11, 2026 at 8:08 PM Gavin Shan <gshan@xxxxxxxxxx> wrote:
> > > As the following calltrace indicates, -EAGAIN is returned from tf-rmm::update_ripas()
> > > because true is returned from s2tte_drain_pending() for the S2TTE corresponding to
> > > IPA 0x80000000. Linux host received error (RMI_ERROR_RTT, level=3) in ripas_change().
> > > Upon this specific error and the IPA range [0x80000000 0x90000000], find_map_level()
> > > returns level of 2, and realm_create_rtt_levels() returns 0 without populating any
> > > RTTs. After that, rmi_rtt_set_ripas() is re-executed and the above loop starts over
> > > again.
> > >
> > >     Linux host
> > >     ==========
> > >     kvm_arch_vcpu_ioctl_run                     // cca/host-v16
> > >       check_vcpu_requests
> > >         kvm_check_request
> > >           kvm_rec_handle_request
> > >             kvm_complete_ripas_change
> > >               realm_set_ipa_state
> > >                 ripas_change
> > >                   rmi_rtt_set_ripas
> > >                     SMC_RMI_RTT_SET_RIPAS
> > >
> > >     TF-RMM
> > >     ======
> > >     SMC_RMI_RTT_SET_RIPAS                      // tf-rmm/topics/rmm-v2.0-poc_3
> > >       smc_rtt_set_ripas
> > >         s2tt_walk_lock_unlock
> > >         rtt_set_ripas_range
> > >           update_ripas
> > >             s2tte_drain_pending                // true, returns -EAGAIN
> > >
> > > The problem is the pending-bit for RTE corresponding IPA address 0x80000000 isn't cleared
> > > when SMC_RMI_RTT_SET_RIPAS is invoked. I didn't figure out how this bit is set and why
> > > it's not cleared in time.
>
> Thanks for the details.
>
> > >
> >
> > Hi Gavin, Suzuki,
> >
> > I think I ran into a similar issue on rmm-v2.0-poc_3 last week.
> > This looks like a potential RMM bug: could bit 32 be part of the physical
> > Address (if PA >= 4 GiB)?
> >
> > It seems s2tte_drain_pending() in lib/s2tt/src/s2tt.c checks bit 32 without
> > checking whether the descriptor is valid or invalid.
> >
> > In my testing, guarding the drain checks with a check for S2TTE_INVALID seemed
> > to resolve the boot hang:
> > --- a/lib/s2tt/src/s2tt.c
> > +++ b/lib/s2tt/src/s2tt.c
> > @@ -1701,6 +1701,10 @@ unsigned long
> > s2tte_clear_drain_pending(unsigned long s2tte)
> >
> >   bool s2tte_drain_pending(unsigned long s2tte)
> >   {
> > + if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
> > + return false;
>
> We should use also consider cases where the entry is INVALID, but
> has HIPAS=ASSIGNED/ASSIGNED_DEV to make it tighter. So, I think
> it is better to use :
>
> s2tte_is_unassigned() or in the library stick to :
>
> if (!s2tte_has_hipas(s2tte, S2TTE_INVALID_HIPAS_UNASSIGNED))
>      return false;
>
> May be we should assert this and make the caller responsible for
> checking the bit. I will leave it to the tf-RMM team to fix.
>
> But for now, please use the above fix.

Both worked for me. With the extra check in place, the realm guest can boot
up successfully.

FYI, The below additional checks in s2tte_tlbi_pending() and s2tte_drain_handle()
aren't needed because they're always guarded by s2tte_drain_pending() in all
calling sites.

Thanks,
Gavin


> Cheers
> Suzuki
>
>
> > + }
> > +
> >    return (s2tte & S2TTE_SW_DRAIN_PENDING_BIT) != 0UL;
> >   }
> >
> > @@ -1730,11 +1734,19 @@ unsigned long
> > s2tte_clear_tlbi_pending(unsigned long s2tte)
> >
> >   bool s2tte_tlbi_pending(unsigned long s2tte)
> >   {
> > + if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
> > + return false;
> > + }
> > +
> >    return (s2tte & S2TTE_SW_TLBI_PENDING_BIT) != 0UL;
> >   }
> >
> >   unsigned int s2tte_drain_handle(unsigned long s2tte)
> >   {
> > + if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
> > + return 0U;
> > + }
> > +
> >    return (unsigned int)EXTRACT(S2TTE_SW_HANDLE, s2tte);
> >   }
>
>
>
>
> > Sharing in case it helps.
> > Thanks,
> > Alper