Re: [PATCH v2] firmware: arm_ffa: support running as a guest in a vm

[Jens Wiklander]

On Wed, Apr 3, 2024 at 3:03 PM Sudeep Holla <sudeep.holla@xxxxxxx> wrote:
>
> On Wed, Mar 27, 2024 at 10:23:35AM +0100, Jens Wiklander wrote:
> > On Tue, Mar 26, 2024 at 4:35 PM Sudeep Holla <sudeep.holla@xxxxxxx> wrote:
> > >
> > > On Mon, Mar 25, 2024 at 09:13:35AM +0100, Jens Wiklander wrote:
> > > > Add support for running the driver in a guest to a hypervisor. The main
> > > > difference is introducing notification pending interrupt and that
> > > > FFA_NOTIFICATION_BITMAP_CREATE doesn't need to be called.
> > > >
> > > > The guest may need to use a notification pending interrupt instead of or
> > > > in addition to the schedule receiver interrupt.
> > >
> > > The above statement makes me worry a bit that we are still not on the same
> > > page about NPI vs SRI. NPI need not exist in addition to SRI. And in v1
> > > you did mention you have SRI in the guest as well. Then why do we need
> > > NPI in addition to that. As part of SRI, the callback  ffa_self_notif_handle
> > > gets registered and will be called as part of SRI handling. What you
> > > do in  notif_pend_irq_handler(), exactly what ffa_self_notif_handle()
> > > already does.
> >
> > That's my understanding of what an NPI handler should do to be able to
> > receive per-vCPU notifications.
> >
> > >
> > > I am still struggling to understand the usecase here. If you just have
> > > NPI and no SRI when running the driver in the VM, then it aligns with
> > > my understanding of possible use-case(not the one you mentioned in v1:
> > > where FF-A driver in VM will have SRI as OPTEE is the secondary scheduler)
> >
> > OP-TEE is not a secondary scheduler. OP-TEE (the SP) is scheduled as
> > usual by the normal world using direct request. OP-TEE doesn't receive
> > FF-A notifications and I'm not sure it will ever be needed.
> >
>
> Sorry for my poor choice of words yet again. I meant VM kernel(running
> as NS virtual endpoint) with OPTEE driver running in it as secondary
> scheduler. IIUC, there will be another instance of OPTEE driver in the
> primary scheduler endpoint(i.e. host kernel) and it will take care of
> running SRI handler ?

Yes, that's what we have in the Xen configuration, except that we
don't use an OP-TEE driver, it's only generic FF-A processing.
The SRI in this case is a physical interrupt, raised by the SPMC.

>
> > >
> > > If we are supporting NPI or SRI, I think we can see if we can further
> > > simplify this change, but I want to get to an agreement with usage model
> > > before we dig into implementation details in this patch.
> >
> > The spec doesn't as far as I know explicitly make NPI and SRI mutually
> > exclusive, it doesn't make sense to use both in all configurations.
> > I'm trying to be as dynamic as possible when configuring the NPI and
> > SRI handlers.
> >
>
> Fair enough
>
> > If the kernel is a physical endpoint, it's easy, it only uses SRI and
> > the SPMC will not give an NPI when asked.
> >
>
> Agreed.
>
> > If the kernel is a virtual endpoint it might be more complicated since
> > a VM may need to act as a secondary scheduler. That's not fully
> > supported in this patch, since it can only schedule itself. SRI is not
> > used in my current configuration. If a hypervisor injects an SRI I
> > expect it to filter what's returned by FFA_NOTIFICATION_INFO_GET for
> > this VM so it doesn't interfere with notifications for other VMs.
> >
>
> OK
>
> > In my current configuration, the hypervisor uses NPI to signal pending
> > notifications to the guest. I do not need a secondary scheduler since
> > OP-TEE doesn't receive notifications. At a later time, we may have SPs
> > that need to be scheduled, but that's not a problem I'm trying to
> > solve here.
>
> Understood. I will take a look at the patch with the above information.

Thanks,
Jens

>
> --
> Regards,
> Sudeep