Re: Candidate Linux ABI for Intel AMX and hypothetical new related features

[Andy Lutomirski]

On Fri, Apr 9, 2021 at 1:53 PM Len Brown <lenb@xxxxxxxxxx> wrote:
>
> On Wed, Mar 31, 2021 at 6:45 PM Andy Lutomirski <luto@xxxxxxxxxx> wrote:
> >
> > On Wed, Mar 31, 2021 at 3:28 PM Len Brown <lenb@xxxxxxxxxx> wrote:
> >
> > > We added compiler annotation for user-level interrupt handlers.
> > > I'm not aware of it failing, or otherwise being confused.
> >
> > I followed your link and found nothing. Can you elaborate?  In the
> > kernel, we have noinstr, and gcc gives approximately no help toward
> > catching problems.
>
> A search for the word "interrupt" on this page
> https://gcc.gnu.org/onlinedocs/gcc/x86-Function-Attributes.html#x86-Function-Attributes
> comes to the description of this attribute:
>
> __attribute__ ((interrupt))
>

I read that and I see no mention of anything saying "this will
generate code that does not touch extended state".  Instead I see,
paraphrasing, "this will generate code with an ABI that is completely
inappropriate for use in a user space signal handler".  Am I missing
something?

> > > dynamic XCR0 breaks the installed base, I thought we had established that.
> >
> > I don't think this is at all established.  If some code thinks it
> > knows the uncompacted XSTATE size and XCR0 changes, it crashes.  This
> > is not necessarily a showstopper.
>
> My working assumption is that crashing applications actually *is* a showstopper.
> Please clarify.

I think you're presuming that some program actually does this.  If no
program does this, it's not an ABI break.

More relevantly, this can only happen in a process that uses XSAVE and
thinks it knows the size that *also* does the prctl to change XCR0.
By construction, existing programs can't break unless they load new
dynamic libraries that break them.

>
> > > We've also established that when running in a VMM, every update to
> > > XCR0 causes a VMEXIT.
> >
> > This is true, it sucks, and Intel could fix it going forward.
>
> What hardware fix do you suggest?
> If a guest is permitted to set XCR0 bits without notifying the VMM,
> what happens when it sets bits that the VMM doesn't know about?

The VM could have a mask of allowed XCR0 bits that don't exist.

TDX solved this problem *somehow* -- XSETBV doesn't (visibly?) exit on
TDX.  Surely plain VMX could fix it too.

>
> > > I thought the goal was to allow new programs to have fast signal handlers.
> > > By default, those fast signal handlers would have a stable state
> > > image, and would
> > > not inherit large architectural state on their stacks, and could thus
> > > have minimal overhead on all hardware.
> >
> > That is *a* goal, but not necessarily the only goal.
>
> I fully support coming up with a scheme for fast future-proof signal handlers,
> and I'm willing to back that up by putting work into it.
>
> I don't see any other goals articulated in this thread.

Before we get too carried away with *fast* signal handlers, something
that works with existing programs is also a pretty strong goal.  RIght
now AVX-512 breaks existing programs, even if they don't use AVX-512.