Re: Candidate Linux ABI for Intel AMX and hypothetical new related features

[Len Brown]

On Tue, Apr 13, 2021 at 4:16 PM Andy Lutomirski <luto@xxxxxxxxxx> wrote:
>
> On Mon, Apr 12, 2021 at 4:46 PM Len Brown <lenb@xxxxxxxxxx> wrote:
> >
> > On Mon, Apr 12, 2021 at 11:21 AM Andy Lutomirski <luto@xxxxxxxxxx> wrote:
> >
> > > AMX: Multiplying a 4x4 matrix probably looks *great* in a
> > > microbenchmark.  Do it once and you permanently allocate 8kB (is that
> > > even a constant?  can it grow in newer parts?), potentially hurts all
> > > future context switches, and does who-knows-what to Turbo licenses and
> > > such.
> >
> > Intel expects that AMX will be extremely valuable to key workloads.
> > It is true that you may never run that kind of workload on the machine
> > in front of you,
> > and so you have every right to be doubtful about the value of AMX.
>
> I fully believe that AMX will be amazing when used for the right
> workload.  The problem is that a library may have no way to tell
> whether a workload is the type of computationally intensive workload
> for which it makes sense.  Imagine you have a little function:
>
> int matrix_times_vector(int dim, float *out, const float *matrix,
> const float *vector);
>
> A clever library might use AMX for this.  If dim == 4 and the caller
> is planning to call it in a long, tight loop, maybe this even makes
> sense.  If dim == 4 and it's being called once, AMX is probably a
> losing proposition.  With previous technologies, at least the impact
> was limited to the function itself and maybe once per call to the
> caller.  But now, with AMX, the program that invoked this takes a
> performance and memory hit *forever* if it uses AMX once.

Again...

As this is a "clever" library, built with a clever toolchain, and the
result is that
TILERELEASE was properly issued at the end of computation.
Thus the hardware knows that the  (volatile) AMX registers are no longer live.

The XSAVE hardware recognizes this INIT=1 condition and
transfers NO DATA "*forever*".  This is true both on context switch (compacted)
where it is automatic, and on (uncompacted) signal delivery, where we
check for this case.

Was that the "performance hit" of concern, or did I miss something?

Yes, it is true that the kernel allocated a context switch buffer for
the lifetime
of that task, and it will not be freed until that task exits.

If this proves to be an issue, there is nothing
preventing us from implementing a re-claim scheme for a rarely used buffer.
After recognizing this situation, we'd simply arm XFD, free the
buffer, and from then onwards, the task behaves exactly as if had
never touched AMX.

However, nobody has yet suggested that would be a common situation
worth an optimization to reclaim that task's 8KB.

> Beyond that, we have the signal handling issue.

I'm unaware of any unresolved feedback on the signal handling series
other than a wistful "wouldn't a new SIGFAIL be more clear (for future apps)
than the existing SIGSEGV?"  I agree with this sentiment, but I don't
think we should hold up a patch to prevent corrupting user data
because a new signal number to describe the scenario doesn't exit.
Particularly since the new code that knows about the new SIGFAIL
will also be new code that has been compiled with the new glibc
that for most cases will prevent this scenario in the first place...

> One solution, going
> off of what WIlly mentioned, is:
>
> bool amx_begin(void *signal_save_buffer);
> void amx_end();
>
> In the amx_begin() region, if you get a signal, the AMX state is saved
> in the buffer.  Outside the region, if you get a signal and AMX is in
> use, the kernel will either unceremoniously kill the task or will
> deliver SIGYOUBLEWIT. [0]

I think it is clear that if a new signal ABI is going to be invented,
that it should be opt-in on state, so that it can run fast on machines
far into the future by not choosing to opt-in on anything.

It isn't clear that changing the signal save state around critical regions
(in multiple threads) so that a single (per process definition) of a signal
handler gets a different result at different times is going to make that
(new) signal handler author especially happy.  More likely they
either always want the state, or they do not.

> I'm really hoping some userspace people can chime in.

Len Brown, Intel Open Source Technology Center