x86 AMX/ACE with >8 tiles

[Christian Ludloff]

If x86 AMX/ACE with >8 tiles does not concern you,
then you can stop reading.

----------------------- 8< -----------------------

I have been asked to document x86 AMX/ACE behavior
of 16-tile and 32-tile implementations.

Background
----------

The goal of a 16-tile or 32-tile HW implementation
is to achieve extra performance by running SW that
takes advantage of the extra tiles. In particular,
said SW is expected to mostly run compute kernels,
minimize process switches, and avoid tile usage in
interrupt handlers for storage and networking. The
use of VMs is possible, but again, rapid switching
amongst guests is not a target scenario.

It is possible but wasteful to run legacy SW using
a degraded mode with just the classic 8 tiles. One
solution is to have a TILE_CTRL MSR, with a set of
32_TILE_DIS and 16_TILE_DIS control bits.

Approach
--------

A 16-tile implementation can be supported with the
existing VEX encodings (as they support TMM0...15)
and the existing TILECFG (as it has room for COLSB
and ROWS fields for TMM8...15).

A 32-tile implementation can be supported with the
existing VEX encodings augmented by EVEX encodings
(as those support TMM0...31) and a TILECFG that is
wider (to accommodate COLSB and ROWS fields needed
for TMM16...31). [The EVEX encodings get allocated
for all of AMX, including TF32 and TRANSPOSE.]

Note how CPUID already supports reporting >8 tiles
and the resulting overall size, and how it already
reports TILECFG width -- no new CPUID feature flag
is required or allocated.

It is possible to grow TMM width (COLSB) or height
(ROWS), but growing TMM count turned out to be the
least intrusive choice. It is also possible to add
another palette (3) or new XSAVE/XRSTOR state, but
again, avoiding all that was least intrusive.

Oversights
----------

An oversight of the AMX spec is the {LD,ST}TILECFG
memory operand width: spelled out as 64 bytes, not
the CPUID-reported width as it should have been --
as a result, existing SW with hard-coded width, or
with hard-coded reserved-bits-must-be-zero checks,
must run in degraded-to-8-tiles mode.

An oversight of the APX spec is that EVEX variants
of {LD,ST}TILECFG and TILE{LOAD,STORE}D (incl. the
T1, RS, and RST1 variants of TILELOAD) were "tied"
to APX (i.e. the ability to use EGPRs for a memory
operand's address) -- TILECFG memory operand width
or the possibility of >16 tiles but no APX did not
get addressed.

An oversight of the ACE spec is the placement of a
few instructions: their opcodes occupy the natural
locations of AMX-{INT8,FP16,BF16} EVEX variants --
as a result, the latter got shifted by one column;
this ultimately proved simpler than overlap-based-
on-palette, W=1, LL, MAP6, etc.

Results
-------

The new COLSB and ROWS fields in TILECFG have been
added here -- in the existing 64 bytes (TMM8...15)
and in the newly added 64 bytes (TMM16...31):

  https://www.sandpile.org/x86/tile.htm

The changes in {LD,ST}TILECFG memory operand width
have been applied here -- search for "TILECFG", to
find all 4 instances:

  https://www.sandpile.org/x86/opc_3.htm#:~:text=TILECFG

The new EVEX AMX encodings have been added here --
search for "TMM16" to find all of them:

  https://www.sandpile.org/x86/opc_2.htm#:~:text=TMM16
  https://www.sandpile.org/x86/opc_3.htm#:~:text=TMM16

An implementation with support for 16 and 32 tiles
has been running at large scale for some time.

Recommendations
---------------

Don't assume 8 tiles. Use CPUID instead.

Don't assume TILEDATA = 8 KiB. Use CPUID.

Don't assume TILECFG = 64-byte. Use CPUID.

Don't hard-code {LD,ST}TILECFG mem ops as 64-byte.

Prepare for EVEX-encoded AMX, needed for 32 tiles.

----------------------- 8< -----------------------

--
C.