Re: [PATCH] x86/tsx: fix KVM guest live migration for tsx=on

[Pawan Gupta]

On Tue, Apr 12, 2022 at 01:36:20PM +0000, Jon Kohler wrote:
> > On Apr 11, 2022, at 7:45 PM, Dave Hansen <dave.hansen@xxxxxxxxx> wrote:
> >
> > On 4/11/22 12:35, Jon Kohler wrote:
> > > Also, while I’ve got you, I’d also like to send out a patch to simply
> > > force abort all transactions even when tsx=on, and just be done with
> > > TSX. Now that we’ve had the patch that introduced this functionality
> > > I’m patching for roughly a year, combined with the microcode going
> > > out, it seems like TSX’s numbered days have come to an end.
> >
> > Could you elaborate a little more here?  Why would we ever want to force
> > abort transactions that don't need to be aborted for some reason?
>
> Sure, I'm talking specifically about when users of tsx=on (or
> CONFIG_X86_INTEL_TSX_MODE_ON) on X86_BUG_TAA CPU SKUs. In this situation,
> TSX features are enabled, as are TAA mitigations. Using our own use case
> as an example, we only do this because of legacy live migration reasons.
>
> This is fine on Skylake (because we're signed up for MDS mitigation anyhow)
> and fine on Ice Lake because TAA_NO=1; however this is wicked painful on
> Cascade Lake, because MDS_NO=1 and TAA_NO=0, so we're still signed up for
> TAA mitigation by default. On CLX, this hits us on host syscalls as well as
> vmexits with the mds clear on every one :(
>
> So tsx=on is this oddball for us, because if we switch to auto, we'll break
> live migration for some of our customers (but TAA overhead is gone), but
> if we leave tsx=on, we keep the feature enabled (but no one likely uses it)
> and still have to pay the TAA tax even if a customer doesn't use it.
>
> So my theory here is to extend the logical effort of the microcode driven
> automatic disablement as well as the tsx=auto automatic disablement and
> have tsx=on force abort all transactions on X86_BUG_TAA SKUs, but leave
> the CPU features enumerated to maintain live migration.

This won't help on CLX as server parts did not get the microcode driven
automatic disablement. On CLX CPUID.RTM_ALWAYS_ABORT will not be set.

What could work on CLX is TSX_CTRL_RTM_DISABLE=1 and
TSX_CTRL_CPUID_CLEAR=0. This can be done for tsx=auto or with a new mode
tsx=fake|compat. IMO, adding a new mode would be better, otherwise
tsx=auto behavior will differ depending on the kernel version.

Provided that software using TSX is following below guidance [*]:

  When Intel TSX is disabled at runtime using TSX_CTRL, but the CPUID
  enumeration of Intel TSX is not cleared, existing software using RTM may
  see aborts for every transaction. The abort will always return a 0
  status code in EAX after XBEGIN. When the software does a number of
  transaction retries, it should never retry for a 0 status value, but go
  to the nontransactional fall back path immediately.

Thanks,
Pawan

[*] TAA document: section -> Implications on Intel TSX software
    https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/intel-tsx-asynchronous-abort.html