Re: [PATCH 00/18] Another attempt at HVO support on arm64

[James Houghton]

On Tue, Jul 14, 2026 at 1:46 AM Pedro Falcato <pfalcato@xxxxxxx> wrote:
>
> Hello,
>
> On Wed, Jul 08, 2026 at 03:11:10AM +0000, James Houghton wrote:
> > -- The AF trick --
> >
> > The trick is that translations with the AF unset cannot be cached in the
> > TLB (see Rule R_DWZCQ in the Arm ARM), so they can be atomically updated
> > without needing a full break-before-make sequence.
> >
> > So the PTE update sequence becomes:
> >   1. Atomically clear the AF on the existing PTE.
> >   2. Invalidate the TLB.
> >   3. cmpxchg the AF=0 PTE with the new PTE. If this fails, goto 1.
> >
>
> So, this sounds really cool but I'm not sure if its correct. Looking at
> D8.17.1 Using break-before-make when updating translation table entries:
> Per IGXGZY:
> For a translation stage with AF hardware management enabled, if a translation
> table entry is modified and the break-before-make sequence is not followed,
> then all of the following failures associated with AF hardware updates can
> occur:
>
> - When a memory location associated with that translation table entry is
> accessed, the AF is not set.

I think this clause is referring to the case where a CPU updates a PTE
using something like STR (as opposed to SWP or CAS). If CPU 0 STRs a
new PTE while CPU 1 accesses memory through the old AF=0 PTE, the AF=1
update will be overwritten by the STR of the new PTE.

This clause might be suggesting that implementations are free to
implement something like: "if the cacheline for the PTE cannot be
grabbed exclusively, drop the AF update but proceed with the access".
To me, this interpretation is incompatible with the rules in D8.5.1,
summarized in I_GXGPN: "When the translation of an architecturally
executed memory access occurs, the architecture requires that AF is
set to 1."

But IMO dropping the AF update isn't actually important. It's okay for
the kernel to use the vmemmap while we are updating it, we just need
the translation not to be cached. AF=0 does that for us (I claim).

> - When hardware updates to that translation table entry are followed by
> stores appearing later in program order, the ordering required is not followed.

I assume the "ordering required" is that the AF=1 update is
ordered-before the later stores. I interpret this as a natural
consequence of the earlier statement. If the AF=0 update is "dropped"
(overwritten by a STR), then seeing the later stores (using the old
translation) does not necessarily mean the new translation will have
AF=1. Maybe I'm interpreting this wrong (I am up late tonight...).

> So apparently per the ARM ARM you can possibly never observe AF=0 even if it's
> cached.
>
> And then other semantics concerns may arise e.g I'm not sure if AF=0 implies
> that the walk cache has nothing for that address, or if it's just that
> particular translation that's not in the TLB. Though this might not
> particularly matter since we're not collapsing the PTEs back to a PMD block
> mapping.

I assume that when the Arm ARM says "cannot be cached", it means "no
TLBI is needed (to avoid TLB conflict aborts)". In other words,
partial walk caches need not be invalidated.

D8.5.1, I_HQBRF: "When software sets the AF to 1, there is no
requirement to perform TLB invalidation after setting the flag because
entries
with an AF set to 0 are never held in a TLB."

If an implementation caches a partial walk that includes a AF=0
translation such that a newer translation can conflict with the cached
partial walk, then a TLBI would be needed and the implementation
wouldn't be compliant.

> Right? Tell me I'm wrong :)

Thanks for looking at this Pedro. It is really late for me (I had a
long day, but I wanted to get back to you), so sorry if what I've
written is a little incoherent.