Re: [PATCH 00/18] Another attempt at HVO support on arm64

[James Houghton]

On Thu, Jul 9, 2026 at 9:59 PM Muchun Song <muchun.song@xxxxxxxxx> wrote:
> > On Jul 8, 2026, at 11:11, James Houghton <jthoughton@xxxxxxxxxx> wrote:
> >
> > Hi everyone,
> >
> > This patch series uses a trick with the Access Flag on CPUs that support
> > hardware update of the AF to update vmemmap page table entries without
> > introducing a time window where CPUs accessing the vmemmap might fault.
>
> Hi James,
>
> I was just looking into the Arm ARM specification regarding the TLB caching
> behavior to better understand the "AF trick" used here, and came across a few
> sections that got me a bit confused.
>
> Specifically, Section D8.5.1 [1] states Rule RDWZCQ:
>
>         "Descriptors with AF set to zero can never be cached in a TLB. For more
>          information about when translation table entries are permitted to be
>          cached in a TLB, see Translation Lookaside Buffers."
>
> This indeed seems to support the core assumption of the AF trick. However, when
> following the reference to Section D8.16 "Translation Lookaside Buffers" [2], it
> defines the following rule for permitted caching (RSQBCS):
>
>         "When address translation is enabled, a translation table entry for an
>          in-context translation regime that does not cause a Translation fault,
>          an Address size fault, or an Access flag fault is permitted to be cached
>          in a TLB or intermediate TLB caching structure as the result of an
>          explicit or speculative access."
>
> This is exactly where my confusion lies, as these two descriptions in the spec
> feel directly contradictory when FEAT_HAF(hardware management of the Access flag)
> is enabled.

I don't think it intends to say that "translation table entries with
AF=0 may be cached" just because they don't cause Access flag faults.

It is likely that this clause was written before FEAT_HAFDBS/FEAT_HAF
existed. Re-reading it with that in mind makes the meaning clearer:
PTEs with AF=1 may be cached if the architecture otherwise allows it.

There is another clause, in D8.5 "Hardware updates of the translation
tables", I_RGQLZ: "If hardware does not update the AF from 0 to 1,
then the descriptor is not permitted to be cached in a TLB." I think
this supports my interpretation.

I am convinced that it is impossible for an implementation not to
conform to Rule R_DWZCQ (PTEs with AF=0 cannot be cached) while
supporting HW AF correctly. Let me try to explain why.

For HW AF to function correctly:
 1. Setting AF=1 must be done atomically, otherwise hardware could set
a bit in an invalid PTE. (This would break Linux swp_entrys.)
 2. The AF=0 translation cannot be used, otherwise we could
potentially drop AF updates.

If an implementation used the translation before coherently setting
AF=1 to perform an architectural memory access, then it must choose
either to set AF=1 on a PTE that may have since changed (perhaps to an
invalid PTE) or drop the AF update entirely. Either way HW AF is
broken.

Please also see I_GXGPN: "When the translation of an architecturally
executed memory access occurs, the architecture requires that AF is
set to 1."

> On one hand, Section D8.5.1 [1] explicitly uses the word "never" for AF=0
> descriptors. On the other hand, under FEAT_HAF, an entry with AF=0 does not cause
> an Access flag fault anymore, because the hardware is capable of updating the AF
> automatically. This technically makes it eligible for speculative caching under
> Rule RSQBCS in Section D8.16 [2].
>
> My question is: which part of the spec reflects the actual architectural intent
> when FEAT_HAF is enabled? Is an entry with AF=0 permitted to be speculatively
> cached in the TLB by the hardware under FEAT_HAF, or does the "never" in RDWZCQ
> still hold absolute priority?
>
> Please let me know if I have misread the specification or missed some overriding
> constraints here.

Thanks for making me look a little harder into this; it's a good
question. I hope my reply above makes you more comfortable with the AF
trick. I also hope to hear more from the Arm folks as well.

Thanks Muchun!