Re: [PATCH 00/18] Another attempt at HVO support on arm64

[Muchun Song]

> On Jul 13, 2026, at 12:59, James Houghton <jthoughton@xxxxxxxxxx> wrote:
> 
> On Thu, Jul 9, 2026 at 9:59 PM Muchun Song <muchun.song@xxxxxxxxx> wrote:
>>> On Jul 8, 2026, at 11:11, James Houghton <jthoughton@xxxxxxxxxx> wrote:
>>> 
>>> Hi everyone,
>>> 
>>> This patch series uses a trick with the Access Flag on CPUs that support
>>> hardware update of the AF to update vmemmap page table entries without
>>> introducing a time window where CPUs accessing the vmemmap might fault.
>> 
>> Hi James,
>> 
>> I was just looking into the Arm ARM specification regarding the TLB caching
>> behavior to better understand the "AF trick" used here, and came across a few
>> sections that got me a bit confused.
>> 
>> Specifically, Section D8.5.1 [1] states Rule RDWZCQ:
>> 
>>        "Descriptors with AF set to zero can never be cached in a TLB. For more
>>         information about when translation table entries are permitted to be
>>         cached in a TLB, see Translation Lookaside Buffers."
>> 
>> This indeed seems to support the core assumption of the AF trick. However, when
>> following the reference to Section D8.16 "Translation Lookaside Buffers" [2], it
>> defines the following rule for permitted caching (RSQBCS):
>> 
>>        "When address translation is enabled, a translation table entry for an
>>         in-context translation regime that does not cause a Translation fault,
>>         an Address size fault, or an Access flag fault is permitted to be cached
>>         in a TLB or intermediate TLB caching structure as the result of an
>>         explicit or speculative access."
>> 
>> This is exactly where my confusion lies, as these two descriptions in the spec
>> feel directly contradictory when FEAT_HAF(hardware management of the Access flag)
>> is enabled.
> 
> I don't think it intends to say that "translation table entries with
> AF=0 may be cached" just because they don't cause Access flag faults.
> 
> It is likely that this clause was written before FEAT_HAFDBS/FEAT_HAF
> existed. Re-reading it with that in mind makes the meaning clearer:
> PTEs with AF=1 may be cached if the architecture otherwise allows it.
> 
> There is another clause, in D8.5 "Hardware updates of the translation
> tables", I_RGQLZ: "If hardware does not update the AF from 0 to 1,
> then the descriptor is not permitted to be cached in a TLB." I think
> this supports my interpretation.
> 
> I am convinced that it is impossible for an implementation not to
> conform to Rule R_DWZCQ (PTEs with AF=0 cannot be cached) while
> supporting HW AF correctly. Let me try to explain why.
> 
> For HW AF to function correctly:
> 1. Setting AF=1 must be done atomically, otherwise hardware could set
> a bit in an invalid PTE. (This would break Linux swp_entrys.)
> 2. The AF=0 translation cannot be used, otherwise we could
> potentially drop AF updates.
> 
> If an implementation used the translation before coherently setting
> AF=1 to perform an architectural memory access, then it must choose
> either to set AF=1 on a PTE that may have since changed (perhaps to an
> invalid PTE) or drop the AF update entirely. Either way HW AF is
> broken.
> 
> Please also see I_GXGPN: "When the translation of an architecturally
> executed memory access occurs, the architecture requires that AF is
> set to 1."

Thanks for your excellent explanation.

> 
>> On one hand, Section D8.5.1 [1] explicitly uses the word "never" for AF=0
>> descriptors. On the other hand, under FEAT_HAF, an entry with AF=0 does not cause
>> an Access flag fault anymore, because the hardware is capable of updating the AF
>> automatically. This technically makes it eligible for speculative caching under
>> Rule RSQBCS in Section D8.16 [2].
>> 
>> My question is: which part of the spec reflects the actual architectural intent
>> when FEAT_HAF is enabled? Is an entry with AF=0 permitted to be speculatively
>> cached in the TLB by the hardware under FEAT_HAF, or does the "never" in RDWZCQ
>> still hold absolute priority?
>> 
>> Please let me know if I have misread the specification or missed some overriding
>> constraints here.
> 
> Thanks for making me look a little harder into this; it's a good
> question. I hope my reply above makes you more comfortable with the AF

At least to me, what you're saying makes sense.

> trick. I also hope to hear more from the Arm folks as well.

Yes, it would be best to have an ARM expert here to confirm that our
understanding of the manual is completely accurate.

Thanks.

> 
> Thanks Muchun!