Re: [PATCH 00/18] Another attempt at HVO support on arm64

[Pedro Falcato]

Hello,

On Wed, Jul 08, 2026 at 03:11:10AM +0000, James Houghton wrote:
> Hi everyone,
> 
> This patch series uses a trick with the Access Flag on CPUs that support
> hardware update of the AF to update vmemmap page table entries without
> introducing a time window where CPUs accessing the vmemmap might fault.
> 
> By avoiding faults, the HugeTLB vmemmap optimization (HVO) can be
> implemented correctly on arm64 in a much more straightforward way than
> previously attempted, most recently here[1] (please see [1] for a
> breakdown of the other approaches attempted before).
> 
> For large-memory systems that allocate most of their available memory to
> HugeTLB, HVO saves a huge amount of memory (1.5% of system memory).
> 
> This series has four parts:
>   1. Some preparatory changes (patches 1-3)
>   1. Bare minimum HVO support (patches 4-10)
>   2. Drop BBML2_NOABORT requirement for HVO (patches 11-13)
>   3. Drop the user-configurable Kconfig for HVO (patches 14-18)
> 
> Parts 3 and 4 are technically optional. More details below.
> 
> The main functional caveat with this series is that bootmem HugeTLB
> pages are not "pre-HVOed". They will be HVOed, but because at pre-HVO
> time SMP CPUs have not been enabled, we cannot query for full system
> support.
> 
> This series is based on 7.2-rc2 (0e35b9b6ec0f).
> 
> This series almost 100% cleanly applies to mm-new, which has some of
> Muchun's HVO patches, with one trivial conflict. I imagine this series
> will conflict pretty heavily with some of Muchun's other patches[2].
> 
> -- The AF trick --
> 
> The trick is that translations with the AF unset cannot be cached in the
> TLB (see Rule R_DWZCQ in the Arm ARM), so they can be atomically updated
> without needing a full break-before-make sequence.
> 
> So the PTE update sequence becomes:
>   1. Atomically clear the AF on the existing PTE.
>   2. Invalidate the TLB.
>   3. cmpxchg the AF=0 PTE with the new PTE. If this fails, goto 1.
> 

So, this sounds really cool but I'm not sure if its correct. Looking at
D8.17.1 Using break-before-make when updating translation table entries:
Per IGXGZY:
For a translation stage with AF hardware management enabled, if a translation
table entry is modified and the break-before-make sequence is not followed,
then all of the following failures associated with AF hardware updates can
occur:

- When a memory location associated with that translation table entry is
accessed, the AF is not set.
- When hardware updates to that translation table entry are followed by
stores appearing later in program order, the ordering required is not followed.

So apparently per the ARM ARM you can possibly never observe AF=0 even if it's
cached.

And then other semantics concerns may arise e.g I'm not sure if AF=0 implies
that the walk cache has nothing for that address, or if it's just that
particular translation that's not in the TLB. Though this might not
particularly matter since we're not collapsing the PTEs back to a PMD block
mapping.

Right? Tell me I'm wrong :)

-- 
Pedro