Re: [PATCH 0/6] mm: fix pfn walker races with memory offline

[Gregory Price (Meta)]

On 2026-10-27 15:25 +0100, David Hildenbrand (Arm) wrote:
> On 10/27/26 14:49, Gregory Price (Meta) wrote:
> > 
> > while these are ai-generated, I'm not posting untested, unreviewed
> > schlopp for the sake of getting patches upstream.
> 
> Appreciated.
> 
> For this long-standing considered low-priority thing in particular it is
> reasonable to go back in time, see how often it was raised (quite often by me)
> and how often it was actually reported (I found one case).
> 
> Then discuss the high level design in more than a single sentence :)
> 
> Discuss why we are not using it for other pfn_to_online_page() users, how it
> interacts with zone->contig etc etc.
> 
> There really isn't a lot on the design in the cover letter.
>

Fair, there's a mix of design vs firefighting tension going on here.

I'm trying to get out ahead of a rapidly evolving scaling issue before
it becomes big.  But I will admit it's sometimes hard to even figure
out what questions to ask without posting the wrong patches first.

(Cunningham's law :P)

> > Some of these don't show up as crashes but instead as hung offline
> > attempts (GFP_KERNEL landing on movable).  That shows up as our
> > hypervisor just failing to overcommit / guests hanging.
> > 
> > The compaction/alloc_contig bug is one I've seen in production.
> 
> Right, but I suspect that's "[PATCH 4/6] mm/page_alloc: don't undo another
> isolation in alloc_contig_range" which is *not at all* related to "mm: fix pfn
> walker races with memory offline".
>

Ehhh... i'd argue it's the same shape.  But i'll concede some of these
could be pulled out ahead as you've said elsewhere.

> > skippable side note on virtio-balloon/mem:
> > 
> > I have some improvements to virtio-balloon and virtio-mem that allow for
> > handoff from balloon directly to mem for hot-unplug without having to
> > drop the balloon, but I haven't posted those (and we don't run those in
> > production, so this is not the nexus of crashes).
> 
> Do you mean free-page reporting or actual balloon inflation/deflation paired
> with virtio-mem. I hope not the latter, because that is just disgusting.
> 

Actual balloon inflation/deflation.  It is disgusting, but it's
effective.  Right now we use hotplug at boot to minimize the size of
cold machines to avoid thundering herd issues, and then add it when
they actually get allocated for work.

At runtime we use ballooning to reduce a VM to its actual RSS based
on PSI feedback (see: that virtio stats extension i sent you some while
ago), but would like to use hotplug to give the host a defense against
a storm of deflate_on_oom scenarios across a large number of guests.

Basically using PSI stats we can make guests do somewhat aggressive
proactive reclaim on without having to build any new tooling, and
while being able to back-off aggressively.

Free-page reporting is also on by default, but this only helps you if
your guests are being extra nice about things.

(spoiler: they're not! they're greedly little imps! :D)

My experiments made balloon prefer to try to reclaim movable regions
in contiguous blocks, and then just hand that contiguous block off to
virtio-mem for offline without having to drop the balloon.

tl;dr: This seems effective and works quite well.

I've become a little confused as to why virtio balloon and mem are
separate entities when they could work together this way.

Haven't had the time to read up on the development history.

> > 
> > Agreed.  Although I fear the structural fix for this will have uglier
> > performance issue than we think.  Then again, maybe it doesn't matter
> > (if you're doing a PFN walk, then you're likely already doing something
> > stupidly expensive).
> 
> Which areas are you mostly concerned about and from which piece do you expect
> the overhead?
> 

largely the zonelist iteration - which looking back through history 
has already been "optimized" multiple times, only for those things
to introduce more bugs :\.

The zonelist races seem more ammenable to RCU solutions than the pfn
walks.  I'm not sure how to safely quiesce every possible PFN walker
without atomics in their fast path.

> For "broader" things that are not just a simple pfn_to_online_page() user, we
> could also have memory online/offline sync in more efficient ways using the
> memory notifier.
> 
> It's also tricky, but maybe for zone->contig + compaction the better solution?
> 

This seems reasonable, I was trying not to schlopp you with a half-baked
architectural fix and instead use the available tools.

But maybe we should chat (after the window).

I'll try to be on IRC more often.

> > On performance:
> > 
> > I've been hammering this with will-it-scale and a bunch of workload
> > tests on a variety of hosts and have not seen any obvious regressions
> > outside of powerpc (which i can't test directly, but SRCU-fast requires
> > atomics for powerpc from what i can tell - so it will eat overhead
> > without a fallback... if there is actually any hotplug ppc hardware out
> > there to care about?)
> 
> So the big problem is the overhead on the read side for srcu?
>

Yes - but specifically this affects PowerPC because it:
 - has NMI
 - does NOT have NMI-safe per-CPU operations
 - supports memory hotplug
 - uses atomics for SRCU-fast

So using SRCU fast in these contexts is not cheap.

For what I could measure in QEMU, the atomics on PPC seem like they
could be significant (15-20% overhead) in microbenchmarks.

Hard to say what that turns into on real hardware.

I implemented a fallback solution for those architectures to just not
free the zonelists in this case.  They have to iterate the dead zones,
but it's a much less overhead comparatively (<1%, but measurable).

For everyone else SRCU-fast is imperceptible overhead.

> If we can find something else to suspend compaction, we could maybe switch to
> rcu for the remaining cases.
> 

Is long-term compaction suspension even safe? I had to make COMPACTION a
requirement for N_MEMORY_USER_NUMA in order to not generate spurious
failures due to fragmentation.

~Gregory

-- 
~Gregory