Re: Path forward for Virtualized Swap?

[Baoquan He]

On 09/15/26 at 01:48pm, Baoquan He wrote:
> On 09/11/26 at 09:45am, Shakeel Butt wrote:
> > On Fri, Sep 11, 2026 at 09:06:29PM +0800, Baoquan He wrote:
> > > On 09/10/26 at 09:39am, Shakeel Butt wrote:
> > > > On Thu, Sep 10, 2026 at 03:09:59PM +0800, Baoquan He wrote:
> > > > > Hi Nhat,
> > > > > 
> > > > > On 09/04/26 at 02:14pm, Nhat Pham wrote:
> > > > > .....snip...
> > > > 
> > > > [...]
> > > > 
> > > > > With VM_SPARSE, xswap's cluster access is exactly the plain-array line the
> > > > > rest of swap already uses:
> > > > > 
> > > > >     return &si->cluster_info[offset / SWAPFILE_CLUSTER];
> > > > > 
> > > > > no branch, no RCU discipline, no tear-down state machine, and no NULL
> > > > > return. So VM_SPARSE doesn't add complexity to close a gap; it lets the
> > > > > cluster layer stay as simple as it already is, which is precisely the
> > > > > part later work (writeback, rmap lookup, memcg charging, THP) has to sit
> > > > > on.
> > > > > 
> > > > > I'm not going to claim xswap wins on throughput. I measured it:
> > > > > on a 64G/64-thread swapout, xswap, vswap and plain swap+zswap are all
> > > > > within ~2-3% of each other, effectively identical. 
> > > > 
> > > > So the claim is VM_SPARSE is simpler than xarray based approach. I feel like
> > > > we are discussing implementation details before deciding the design and
> > > > architecture. So, instead of VM_SPARSE vs xarray, let's discuss and decide the
> > > > need for dynamic growth. Why we want dynamic growth upfront or can it be added
> > > > later? Once we decide that then it will be very easy to pick an implementation
> > > > that would take us there.
> > > 
> > > Hi Shakeel,
> > > 
> > > Thank you for joining the discussion and for taking the time to comment.
> > 
> > Hi Baoquan,
> > 
> > I am mainly trying to facilitate the discussion but your use of LLM is causing
> > more confusion. LLM use is fine but please at least re-read before sending that
> > the sentences flow and makes sense.
> 
> Sorry, my bad. I used LLM to find Nhat's words. But I did check it by
> myself. I wrote most of them by myself. While at it ath the moment, my
> logic could be unclear.
> 
> As said, how swap_cluster_info[] is built is the foundation. Whatever
> you do, you have to make swap_cluster_info[] ready, then you can do
> writeback, rmap lookup, thp support, etc, on top of it.
> swap_cluster_info[] is the basement, then you continue building 2nd
> floor, 3rd floor, till a high building is done with things added. Nobody
> wants to claim he just need the high building, while no basement.
> 
> Now, the foundation has been built with the lazy vmalloc, it can grow on
> demand. It keeps swap_cluster_info accessing as swap_cluster_info[],
> a basic array semantics. And since we our target is to support a very
> large swap device with an extendable logical space, grow on demand and
> shrink becomes important. Now it is there.

By the way, with my understanding, only grow is enough for xswap. You
can reserve a huge space for it, while in fact you could only really use
it within a small space. Then it's fine, grow the swap_cluster_info[] to
the place it ever used, and it mostly will be used again. E.g on a small
system with 10G RAM, we set si->max as 2x10=20G. In fact it could only
reach 2G swap space. That's fine. 2G is the place we need, keep it. The
left 18G is untouched and surely no swap_cluster_info[] built for it.
Anyway, Nhat strongly suggested shrink is necessary. I am wondering if
there's really use case.

> 
> That's my understanding, not sure if there's anything I can't get so
> that writeback need be made first.
> 
> [I type each of above by hand.]
> > 
> > > 
> > > Agreed on requirement first - but this one was already decided, and not by me. In
> > > the July ghost swapfile thread Nhat rejected exactly the shape of "grow only, can
> > > be added later":
> > > 
> > >     "Except for my virtual swap design, which does support dynamic growth AND
> > >      shrinking of capacity on demand ;) If it cannot grow (and furthermore, if it
> > >      requires userspace operation to trigger swapfile growth), why do we need this
> > >      at all? Might as well create a new swapfile with swapon?"
> > > 
> > > To me what it converged on was "dynamic growth and shrink, no writeback yet".
> > 
> > I am not getting how out of context above paragraph shows the conclusion about
> > dynamic growth/shrink and *no writeback*.
> > 
> > > So automatic growth *and* shrink is the requirement, and the simpler alternative
> > > was already on the table.
> > > 
> > > And I keep mentioning it in the cover-letter of each version of my posting. I
> > > only did the foundtation via lazy vmalloc. And Nhat will do the core
> > > part including writabck, rmap lookup, memcg accounting, zero page fill,
> > > etc.
> > 
> > I don't see any evidense of this decision. Actually this whole email thread
> > shows that there is no such decision.
> > 
> > > 
> > > What is still genuinely open, and I would like us to settle, is how large the
> > > device's address space should be, because the metadata scales with it:
> > 
> > With dynamic growth/shrink, is this really a blocker?
> > 
> > Anyways, I will let Nhat and others discuss the technical details (unless I am
> > asked for it). My main reason to join the conversation is to converge the
> > discussion to a decision and resolution.