Re: [RFC PATCH 00/11] mm, swap: dynamic cluster management for xswap devices

[Nhat Pham]

On Mon, Jul 27, 2026 at 6:50 AM Baoquan He <baoquan.he@xxxxxxxxx> wrote:
>
> This series implements step 3 of the incremental path proposed in [1]:
> a virtual swap device with dynamic cluster growth and shrink, backed
> only by zswap, no writeback yet.
>
> During the discussion in [2], Chris and Nhat debated xarray-based vs
> array-based cluster lookup for virtual swap.  Chris argued for keeping
> the swap table and cluster_info in the same place with O(1) array
> indexing, and suggested that the kvmalloc grow approach here should
> provide the cluster management VS needs.  He further proposed using
> the same grow technique for the future per-slot backend pointer array
> (vs_table), keeping a single coherent array layout throughout.
>
> This series therefore serves as the foundation: once the dynamic
> cluster management lands, Nhat's VS series can build the per-slot
> backend pointer, writeback, and rmap on top of it — without an extra
> xarray lookup in the cluster access path.

Thanks Baoquan. I've given it some more thoughts over the past week or
so, to figure out a way to so that everyone would be happy with the
final results.

To be perfectly clear, I'm not married with the xarray data structure.
I only care about use case aspect (as you have seen, I switch over to
swap table the moment I figure out a way to realize Kairui's design
proposal - it's way cleaner) - and I've seen that you have addressed a
lot of the design concerns I had that would hindered the use cases I
presented. If the vmalloc data structure outperforms the xarray, it
should *definitely* be the data structure we use when everything
settles.

That just leaves the sequencing of our efforts :)

My only fear with landing one piece at a time is that that we land
something that ended up not working for the entire big series in some
unforeseen cases, and we have to rip it out or rewrite a huge chunks
of it. Besides, without an actual use case, we can't really test its
correctness, and purported performance improvement benefits.

That's why so far I have bitten the bullet and sent out the whole
series (even if it's really large) with the core use cases
implemented. I understand that it is big, and is burdensome for
reviewers - but at least you know that the core use cases works - and
if you don't take my work for it, you can test with it and benchmark
it, etc. And with Kairui's proposal, we still maintain the old
machinery, which allows us to even merge with something a bit
suboptimal, then optimize later. FWIW, so far in my testing, the
numbers have been quite close, at least for zswap backend, and the
core concern (memory overhead) has been resolved. So it might actually
already be usable out of the box? :)

This patch series has reached 700 LoCs, and I still have some
correctness concerns - I have commented one in another patch. It has
gotten more complicated now, which is no fault of you - we just need
more logic to handle the things xarray provided us out of the box,
with the hope that it would be more optimal. And that's fine. I just
think it would be better to land the xarray version first to unblock
users, then just slot your patch series on top as a follow-up
optimization. It would show the actual win of your data structure over
xarray data structure with concrete numbers :) Win-win?

But thanks again for taking a stab at it. Seriously, you're awesome. I
want to re-iterate again - this is NOT a Nacked-by Nhat. Not with this
patch series, nor with the direction. I'm just trying to figure out
the sequencing of our efforts that would make everything work. I'll
spend sometimes on my own trying to synthesize this data structure
with vswap design too, to see if I can fish out more gaps, and to see
if I can clean things up in a way that would make the switch from
xarray to this new data structure as painless as possible. As well as
the other design points we have touched on in our last conversation -
it makes sense to me at the time, but I need to code some prototype
out and stare at it a bit more.