Re: [RFC PATCH 00/10] mm/swap: ghost swapfile with backend switching via Redirect entries

[Baoquan He]

On 07/17/26 at 09:02am, Nhat Pham wrote:
> On Fri, Jul 17, 2026 at 7:27 AM Baoquan He <baoquan.he@xxxxxxxxx> wrote:
> >
> > On 07/16/26 at 09:49am, Nhat Pham wrote:
> >
> > Thanks for the thoughtful reply and the very constructive proposal.
> > Your suggestion about moving forward in small, incremental steps in
> > last mail is really appealing — I think that's the right way to make
> > progress on this. I need a little time to think it through more
> > carefully before giving you a proper response. Will get back to you
> > soon.
> 
> Yeah let me know what you think. I'm planning to send out v3 with some
> more cleanups and fixes, but I'm happy to hear what you and Chris
> wants to make it as useful as possible for everyone.

Hi Nhat,

I've thought about it more carefully. Here's what I think the path
forward could look like. I'll try to lay it out from the bottom up.

1. Data structure

   Instead of introducing a separate swap_cluster_info_dynamic wrapper,
   I'd prefer to add a per-slot backend pointer array directly into
   struct swap_cluster_info:

   struct swap_cluster_info {
       ...
   #ifdef CONFIG_ZSWAP
       atomic_long_t *vs_pointer;   /* or xs_pointer — per-slot backend pointers */
   #endif
       struct list_head list;
   };

   This is allocated together with or alongside the swap table when the
   cluster is first used.  No wrapper struct, no new ->cluster_info
   pointer type change — just one extra field that is NULL for clusters
   that don't need it.

   The name is bike-sheddable: vs_pointer (virtual-space pointer) or
   xs_pointer (cross-space pointer), xs_table (xswap extension table),
   aux_table (auxiliary table), priv_table (private data table),
   slot_priv (per-slot private data) or s_ext (slot extension).  The
   point is that it stores a pointer to the backend owner of each slot —
   initially just struct zswap_entry *, eventually extensible to other
   backend types. I prefer to name it ext_table, but we have had
   extend_table. My second favorite option is aux_table. What's your
   choice?

2. Eliminate the zswap xarray

   Once vs_pointer is in place, zswap can store and load entries
   through vs_pointer[ci_off] instead of xa_store/xa_load on the
   per-device zswap xarray.  The cluster lock protects the access.
   This is exactly the same idea as your virtual_table, just placed
   directly in swap_cluster_info rather than in a wrapper.

   The benefit: zswap_swapon and zswap_swapoff become no-ops (no
   global xarray to allocate or tear down).  The per-cluster allocation
   is lazy — only clusters that actually see a zswap store allocate the
   array.

3. First step for ghost swap / virtual swap: dynamic growth and shrink,
   no writeback yet

   I think we agree that the first landing should be narrow in scope:
   - A virtual swap device that decouples PTE swap entries from
     physical backing.
   - Backed only by zswap (and zero pages) initially.
   - No writeback to physical swap.
   - Dynamic growth and shrink of the cluster space.

   For the dynamic cluster management, I'd like to try a lazy vmalloc
   approach: allocate the cluster_info array as a vmalloc region, and
   use vm_area_map_pages() to map newly populated clusters on demand.
   Growth is automatic via page faults; shrink means reclaiming the
   physical pages backing freed clusters.

   The vmalloc approach has O(1) lookup (simple array index), which
   matters on the swapout hot path.  The trade-off is that shrink is
   coarser than xarray-based management (we can't easily unmap an
   individual cluster from the middle of the vmalloc region).  But for
   the initial landing where swap usage tends to be ratchet-like, I
   think this is a reasonable trade-off.  We can always switch to
   xarray later if shrink granularity becomes a real issue.

4. Writeback and rmap as follow-up

   Once the basic virtual swap device is upstream, we add writeback
   support:
   - When zswap store fails, fall back to allocating a physical swap
     slot and writing out.
   - The vs_pointer entry is updated to point to the physical slot
     (tagged to distinguish from a zswap_entry pointer).
   - rmap for physical -> virtual reverse lookup, needed for swapoff
     and physical slot reclaim.

   This is a natural extension: the vs_pointer is already per-slot, so
   adding a new backend type is a tag-bit change, not a rewrite.

5. Longer-term: swap tiering

   Looking further ahead, the per-slot pointer can also serve as the
   basis for a swap tiering model.  Imagine a memcg with multiple
   tiers: ghost/virtual swap at the top, then pmem, then SSD, then
   HDD.  Swapout goes to the top tier first; when that fills, data
   cascades down to the next tier.  Each swap device only needs to
   know about the device directly below it — a simple "push down"
   interface.

   This is a longer-term vision, but having a uniform per-slot backend
   pointer from the start keeps the door open without painting us into
   a corner.

6. Naming

   We have three names floating around: "ghost swap", "virtual swap",
   and Chris suggested "xswap".  We're going to be talking about this
   thing a lot, and having a single agreed-upon name would help avoid
   confusion in discussions, commit messages, and documentation.  I
   don't have strong feelings about which one — but I'd like us to
   settle on something everyone can live with.

7. How to swapon a ghost / virtual swap device

   There are several approaches on the table right now:

   a) A sparse file created via truncate (Chris's approach): a swap
      file that looks large (e.g. 1GiB) but only has one real backing
      page.  Works with the existing swapon syscall, but the sparse
      file has to live somewhere and the filesystem interaction is a
      bit of a hack.

   b) A boot-time, always-present device (your VSS approach): the
      vswap device is created unconditionally at late_initcall when
      CONFIG_VSWAP=y.  No userspace action needed, but no runtime
      control either — you can't create or destroy it, and you can't
      have per-memcg or per-workload virtual swap spaces.

   c) A swapon flag: something like SWAP_FLAG_VIRTUAL or
      SWAP_FLAG_GHOST passed to swapon(2), which tells the kernel to
      treat the swap device as a ghost device with no backing storage.

   d) A dedicated sysfs / debugfs knob: create and size the ghost
      swap device at runtime via a file under /sys/kernel/mm/swap/.

   e) Something I'm missing?

   Each has trade-offs between flexibility, simplicity, and the
   existing swap management model.  I'd like to hear what you and
   Chris think — especially whether the boot-time-only nature of the
   current VSS design is a deliberate simplification or something
   you'd be open to revisiting.

Curious to hear what you think — especially if you see any structural
issues with putting vs_pointer directly in swap_cluster_info rather
than in a wrapper.  If the wrapper is important for your v3 design,
I'd like to understand the rationale better.

Thanks,
Baoquan