Re: Path forward for Virtualized Swap?

[Nhat Pham]

On Sat, Oct 24, 2026 at 9:22 PM Kairui Song <ryncsn@xxxxxxxxx> wrote:
>
> Rik van Riel <riel@xxxxxxxxxxx> 于 2026年10月25日周日 02:16写道：
> >
> > On Sat, 2026-10-24 at 23:50 +0800, Kairui Song wrote:
> > > On Sat, Oct 24, 2026 at 4:03 AM Rik van Riel <riel@xxxxxxxxxxx>
> > > wrote:
> > > >
> > > > On Fri, 2026-10-23 at 10:24 -0700, Nhat Pham wrote:
> > > > >
> > > > > * Swapoff: this was brought up as a request, but we ran out of
> > > > > time
> > > > > before we could discuss the motivations and use cases behind
> > > > > them.
> > > > > Folks interested, could you explain as a follow-up?
> > > >
> > > > This is one that I really don't understand.
> > > >
> > > > In my mind, we have vswap sitting as kind
> > > > of a swap VFS between the rest of the system
> > > > and the swap devices:
> > > >
> > > > +-------+     +------------+
> > > > > vswap |-----| swap cache |
> > > > +---+---+     +------------+
> > > >     |
> > > >     +----------+-----------+
> > > >     |          |           |
> > > > +---+---+  +---+---+   +---+---+
> > > > > swap  |  | zswap |   | zram  |
> > > > > file  |  |       |   |       |
> > > > +-------+  +-------+   +-------+
> > > >
> > > > When a page is unmapped from userspace by the
> > > > reclaim code, it goes into the swap cache.
> > > >
> > > > The vswap layer then decides where to store it
> > > > at eviction time. This is configurable by userspace.
> > > >
> > > > If the zswap pool is shrunk, the vswap layer knows
> > > > where to put the data next. There is no need to
> > > > change page table entries to indicate that the
> > > > data was moved from zswap to a swap file.
> > > >
> > > > If we decide to disable one of the backing swap
> > > > devices, the vswap layer can just move the data
> > > > into the swap cache, and the page reclaim code
> > > > can evict it into a different backing store, if
> > > > needed.
> > >
> > > And in this case we want to disable vswap itself.
> >
> > OK, so you are looking for a way to totally depopulate
> > the swap cache, in addition to swap files, zswap, etc?
> >
> > I think we can accomplish that by simply keeping the
> > current swapoff code around, which scans through the
> > virtual memory of all processes and brings the memory
> > out of swap.
>
> Yeah, that's the right starting point. The scan itself can also be
> optimized to be cheaper or better later on, but that's orthogonal.
>
> > > I also hope there is a way to release swap metadata completely too.
> > > e.g. for fs, you at least have umount, and can rmmod the fs itself
> > > afterwards. We also have many ways to bypass the page cache. In cases
> > > where a virtual layer is just a burden, we want raw physical swap,
> > > and
> > > the ability to swap off that too.
> >
> > The VM cannot operate without the swap cache, though.
> >
> > That is very much built into core parts of the VM.
>
> I never meant to strip swap cache out of VM... The opposite, actually
> :), most of my work over the last couple of years has been about
> removing the paths that bypass the swap cache (e.g. the
> SWP_SYNCHRONOUS_IO fast path), to get a simpler model and better
> performance.
>
> What I'd like is for swapoff of the last swap device to return the
> system to its pre-swapon state: no leftover swap metadata, no leftover
> per-slot or per-cgroup data or statistic, and ideally no residual cost
> in the fast paths (e.g. minor fault).

So basically remapping? I don't think that's terribly hard to do. I
mean the swapoff code is still there, so it's just the question of
exposing an off knob, basically?

But wouldn't MADV_WILLNEED be the better tool here? Or is it not reliable?

>
> > However, having swapoff remove all data from the swap
> > cache, after mapping the memory back into userspace
> > page tables, seems like it would be fairly easy to do.
> >
> > Does that meet the needs of your use case?
>
> Right, it's nothing hard, as long as we don't add mandatory new
> layers. For very high bandwidth swap devices, a mandatory indirection
> layer is pure overhead. I posted a per-CPU queue based allocator for
> that case (https://lore.kernel.org/linux-mm/20260829-swap-pcp-priq-v2-resend-0-68d3d925578c@xxxxxxxxx/).

I skipped non-zswap cgroups for now, so there is no overhead :)

We can revisit this once a new feature is implemented (and we can
figure out what interface is best for this, or can the kernel figure
this out on its own).