Re: Path forward for Virtualized Swap?

[Kairui Song]

Rik van Riel <riel@xxxxxxxxxxx> 于 2026年10月25日周日 02:16写道：
>
> On Sat, 2026-10-24 at 23:50 +0800, Kairui Song wrote:
> > On Sat, Oct 24, 2026 at 4:03 AM Rik van Riel <riel@xxxxxxxxxxx>
> > wrote:
> > >
> > > On Fri, 2026-10-23 at 10:24 -0700, Nhat Pham wrote:
> > > >
> > > > * Swapoff: this was brought up as a request, but we ran out of
> > > > time
> > > > before we could discuss the motivations and use cases behind
> > > > them.
> > > > Folks interested, could you explain as a follow-up?
> > >
> > > This is one that I really don't understand.
> > >
> > > In my mind, we have vswap sitting as kind
> > > of a swap VFS between the rest of the system
> > > and the swap devices:
> > >
> > > +-------+     +------------+
> > > > vswap |-----| swap cache |
> > > +---+---+     +------------+
> > >     |
> > >     +----------+-----------+
> > >     |          |           |
> > > +---+---+  +---+---+   +---+---+
> > > > swap  |  | zswap |   | zram  |
> > > > file  |  |       |   |       |
> > > +-------+  +-------+   +-------+
> > >
> > > When a page is unmapped from userspace by the
> > > reclaim code, it goes into the swap cache.
> > >
> > > The vswap layer then decides where to store it
> > > at eviction time. This is configurable by userspace.
> > >
> > > If the zswap pool is shrunk, the vswap layer knows
> > > where to put the data next. There is no need to
> > > change page table entries to indicate that the
> > > data was moved from zswap to a swap file.
> > >
> > > If we decide to disable one of the backing swap
> > > devices, the vswap layer can just move the data
> > > into the swap cache, and the page reclaim code
> > > can evict it into a different backing store, if
> > > needed.
> >
> > And in this case we want to disable vswap itself.
>
> OK, so you are looking for a way to totally depopulate
> the swap cache, in addition to swap files, zswap, etc?
>
> I think we can accomplish that by simply keeping the
> current swapoff code around, which scans through the
> virtual memory of all processes and brings the memory
> out of swap.

Yeah, that's the right starting point. The scan itself can also be
optimized to be cheaper or better later on, but that's orthogonal.

> > I also hope there is a way to release swap metadata completely too.
> > e.g. for fs, you at least have umount, and can rmmod the fs itself
> > afterwards. We also have many ways to bypass the page cache. In cases
> > where a virtual layer is just a burden, we want raw physical swap,
> > and
> > the ability to swap off that too.
>
> The VM cannot operate without the swap cache, though.
>
> That is very much built into core parts of the VM.

I never meant to strip swap cache out of VM... The opposite, actually
:), most of my work over the last couple of years has been about
removing the paths that bypass the swap cache (e.g. the
SWP_SYNCHRONOUS_IO fast path), to get a simpler model and better
performance.

What I'd like is for swapoff of the last swap device to return the
system to its pre-swapon state: no leftover swap metadata, no leftover
per-slot or per-cgroup data or statistic, and ideally no residual cost
in the fast paths (e.g. minor fault).

> However, having swapoff remove all data from the swap
> cache, after mapping the memory back into userspace
> page tables, seems like it would be fairly easy to do.
>
> Does that meet the needs of your use case?

Right, it's nothing hard, as long as we don't add mandatory new
layers. For very high bandwidth swap devices, a mandatory indirection
layer is pure overhead. I posted a per-CPU queue based allocator for
that case (https://lore.kernel.org/linux-mm/20260829-swap-pcp-priq-v2-resend-0-68d3d925578c@xxxxxxxxx/).