Re: [RFC PATCH 1/1] mm/swap: allocate contiguous swap slots for a reclaim batch

[Alexandre Ghiti]

Hi Nhat,

On Fri, Oct 23, 2026 at 6:05 PM Nhat Pham <nphamcs@xxxxxxxxx> wrote:
>
> >
> On Thu, Oct 22, 2026 at 7:57 AM Alexandre Ghiti <alex@xxxxxxxx> wrote:
> >
> > Reclaim allocates swap slots one folio at a time and only then batches
> > the writes, merging folios whose slots happen to be contiguous. Order-0
> > allocations scan the nonfull clusters before taking a free cluster, so
> > once frees have left holes in otherwise full clusters, a batch of LRU-
> > adjacent folios is scattered over those holes and written with many
> > small requests. Since swap slots are freed on exclusive write faults,
> > this happens with any workload that rewrites swapped out memory.
> >
> > Before swapping out a batch, record how many pages of each small order
> > need a slot, and make those allocations follow contiguous free slots,
> > SWAP_CLUSTER_MAX at a time: in a nonfull cluster if one has them,
> > otherwise in a free cluster. When neither has them, the rest of the
> > batch fills the holes as before. Nothing is reserved.
> >
> > Signed-off-by: Alexandre Ghiti <alex@xxxxxxxx>
>
> Thanks for looking into this! There are still workloads that benefit
> more from disk swapping than compression swapping... We should make
> the swap subsystem work for them.
>
> I haven't read the code too closely, as it's an RFC meant to show the
> general idea. So I'll just make some comments and ask some questions
> about the idea:
>
> >  static bool __swap_cluster_alloc_entries(struct swap_info_struct *si,
> >                                          struct swap_cluster_info *ci,
> >                                          struct folio *folio,
> > @@ -1042,6 +1079,59 @@ static unsigned int alloc_swap_scan_list(struct swap_info_struct *si,
> >         return found;
> >  }
> >
> > +/* Larger folios already make large enough write requests on their own */
> > +#define SWAP_BATCH_MAX_ORDER   (ilog2(SWAP_CLUSTER_MAX) - 1)
>
> Ah so we don't invoke this heuristics for folio larger than certain
> orders? Makes sense.

Yes, large folio get their own allocation scheme, here the idea is to
"pack" small folios, I'll add that to the changelog.

>
> > +
> > +static inline unsigned int swap_batch_contig(unsigned int left)
> > +{
> > +       return min_t(unsigned int, left, SWAP_CLUSTER_MAX);
> > +}
> > +
> > +/*
> > + * Like alloc_swap_scan_list(), but only allocate at the start of @nr_contig
> > + * free slots. Order 0 clusters without them move to the frag list until a
> > + * slot is freed, larger orders stay nonfull as the folio may still fit.
> > + */
> > +static unsigned int alloc_swap_scan_list_contig(struct swap_info_struct *si,
> > +                                               struct list_head *list,
> > +                                               struct folio *folio,
> > +                                               unsigned int nr_contig)
> > +{
> > +       unsigned int found = SWAP_ENTRY_INVALID, order = folio_order(folio);
> > +       struct swap_cluster_info *ci;
> > +       unsigned long start, offset;
> > +       LIST_HEAD(skipped);
> > +
> > +       while ((ci = isolate_lock_cluster(si, list))) {
> > +               start = cluster_offset(si, ci);
> > +               offset = SWAP_ENTRY_INVALID;
> > +               if (SWAPFILE_CLUSTER - ci->count >= nr_contig)
> > +                       offset = cluster_find_free_contig(ci, start,
> > +                                                         start + SWAPFILE_CLUSTER,
> > +                                                         1 << order, nr_contig);
> > +               if (offset == SWAP_ENTRY_INVALID) {
> > +                       if (order)
> > +                               move_cluster(si, ci, &skipped,
> > +                                            CLUSTER_FLAG_NONFULL);
> > +                       else
> > +                               relocate_cluster(si, ci);
> > +                       swap_cluster_unlock(ci);
> > +                       continue;
> > +               }
> > +               found = alloc_swap_scan_cluster(si, ci, folio, offset);
> > +               if (found)
> > +                       break;
> > +       }
> > +
> > +       if (!list_empty(&skipped)) {
> > +               spin_lock(&si->lock);
> > +               list_splice_tail(&skipped, list);
> > +               spin_unlock(&si->lock);
> > +       }
> > +
> > +       return found;
> > +}
> > +
> >  static void swap_reclaim_full_clusters(struct swap_info_struct *si, bool force)
> >  {
> >         long to_scan = 1;
> > @@ -1109,6 +1199,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
> >         struct swap_cluster_info *ci;
> >         unsigned int order = likely(folio) ? folio_order(folio) : 0;
> >         unsigned int offset = SWAP_ENTRY_INVALID, found = SWAP_ENTRY_INVALID;
> > +       unsigned int left;
> >
> >         /*
> >          * Swapfile is not block device so unable
> > @@ -1129,7 +1220,14 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
> >                 if (cluster_is_usable(ci, order)) {
> >                         if (cluster_is_empty(ci))
> >                                 offset = cluster_offset(si, ci);
> > -                       found = alloc_swap_scan_cluster(si, ci, folio, offset);
> > +                       /* The batch's contiguous slots are used up */
> > +                       if (likely(folio) &&
> > +                           this_cpu_read(percpu_swap_cluster.batch_left[order]) &&
> > +                           !cluster_slots_free(ci, offset, 1 << order))
> > +                               swap_cluster_unlock(ci);
> > +                       else
> > +                               found = alloc_swap_scan_cluster(si, ci, folio,
> > +                                                               offset);
> >                 } else {
> >                         swap_cluster_unlock(ci);
> >                 }
> > @@ -1148,6 +1246,21 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
> >                         goto done;
> >         }
> >
> > +       /* Keep a reclaim batch on contiguous free slots, not in small holes */
> > +       left = likely(folio) ?
> > +              this_cpu_read(percpu_swap_cluster.batch_left[order]) : 0;
> > +       if (left && order < PMD_ORDER) {
> > +               found = alloc_swap_scan_list_contig(si, &si->nonfull_clusters[order],
> > +                                                   folio, swap_batch_contig(left));
> > +               if (found)
> > +                       goto done;
> > +               found = alloc_swap_scan_list(si, &si->free_clusters, folio, false);
> > +               if (found)
> > +                       goto done;
> > +               /* No contiguous slots left, allocate the rest of the batch as usual */
> > +               this_cpu_write(percpu_swap_cluster.batch_left[order], 0);
> > +       }
> > +
> >         if (order < PMD_ORDER) {
> >                 found = alloc_swap_scan_list(si, &si->nonfull_clusters[order], folio, true);
>
> So we prefer non-full clusters with contiguous slots, over free
> clusters, over other nonfull clusters.
>
> I think this part makes sense to me FWIW. We do skip fast path and go
> for slow paths more often, so do make sure we're not paying with too
> much lock contention, especially as the swapfile fils or get more
> fragmented (which this patch series does nudge us towards). But FWIW,
> we pretty much go back to the old behavior the moment we fail to find
> contiguity for the first time in the batch, so I think we'll be
> fine...?

I'll check the lock contention, thanks.

>
> >                 if (found)
> > @@ -1390,7 +1503,15 @@ static bool swap_alloc_fast(struct folio *folio)
> >         if (cluster_is_usable(ci, order)) {
> >                 if (cluster_is_empty(ci))
> >                         offset = cluster_offset(si, ci);
> > -               alloc_swap_scan_cluster(si, ci, folio, offset);
> > +               /* The batch's contiguous slots are used up */
> > +               if (this_cpu_read(percpu_swap_cluster.batch_left[order]) &&
> > +                   !cluster_slots_free(ci, offset, 1 << order)) {
> > +                       swap_cluster_unlock(ci);
> > +                       this_cpu_write(percpu_swap_cluster.offset[order],
> > +                                      SWAP_ENTRY_INVALID);
> > +               } else {
> > +                       alloc_swap_scan_cluster(si, ci, folio, offset);
> > +               }
> >         } else {
> >                 swap_cluster_unlock(ci);
> >         }
> > @@ -1729,6 +1850,86 @@ static int swap_dup_entries_cluster(struct swap_info_struct *si,
> >         return err;
> >  }
> >
> > +static void swap_alloc_prepare(unsigned int order, unsigned int nr)
> > +{
> > +       unsigned int nr_contig = swap_batch_contig(nr);
> > +       struct swap_cluster_info *ci;
> > +       struct swap_info_struct *si;
> > +       unsigned long offset;
> > +       bool keep = false;
> > +
> > +       local_lock(&percpu_swap_cluster.lock);
> > +       if (nr <= (1 << order)) {
> > +               this_cpu_write(percpu_swap_cluster.batch_left[order], 0);
> > +               goto out;
> > +       }
> > +       this_cpu_write(percpu_swap_cluster.batch_left[order], nr);
> > +
> > +       /* Keep the cursor if enough free slots follow it */
> > +       si = this_cpu_read(percpu_swap_cluster.si[order]);
> > +       offset = this_cpu_read(percpu_swap_cluster.offset[order]);
> > +       if (si && offset && get_swap_device_info(si)) {
> > +               ci = swap_cluster_lock(si, offset);
> > +               if (cluster_is_usable(ci, order)) {
> > +                       if (cluster_is_empty(ci))
> > +                               offset = cluster_offset(si, ci);
> > +                       keep = offset % SWAPFILE_CLUSTER + nr_contig <=
> > +                              SWAPFILE_CLUSTER &&
> > +                              cluster_find_free_contig(ci, offset,
> > +                                                       offset + nr_contig,
> > +                                                       1 << order,
> > +                                                       nr_contig) == offset;
> > +               }
> > +               swap_cluster_unlock(ci);
> > +               put_swap_device(si);
> > +       }
> > +       if (!keep)
> > +               this_cpu_write(percpu_swap_cluster.offset[order],
> > +                              SWAP_ENTRY_INVALID);
> > +out:
> > +       local_unlock(&percpu_swap_cluster.lock);
> > +}
> > +
> > +/**
> > + * folio_alloc_swap_prepare - prepare swap allocation for a reclaim batch
> > + * @folios: list of folios reclaim is about to swap out
> > + *
> > + * Record how many pages of each small order the batch will swap out, so
> > + * that their slots are allocated contiguously. Nothing is reserved.
> > + */
> > +void folio_alloc_swap_prepare(struct list_head *folios)
> > +{
> > +       unsigned int max_order = min(SWAP_BATCH_MAX_ORDER, SWAP_NR_ORDERS - 1);
> > +       unsigned int nr[SWAP_BATCH_MAX_ORDER + 1] = {};
> > +       struct folio *folio;
> > +       unsigned int order;
> > +
> > +       if (!total_swap_pages)
> > +               return;
> > +
> > +       list_for_each_entry(folio, folios, lru) {
> > +               if (!folio_test_swapbacked(folio) || folio_test_swapcache(folio))
> > +                       continue;
> > +               order = folio_order(folio);
> > +               if (order <= max_order)
> > +                       nr[order] += 1 << order;
> > +       }
> > +
> > +       for (order = 0; order <= max_order; order++)
> > +               swap_alloc_prepare(order, nr[order]);
> > +}
> > +
> > +/**
> > + * folio_alloc_swap_split - account for a large folio split by reclaim
> > + * @nr_pages: number of order-0 folios the split produced
> > + */
> > +void folio_alloc_swap_split(unsigned int nr_pages)
> > +{
> > +       local_lock(&percpu_swap_cluster.lock);
> > +       this_cpu_add(percpu_swap_cluster.batch_left[0], nr_pages);
> > +       local_unlock(&percpu_swap_cluster.lock);
> > +}
> > +
> >  /**
> >   * folio_alloc_swap - allocate swap space for a folio
> >   * @folio: folio we want to move to swap
> > @@ -1771,6 +1972,12 @@ int folio_alloc_swap(struct folio *folio)
> >         local_lock(&percpu_swap_cluster.lock);
> >         if (!swap_alloc_fast(folio))
> >                 swap_alloc_slow(folio);
> > +       if (folio_test_swapcache(folio)) {
> > +               unsigned int left = this_cpu_read(percpu_swap_cluster.batch_left[order]);
> > +
> > +               this_cpu_write(percpu_swap_cluster.batch_left[order],
> > +                              left > size ? left - size : 0);
> > +       }
> >         local_unlock(&percpu_swap_cluster.lock);
> >
> >         if (!order && unlikely(!folio_test_swapcache(folio))) {
> > diff --git a/mm/vmscan.c b/mm/vmscan.c
> > index ea2d4c8d1890..86245036872c 100644
> > --- a/mm/vmscan.c
> > +++ b/mm/vmscan.c
> > @@ -1175,6 +1175,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
> >         memset(stat, 0, sizeof(*stat));
> >         cond_resched();
> >         do_demote_pass = can_demote(pgdat->node_id, sc, memcg);
> > +       if (sc->gfp_mask & __GFP_IO)
> > +               folio_alloc_swap_prepare(folio_list);
> >
>
> Some general questions:
>
> 1. How many pages in folio_list typically do get reclaimed? There can
> be quite a few with references, which we rotate/promote, when we scan,
> right? Would constantly overshooting the estimate create problem down
> the line?
>

I need to trace some live hosts, I'll get back with numbers.

> (This is one of the few places where indirection (vswap) would have
> helped - we can collect a batch of pages after unmapping, then pass
> them to the phys swap allocator (after checking zswappability, etc.).
> Maybe we can solve it better after it's landed, and accept an
> imperfect solution for now?)
>
> 2. Should we skip this heuristics for cgroups that enables zswap (and
> maybe zram)? Contiguity matters less there... unless it help at
> writeback time? Hard to say without experimentation - maybe you can do
> the same check you did for your zswap writeback batching patch series?
>

I was wondering the same, I'll check how it impacts zswap + writeback.

Thanks!

Alex