[PATCH 0/2] mm/shrinker: per-memcg shrinker units in an xarray, allocated on demand

[Konstantin Khorenko]

Every memory cgroup allocates, per NUMA node, one shrinker_info_unit
per 64 memcg-aware shrinker ids up to shrinker_nr_max, whether or not
it ever charges an object to those shrinkers, plus a pointer array
sized for shrinker_nr_max. Every superblock registers such a shrinker,
and shrinker_nr_max never comes down, so a host that mounts many
filesystems and runs many cgroups pays for every superblock in every
cgroup, live or dying, and keeps paying for new cgroups after the
mounts are gone.

This replaces patch 2/2 of v2, "mm/shrinker: allocate the per-memcg
shrinker units on demand", with the xarray variant discussed in that
thread. It applies on top of "mm/shrinker: allocate shrinker_info_unit
from a dedicated kmem_cache", patch 1/2 of v2 sent on its own as v3:
https://patch.msgid.link/20261019-vstor-144715-shrinker_info_unit-kmem_cache-v3-1-0fb061e09bf4@xxxxxxxxxxxxx

Patch 1 keeps the units in an xarray per memcg and node instead of the
pointer array. A new unit is inserted where it belongs: nothing is
reallocated, copied or published under RCU when shrinker_nr_max grows,
and the readers find a unit with xa_load() or walk the present ones
with xa_for_each(). The units are still allocated up front, so this
patch changes no behaviour.

Patch 2 allocates a unit only when the cgroup gets a list on that
shrinker's list_lru, which is where map bits are set: a cgroup pays for
the shrinkers it holds objects of, not for every superblock on the
host. The install is an xarray insert with the gfp of the list
allocation; there is no array to race with, so no lock of its own.
The root memcg keeps its units for the deferred work of global reclaim
and is the only memcg a shrinker registration still visits.

Measured on a 4-CPU, single-node VM, 20000 tmpfs mounts (shrinker_nr_max
20096) and 2000 memory cgroups, on three kernels:

  base                 v7.3-rc7 with the kmem_cache patch: every cgroup
                       has every unit, in a pointer array
  xarray               base + this series: units on demand, in an xarray
  old array on demand  base + patch 2/2 of v2, rebased: units on demand,
                       in the pointer array

                                                     old array
                                      base    xarray  on demand
  slab per empty cgroup               185 KB  18 KB   22 KB
  shrink_slab_memcg(), empty cgroup   1.6 us  0.2 us  1.6 us
  same, cgroup with objects on 100 sb 1.9 us  0.9 us  1.7 us
  mkdir/rmdir the cgroups, 4000 more
  mounts with them alive              same within noise

Changes from the v2 patch 2/2:
- The pointer array is replaced by an xarray in a patch of its own, so
  the on-demand patch no longer needs the spinlock that ordered an
  install against the array copy in expand_one_shrinker_info().
- Link to v2: https://patch.msgid.link/20261014-b4-prep-vstor-144715-size-shrinker_info_unit-to-a-power-of-two-v2-0-51af2aedbe6c@xxxxxxxxxxxxx

---
Konstantin Khorenko (2):
      mm/shrinker: keep the per-memcg shrinker units in an xarray
      mm/shrinker: allocate the per-memcg shrinker units on demand

 include/linux/memcontrol.h |   5 +-
 include/linux/shrinker.h   |   6 -
 mm/list_lru.c              |  15 +-
 mm/memcontrol.c            |   6 +-
 mm/shrinker.c              | 374 ++++++++++++++++++---------------------------
 5 files changed, 163 insertions(+), 243 deletions(-)
---
base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907
change-id: 20261019-b4-prep-vstor-144717-per-memcg-shrinker-units-on-demand-xarray-fbfd27a8cd69
prerequisite-message-id: <20261019-vstor-144715-shrinker_info_unit-kmem_cache-v3-1-0fb061e09bf4@xxxxxxxxxxxxx>
prerequisite-patch-id: 83371af931f7c67d55b59e51720ebaab143aefb6

Best regards,
--  
Konstantin Khorenko <khorenko@xxxxxxxxxxxxx>