Re: [PATCH RFC 00/14] The new slab memory controller

[Roman Gushchin]

On Tue, Sep 17, 2019 at 03:48:57PM -0400, Waiman Long wrote:
> On 9/5/19 5:45 PM, Roman Gushchin wrote:
> > The existing slab memory controller is based on the idea of replicating
> > slab allocator internals for each memory cgroup. This approach promises
> > a low memory overhead (one pointer per page), and isn't adding too much
> > code on hot allocation and release paths. But is has a very serious flaw:
> > it leads to a low slab utilization.
> >
> > Using a drgn* script I've got an estimation of slab utilization on
> > a number of machines running different production workloads. In most
> > cases it was between 45% and 65%, and the best number I've seen was
> > around 85%. Turning kmem accounting off brings it to high 90s. Also
> > it brings back 30-50% of slab memory. It means that the real price
> > of the existing slab memory controller is way bigger than a pointer
> > per page.
> >
> > The real reason why the existing design leads to a low slab utilization
> > is simple: slab pages are used exclusively by one memory cgroup.
> > If there are only few allocations of certain size made by a cgroup,
> > or if some active objects (e.g. dentries) are left after the cgroup is
> > deleted, or the cgroup contains a single-threaded application which is
> > barely allocating any kernel objects, but does it every time on a new CPU:
> > in all these cases the resulting slab utilization is very low.
> > If kmem accounting is off, the kernel is able to use free space
> > on slab pages for other allocations.
> >
> > Arguably it wasn't an issue back to days when the kmem controller was
> > introduced and was an opt-in feature, which had to be turned on
> > individually for each memory cgroup. But now it's turned on by default
> > on both cgroup v1 and v2. And modern systemd-based systems tend to
> > create a large number of cgroups.
> >
> > This patchset provides a new implementation of the slab memory controller,
> > which aims to reach a much better slab utilization by sharing slab pages
> > between multiple memory cgroups. Below is the short description of the new
> > design (more details in commit messages).
> >
> > Accounting is performed per-object instead of per-page. Slab-related
> > vmstat counters are converted to bytes. Charging is performed on page-basis,
> > with rounding up and remembering leftovers.
> >
> > Memcg ownership data is stored in a per-slab-page vector: for each slab page
> > a vector of corresponding size is allocated. To keep slab memory reparenting
> > working, instead of saving a pointer to the memory cgroup directly an
> > intermediate object is used. It's simply a pointer to a memcg (which can be
> > easily changed to the parent) with a built-in reference counter. This scheme
> > allows to reparent all allocated objects without walking them over and changing
> > memcg pointer to the parent.
> >
> > Instead of creating an individual set of kmem_caches for each memory cgroup,
> > two global sets are used: the root set for non-accounted and root-cgroup
> > allocations and the second set for all other allocations. This allows to
> > simplify the lifetime management of individual kmem_caches: they are destroyed
> > with root counterparts. It allows to remove a good amount of code and make
> > things generally simpler.
> >
> > The patchset contains a couple of semi-independent parts, which can find their
> > usage outside of the slab memory controller too:
> > 1) subpage charging API, which can be used in the future for accounting of
> >    other non-page-sized objects, e.g. percpu allocations.
> > 2) mem_cgroup_ptr API (refcounted pointers to a memcg, can be reused
> >    for the efficient reparenting of other objects, e.g. pagecache.
> >
> > The patchset has been tested on a number of different workloads in our
> > production. In all cases, it saved hefty amounts of memory:
> > 1) web frontend, 650-700 Mb, ~42% of slab memory
> > 2) database cache, 750-800 Mb, ~35% of slab memory
> > 3) dns server, 700 Mb, ~36% of slab memory
> >
> > So far I haven't found any regression on all tested workloads, but
> > potential CPU regression caused by more precise accounting is a concern.
> >
> > Obviously the amount of saved memory depend on the number of memory cgroups,
> > uptime and specific workloads, but overall it feels like the new controller
> > saves 30-40% of slab memory, sometimes more. Additionally, it should lead
> > to a lower memory fragmentation, just because of a smaller number of
> > non-movable pages and also because there is no more need to move all
> > slab objects to a new set of pages when a workload is restarted in a new
> > memory cgroup.
> >
> > * https://github.com/osandov/drgn
> >
> >
> > Roman Gushchin (14):
> >   mm: memcg: subpage charging API
> >   mm: memcg: introduce mem_cgroup_ptr
> >   mm: vmstat: use s32 for vm_node_stat_diff in struct per_cpu_nodestat
> >   mm: vmstat: convert slab vmstat counter to bytes
> >   mm: memcg/slab: allocate space for memcg ownership data for non-root
> >     slabs
> >   mm: slub: implement SLUB version of obj_to_index()
> >   mm: memcg/slab: save memcg ownership data for non-root slab objects
> >   mm: memcg: move memcg_kmem_bypass() to memcontrol.h
> >   mm: memcg: introduce __mod_lruvec_memcg_state()
> >   mm: memcg/slab: charge individual slab objects instead of pages
> >   mm: memcg: move get_mem_cgroup_from_current() to memcontrol.h
> >   mm: memcg/slab: replace memcg_from_slab_page() with
> >     memcg_from_slab_obj()
> >   mm: memcg/slab: use one set of kmem_caches for all memory cgroups
> >   mm: slab: remove redundant check in memcg_accumulate_slabinfo()
> >
> >  drivers/base/node.c        |  11 +-
> >  fs/proc/meminfo.c          |   4 +-
> >  include/linux/memcontrol.h | 102 ++++++++-
> >  include/linux/mm_types.h   |   5 +-
> >  include/linux/mmzone.h     |  12 +-
> >  include/linux/slab.h       |   3 +-
> >  include/linux/slub_def.h   |   9 +
> >  include/linux/vmstat.h     |   8 +
> >  kernel/power/snapshot.c    |   2 +-
> >  mm/list_lru.c              |  12 +-
> >  mm/memcontrol.c            | 431 +++++++++++++++++++++--------------
> >  mm/oom_kill.c              |   2 +-
> >  mm/page_alloc.c            |   8 +-
> >  mm/slab.c                  |  37 ++-
> >  mm/slab.h                  | 300 +++++++++++++------------
> >  mm/slab_common.c           | 449 ++++---------------------------------
> >  mm/slob.c                  |  12 +-
> >  mm/slub.c                  |  63 ++----
> >  mm/vmscan.c                |   3 +-
> >  mm/vmstat.c                |  38 +++-
> >  mm/workingset.c            |   6 +-
> >  21 files changed, 683 insertions(+), 834 deletions(-)
> >
> I can only see the first 9 patches. Patches 10-14 are not there.

Hm, strange. I'll rebase the patchset on top of the current mm tree and resend.

In the meantime you can find the original patchset here:
  https://github.com/rgushchin/linux/tree/new_slab.rfc
or on top of the 5.3 release, which might be better for testing here:
  https://github.com/rgushchin/linux/tree/new_slab.rfc.v5.3

Thanks!