[PATCH v2 0/1] rcu: make userspace barrier hook drain kvfree_rcu work

[Matthias Goergens]

The rcutree.do_rcu_barrier hook currently waits for ordinary RCU
callbacks, but objects may still be retained in kfree_rcu() batching or a
partial per-CPU SLUB sheaf. This is consistent with the hook's documented
rcu_barrier() operation, but incomplete for its intended use as a boundary
between userspace tests.

The immediate trigger was a false allocation-leak failure in the bcachefs
ktest suite while testing performance changes. Its end check writes the
hook before reading /proc/allocinfo, assuming a complete deferred-free
drain. Small objects remained visible after repeated hook writes and
20 seconds of waiting, so otherwise clean tests failed their leak check.

Changing the hook to drain kvfree_rcu() work let the same unmodified
bcachefs workload pass its allocation check. All eight checkpoints in
one VM, after 50 through 400 option changes, reported zero retained
reconcile_scan objects. The retained population on the original kernel
eventually fell as a sheaf filled; there is no evidence here of unbounded
growth or OOM.

Calling kvfree_rcu_barrier() from rcu_barrier_throttled() was proposed and
agreed during review of the former API in 2024, specifically to restore a
clean baseline between userspace benchmark runs:

  https://lore.kernel.org/all/20240820155935.1167988-1-urezki@xxxxxxxxx/

This patch implements that follow-up and documents the expanded hook. It
also removes the old ordinary-barrier completion shortcut: an unrelated
rcu_barrier() does not establish that kvfree_rcu() work was drained.

Four counterbalanced fresh-VM pairs with the full private-cache fixture
reported 60 to 60 active objects on the unpatched kernel and 60 to 59 on
the patched kernel. A separate ordinary-callback regression test passed
on both kernels.

The simplified reproducer below removes that separate regression
machinery. One additional fresh control/treatment pair with this exact
41-line source confirmed the same 60 to 60 versus 60 to 59 split. These
counts reflect the slab layout in the tested configuration.

Save the source as rcu_barrier_sheaf_repro.c and create a Makefile
containing:

  obj-m := rcu_barrier_sheaf_repro.o

Build it with:

  make -C /lib/modules/$(uname -r)/build M="$PWD" modules

Then, as root on a disposable test kernel:

  insmod rcu_barrier_sheaf_repro.ko
  awk '$1 == "rcu_barrier_sheaf_repro" { print $2 }' /proc/slabinfo
  cat /sys/kernel/slab/rcu_barrier_sheaf_repro/sheaf_capacity
  echo 1 > /sys/module/rcutree/parameters/do_rcu_barrier
  awk '$1 == "rcu_barrier_sheaf_repro" { print $2 }' /proc/slabinfo
  rmmod rcu_barrier_sheaf_repro

The first and second slabinfo readings are 60 and 60 without the patch,
and 60 and 59 with it.  kmem_cache_destroy() performs per-cache
deferred-free cleanup when the module is removed, after the measurement.

// SPDX-License-Identifier: GPL-2.0
#include <linux/init.h>
#include <linux/module.h>
#include <linux/rcupdate.h>
#include <linux/slab.h>

struct repro_object {
	struct rcu_head rcu;
	unsigned long payload;
};

static struct kmem_cache *repro_cache;

static int __init rcu_barrier_sheaf_repro_init(void)
{
	struct repro_object *object;

	repro_cache = kmem_cache_create("rcu_barrier_sheaf_repro",
					sizeof(*object), 0, SLAB_NO_MERGE, NULL);
	if (!repro_cache)
		return -ENOMEM;

	object = kmem_cache_alloc(repro_cache, GFP_KERNEL);
	if (!object) {
		kmem_cache_destroy(repro_cache);
		return -ENOMEM;
	}

	kfree_rcu(object, rcu);
	return 0;
}

static void __exit rcu_barrier_sheaf_repro_exit(void)
{
	kmem_cache_destroy(repro_cache);
}

module_init(rcu_barrier_sheaf_repro_init);
module_exit(rcu_barrier_sheaf_repro_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Reproduce incomplete rcutree.do_rcu_barrier drains");

---
Changes since v1:
- Add the motivating bcachefs failure and the successful unmodified
  workload result to both the cover letter and commit message.
- Drop the incorrect sheaf Fixes: tag and regression framing; describe
  this as a strengthening of the existing test interface.
- Credit the agreed 2024 proposal for this extension.
- Broaden the subject and changelog from sheaves to kvfree_rcu work.
- Hard-wrap the prose for text-based mail readers.

The code diff is unchanged from v1. The results above are the existing
validation results; no new kernel tests were run for this prose revision.

v1:
https://lore.kernel.org/all/20260910101112.1648978-1-matthias.goergens@xxxxxxxxx/

Matthias Goergens (1):
  rcu: make userspace barrier hook drain kvfree_rcu work

 .../admin-guide/kernel-parameters.txt         |  7 ++---
 kernel/rcu/tree.c                             | 27 ++++++++++++-------
 2 files changed, 21 insertions(+), 13 deletions(-)


base-commit: 50d05c7c76c96b90462f24debacca971d2e86713
-- 
2.55.0