[PATCH 0/4] srcu: NMI-safe atomic SRCU readers, and their irqsave forms

[Mathieu Desnoyers]

Hi Paul,

This series adapts atomic SRCU to the needs of RCU pseudo-transactions
[1], as we discussed [2].  Their readers resolve a proxy within a short
section, in any context, NMI included.  A writer waits for these
sections without sleeping, sometimes with interrupts disabled.  Atomic
SRCU fits, with two additions:

- Patch 1 makes the atomic readers NMI-safe.  They use the counter
  increments of __srcu_read_lock_nmisafe(), which keep the full barriers
  that synchronize_srcu_atomic() pairs with.
- Patch 2 adds srcu_read_lock_atomic_irqsave() and
  srcu_read_unlock_atomic_irqrestore().  They disable interrupts instead
  of preemption, for a domain whose writers wait with interrupts
  disabled.
- Patch 3 makes rcutorture run readers from NMI, for the NMI-safe reader
  flavors.
- Patch 4 makes rcutorture use the irqsave forms for half of the atomic
  readers, with a new atomic_irqsave parameter.  It also adds an SRCU-P
  run of this mode to torture.sh.

On architectures without NMI-safe this_cpu_inc(), every atomic reader
now pays an atomic increment at lock and at unlock.  The update side is
unchanged.  On architectures with NMI-safe this_cpu_inc(), such as x86,
the readers use the same counter increments as before.  David agreed to
the change, as long as synchronize_srcu_atomic() stays within the 4-16us
it was designed for [3].

Testing was on x86_64 under QEMU/KVM, with
CONFIG_FORCE_NEED_SRCU_NMI_SAFE=y:

- SRCU-P with atomic readers, half of them irqsave, for 5 minutes: no
  error.  It ran 32529 grace periods and 104114 readers in NMI.
- SRCU-P with the NMI-safe reader flavor, then with the fast one, for 2
  minutes each: no error.  Both ran readers in NMI.
- SRCU-T (Tiny SRCU) with atomic readers, half of them irqsave, for 2
  minutes: no error.

Each new check fails when what it checks is broken:

- A reader's counter update is split in two, with an NMI reader in
  between: the writer stalls in its grace period.
- The flavor check rejects atomic readers in NMI: it warns.
- An irqsave unlock restores interrupts while another irqsave reader of
  its CPU is held: the new check in rcutorture warns.

The readers in NMI need a PMU in the guest.  kvm.sh starts x86 guests
with -cpu kvm64, which has none.  So these runs added --qemu-args
"-cpu host,pmu=on".  Without it, rcutorture reports that NMIs went
untested.  This also holds for the existing test of ->call() from NMI.

This series was developed with assistance from an LLM.  I reviewed the
code and the tests.

The series is based on 8a36f53fd5ec of Linus' master (7.4 merge
window).

[1] https://lore.kernel.org/all/20261016133319.433617-1-mathieu.desnoyers@xxxxxxxxxxxx/
[2] https://lore.kernel.org/all/853d8469-94a4-4b33-a0d5-d6d2138fe3ef@paulmck-laptop/
[3] https://lore.kernel.org/all/84fdca577207092bd16fc8f3949c9b7a776750ba.camel@xxxxxxxxxxxxx/

Thanks,

Mathieu

Mathieu Desnoyers (4):
  srcu: Make atomic SRCU readers NMI-safe
  srcu: Add irqsave variants of the atomic SRCU readers
  rcutorture: Run NMI-safe SRCU readers from NMI
  rcutorture: Test the irqsave forms of atomic SRCU readers

 include/linux/srcu.h                          | 108 ++++++++++----
 kernel/rcu/rcutorture.c                       | 134 ++++++++++++++++--
 kernel/rcu/srcutree.c                         |   3 +-
 .../selftests/rcutorture/bin/torture.sh       |   6 +
 4 files changed, 216 insertions(+), 35 deletions(-)


base-commit: 8a36f53fd5ec3eff01f1b637d42cbdc3821d0493
-- 
2.47.3