[RFC PATCH 0/7] queued static-key API reduces IPIs to 134/16154 in dyndbg

[Jim Cromie]

This patchset formalizes and exposes the internal
architecture-specific arch_jump_label_transform_queue API as a public
kernel interface (static_key_*_queued).

Currently, dynamic_debug toggles static keys individually, which
triggers a machine-wide synchronization (IPI) for every callsite.
This causes $O(N)$ overhead.

By using the new queued API, (which also adds an insert-sort to the
queue) dynamic_debug can now toggle up to 256 sites (x86 queue length)
with a single IPI.

Without the sort, dyndbg saw a far more modest IPI reduction; ~6k/16k.
Ordered descriptors does not insure ordered patch-addresses.
Currently, only x86 has the sort.

I tested on virtme-ng, using:

  echo +p > /proc/dynamic_debug/control
  echo -p > /proc/dynamic_debug/control

With this patch and test, IPIs reduced to 134 from 16154.

Patchset also changes virtio.c to use pr_debug_ratelimited() instead
of pr_debug(), which otherwise flooded my test-setup, obscuring
results.

serial_core.c also gets 1 _ratelimited() change, though I didnt see
them in my setup, and likely missed a few conversion candidates.

Patchset also includes a new query-cmd syntax for dyndbg:

  echo 'module !virtio* +p' > /proc/dynamic_debug/control
  echo -p > /proc/dynamic_debug/control

It further reduced the console output, so might have sufficient
utility for eventual inclusion, despite lacking and/or logic.

The patch hoisting the static_key_apply_queued() out of
ddebug_change() up to ddebug_exec_queries() does nothing in this case,
and would only affect multi-queries:

  echo 'module !virtio* +p ; module serial -p' > /proc/dynamic_debug/control

ISTM such queries are currently rare, but could be leveraged in
classmap-params, to optimize drm.debug=0x1ff, which currently would
get one query-command per bit (12+ IPIs).

DRM is the biggest potential beneficiary of this:

root@drm-misc-fixes-2026-02-26-78-g535e886b182f:/home/jimc/projects/lx/wk-D/b0-dd-drm-all# modprobe i915
[   20.405557] dyndbg:  25 debug prints in module i2c_core
[   20.459373] dyndbg: 340 debug prints in module drm
[   20.459851] ACPI: bus type drm_connector registered
[   20.471366] dyndbg:  89 debug prints in module drm_kms_helper
[   20.482336] dyndbg: 155 debug prints in module drm_display_helper
[   20.496153] dyndbg:   2 debug prints in module ttm
[   21.136619] dyndbg: 1801 debug prints in module i915
root@drm-misc-fixes-2026-02-26-78-g535e886b182f:/home/jimc/projects/lx/wk-D/b0-dd-drm-all# modprobe amdgpu
[   32.907485] dyndbg: 4532 debug prints in module amdgpu


Jim Cromie (7):
  jump_label: expose queueing API for batched static key updates
  virtio: use pr_debug_ratelimited to avoid flooding
  drivers/tty/serial/serial_core: ratelimit uart_wait_until_sent
  dyndbg: use static-key queueing API in dynamic-debug
  dyndbg: hoist static_key_apply_queued up
  lib/dynamic_debug: add negation support to queries
  dyndbg-test: test keyword !value negation

 arch/Kconfig                                  |   3 +
 arch/x86/Kconfig                              |   1 +
 arch/x86/kernel/alternative.c                 |  50 ++++---
 arch/x86/kernel/jump_label.c                  |  13 +-
 drivers/tty/serial/serial_core.c              |   4 +-
 drivers/virtio/virtio_ring.c                  |  12 +-
 include/linux/jump_label.h                    |  24 ++++
 kernel/jump_label.c                           | 125 ++++++++++++++++--
 lib/dynamic_debug.c                           |  88 ++++++++----
 .../dynamic_debug/dyndbg_selftest.sh          |  35 +++++
 10 files changed, 290 insertions(+), 65 deletions(-)

-- 
2.53.0