[RFC PATCH v2 0/5] panic, crash_core, sys_info: Introduce crash_kexec_in_memory_sys_info

[Aaron Tomlin]

In modern fleet environments, capturing a full multi-gigabyte vmcore via
kdump can occasionally fail or be intentionally skipped due to storage,
network, or encryption constraints. In these scenarios, the extracted
dmesg buffer (e.g. vmcore-dmesg.txt) is often the sole artefact
available for post-mortem triage.

Currently, appending supplemental system state (such as memory usage, held
locks, active timers, task states, or ftrace buffers) via panic_print prior
to kdump requires enabling crash_kexec_post_notifiers. Doing so exposes the
crash path to registered panic notifiers (vastly increasing the risk of
hanging or deadlocking) and forces panic_print output to be flushed
synchronously to physical consoles. On a slow serial console or IPMI
Serial-over-LAN (e.g. 115200 baud), flushing comprehensive sys_info()
output (e.g. tasks or stacktraces) can exceed 10 seconds, allowing the
hardware timer to expire and reset the system before kdump can initialise.
Furthermore, on large systems, high-volume telemetry can wrap the printk
ring buffer and overwrite the initial Oops, BUG, or panic stack trace.

This series introduces the "crash_kexec_in_memory_sys_info" kernel
command-line parameter to decouple in-memory telemetry collection from
both panic notifiers and physical console flushing when kdump is active:

    1.  Fixes duplicate and dropped all-CPU backtraces when SYS_INFO_ALL_BT
        is configured in panic_print across early SMP shutdown and
        __sys_info().

    2.  Avoids spinning indefinitely on zone->lock in show_free_areas()
        during NMI, Oops, or panic contexts by falling back to non-blocking
        trylock acquisitions and skipping contended zones.

    3.  Avoids blocking on timekeeping seqcounts and hrtimer_cpu_base locks
        in sysrq_timer_list_show() and print_active_timers() during NMI,
        Oops, or panic contexts.

    4.  Adds printk console flush suppression across nbcon, legacy
        consoles, and klogd, alongside ring buffer tail pinning so
        pre-crash records can still be recycled while protecting the
        initial crash output from being overwritten by subsequent telemetry
        or ftrace dumps.

    5.  Introduces crash_kexec_in_memory_sys_info and invokes
        panic_in_memory_sys_info() from __crash_kexec() prior to
        machine_crash_shutdown() when crash_kexec_post_notifiers is not
        enabled. It also marks the start of crash diagnostics across
        bust_spinlocks(), __report_bug(), oops_enter(), and vpanic(), and
        reorders __sys_info() so compact, high-signal dumps (SYS_INFO_MEM,
        SYS_INFO_LOCKS, SYS_INFO_TIMERS, SYS_INFO_BLOCKED_TASKS) execute
        before high-volume dumps (SYS_INFO_ALL_BT, SYS_INFO_TASKS,
        SYS_INFO_FTRACE).

Changes since v1:

    - Split the single patch into a 5-patch series separating prerequisite
      panic all-CPU backtrace fixes, crash-time lock safety in show_mem()
      and timer_list, printk/tracing infrastructure, and the crash_kexec
      integration

    - Moved in-memory sys_info() collection from vpanic() into
      __crash_kexec() via panic_in_memory_sys_info() with local interrupts
      disabled so all crash_kexec() entry points (including direct Oops/die
      paths prior to vpanic()) are covered, and kept console flush
      suppression and tail pinning active through machine_crash_shutdown()
      and machine_kexec() unless machine_kexec() returns

    - Replaced unconditional ring buffer tail freezing with crash-start
      sequence pinning (printk_mark_panic_start(),
      printk_clear_panic_start(), and printk_is_tail_full()), allowing
      pre-crash records to be recycled while preserving the initial
      Oops/panic trace, finalising the last record when full, and stopping
      ftrace_dump_one() before consuming unprinted trace entries

    - Extended printk_suppress_console_flush() to cover nbcon atomic
      flushes and kthreads, legacy console_flush_one_record() and kthreads,
      and wake_up_klogd(), and dropped unnecessary EXPORT_SYMBOL_GPL
      exports

    - Added crash-context trylock and NMI-safe timekeeping handling in
      show_free_areas(), sysrq_timer_list_show(), and
      print_active_timers(), and fixed duplicate/dropped all-CPU backtraces
      when SYS_INFO_ALL_BT is used during panic

 - Link to v1: https://lore.kernel.org/lkml/20260916201546.661384-1-atomlin@xxxxxxxxxxx/

Aaron Tomlin (5):
  panic, sys_info: Fix duplicate and dropped all-CPU backtraces in panic
  mm/show_mem: Avoid blocking on zone->lock during Oops or panic
  timers: Avoid blocking on hrtimer_cpu_base lock during Oops or panic
  printk, tracing: Support console flush suppression and tail pinning
  panic, crash_core, sys_info: Introduce crash_kexec_in_memory_sys_info

 .../admin-guide/kernel-parameters.txt         |  11 ++
 include/linux/panic.h                         |   2 +
 include/linux/printk.h                        |  27 +++
 include/linux/sched.h                         |   3 +
 include/linux/sys_info.h                      |   4 +
 kernel/crash_core.c                           |   6 +
 kernel/exit.c                                 |  10 ++
 kernel/fork.c                                 |   3 +
 kernel/panic.c                                |  87 ++++++++-
 kernel/printk/internal.h                      |   4 +
 kernel/printk/nbcon.c                         |  11 +-
 kernel/printk/printk.c                        |  29 ++-
 kernel/printk/printk_ringbuffer.c             | 168 +++++++++++++++++-
 kernel/time/timer_list.c                      |  13 +-
 kernel/trace/trace.c                          |  20 ++-
 lib/bug.c                                     |  18 +-
 lib/bust_spinlocks.c                          |  22 ++-
 lib/sys_info.c                                |  33 ++--
 mm/show_mem.c                                 |  15 +-
 19 files changed, 447 insertions(+), 39 deletions(-)

-- 
2.55.0