PCI: hotplug: shpchp: use-after-free races in the controller release path (objects destroyed before the event sources are stopped; self-rearming poll timer deleted with timer_delete)

[徐晗翔]

shpchp_release_ctlr() tears down the objects the event handlers need
(the per-slot structures and workqueues, via cleanup_slots()) BEFORE
it deactivates the event sources (the poll timer in poll mode, the
IRQ otherwise), and it deletes the poll timer with timer_delete() --
which is neither synchronous nor does it prevent rearming, while the
timer's callback unconditionally re-arms itself at its tail.  A poll
callback that is past its INTR_LOC check (or that re-arms after the
delete) then runs on freed and unmapped memory.  I verified both
shapes on a pristine Linux 7.3 KASAN kernel with QEMU's native SHPC
controller (q35 + pcie-pci-bridge, poll mode): with the callback held
at a controlled point, unbind produced a KASAN slab-use-after-free
read of the freed controller (shpc_isr), then a page fault reading
the iounmap()ed registers, ending in "Fatal exception in interrupt";
and with the callback's tail add_timer() held instead, the freed
timer was re-armed after kfree(ctrl) and went on firing -- 27
use-after-free reports from the add_timer family, and the resurrected
timer kept entering shpc_isr on the freed controller afterwards.  The
sibling hotplug drivers already do this in the right order (pciehp
stops notification before cleanup_slot(); cpci stops its thread
synchronously before iounmap), so this is an shpchp-local defect.
Triggering needs root (driver unbind / slot sysfs writes); the fix is
a small reordering plus timer_shutdown_sync().

Three stacked root causes (drivers/pci/hotplug/, 7.3 line numbers):

1. Wrong order.  shpchp_release_ctlr() (shpchp_hpc.c:537-575) masks
   per-slot event interrupts, then runs cleanup_slots() -- list_del +
   destroy_workqueue + kfree of every slot (shpchp_core.c:126-141) --
   and only afterwards deletes the poll timer (timer_delete) or
   free_irq()s, then iounmap(ctrl->creg) and finally shpc_remove()
   kfrees the controller.  The per-slot mask only stops NEW events
   from latching into INTR_LOC; a callback that already read INTR_LOC
   non-zero is not waited for by anything.  The probe-failure path
   (shpchp_core.c:283-291) runs cleanup_slots() before any event
   source is stopped at all.

2. Wrong sync strength.  int_poll_timeout() re-arms itself at its
   tail through start_int_poll_timer() -> add_timer()
   (shpchp_hpc.c:212-236) -- even the idle path does.  The timer
   documentation for timer_delete() says explicitly that it "does not
   take into account whether the timer's callback function is
   concurrently executed" and "neither prevents rearming"; for a
   self-rearming timer the return value is meaningless.  The teardown
   primitive for this shape is timer_shutdown_sync().

3. Unlocked shared state.  The event chain (shpc_isr ->
   shpchp_handle_presence_change -> shpchp_find_slot()'s unlocked
   list_for_each_entry over ctrl->slot_list -> queue_work() on the
   per-slot workqueue; shpchp_ctrl.c:27-43,106-137; shpchp.h:197-208)
   has no lock against the concurrent teardown; callers don't even
   check find_slot()'s NULL return before dereferencing
   (shpchp_ctrl.c:114-116).

The detailed description:

* Test environment / affected version
  Linux version 7.3.0 (gcc 13.3.0, GNU ld 2.42) #3 SMP PREEMPT_DYNAMIC
  -- vanilla mainline 7.3 tarball + CONFIG_HOTPLUG_PCI_SHPC=y + KASAN,
  QEMU/KVM x86_64 (8 vCPU, q35 + -device pcie-pci-bridge: a native
  SHPC controller, 1b36:000e, 31 slots), kernel command line
  shpchp.shpchp_poll_mode=1 shpchp.shpchp_poll_time=1.  The
  instrument (attached) is a loadable kprobe module; the kernel
  source is unmodified.  The code shape is long-lived (timer_delete
  is the old del_timer renamed).

* Failure forms and their windows (stated honestly)
  - Form 0 (default interrupt mode): an ISR that already read
    INTR_LOC non-zero while release's cleanup_slots() proceeds --
    unlocked list traversal of freed slots, queue_work() onto a
    destroyed workqueue.  Microsecond window, needs a real slot event
    concurrent with unbind.
  - Form 1 (poll mode): same, but the event may have latched up to
    one poll period earlier.  Additionally the callback's read-modify-
    write of SLOT_REG (shpchp_hpc.c:794 read, :815 write-back) races
    release's mask writes and can undo the per-slot mask, reopening
    the event path.  A userspace trigger exists: writing a slot sysfs
    attribute (e.g. .../attention) raises Command Complete, which the
    next poll picks up -- no physical event needed.
  - Form 2: the same running callback keeps issuing shpc_readl/
    shpc_writel after iounmap(ctrl->creg) -- page fault on the
    unmapped MMIO range.
  - Form 3 (no event needed, deterministic under the window): the
    callback tail's add_timer() re-arms a timer embedded in memory
    that release has already kfree()d; timer_delete() cannot prevent
    this.  The resurrected timer fires within one poll period and
    unconditionally dereferences the freed controller and its
    unmapped creg.
  - Side note (same root, not a race): with the driver probed in poll
    mode, writing 0 to the 0644 module parameter shpchp_poll_mode
    makes release take the free_irq() branch -- the poll timer is
    then never deleted at all, and the UAF after kfree(ctrl) is
    guaranteed rather than raced.

* Evidence (pristine kernel, kprobe-controlled windows; instrument
  attached)
  Form 1+2 (callback held 5 s at int_poll_timeout entry; attention
  write + unbind inside the window):
    BUG: KASAN: slab-use-after-free in shpc_isr+0x24/0x240
    Read of size 8 ... by task swapper/0        (timer softirq)
      shpc_isr <- int_poll_timeout <- call_timer_fn <- __run_timers
    Allocated by task 1: shpc_probe <- local_pci_probe <- ...
    Freed by task (userspace unbind): kfree <- pci_device_remove
                      <- device_release_driver_internal <- unbind_store
    object: kmalloc-192, read at ctrl->creg offset (+136)
    then: BUG: unable to handle page fault for address: ffa0000000025018
          (readl of the iounmap()ed creg)
    then: Kernel panic - not syncing: Fatal exception in interrupt
  Form 3 (the callback tail's add_timer() held; unbind completes
  kfree(ctrl) inside the window; stall released):
    27 slab-use-after-free reports: add_timer+0x11/+0x23,
    lock_timer_base, __mod_timer, detach_if_pending, enqueue_timer,
    __run_timers -- the re-arm of the freed timer -- followed by
    shpc_isr+0x24 use-after-free entries as the resurrected timer
    keeps firing on the freed controller.
  Natural (no instrumentation) run: 2000 bind/unbind rounds with the
  attention event stream -- zero hits.  This matches the window
  analysis: form 3's natural window is the microsecond "callback
  popped but not yet at add_timer()" interval meeting the one
  timer_delete() per round, ~1e-5 per round; sustained hammering or
  scheduling jitter accumulates it, and the controlled runs above are
  the deterministic amplification of exactly that interleaving.

* Instrument usage (attached shpchp_stall.c, external module)
  Two modes: mode=1 parks the poll callback at int_poll_timeout entry
  (form 1+2 choreography: during the park, write a slot attribute and
  unbind); mode=2 parks the tail add_timer() call -- the entry probe
  records the timer pointer (rdi) and the add_timer probe matches its
  first argument, because start_int_poll_timer() is inlined and
  add_timer is a tail call (no usable return address).  Environment
  notes for reproduction: QEMU needs q35 + -device pcie-pci-bridge
  (the i440fx pci-bridge is rejected by the ACPI glue); the parameter
  is shpchp_poll_mode (not poll_mode -- the wrong name fails
  silently and the driver runs in interrupt mode); pin the orchestration
  shell off CPU0 so the parked softirq does not starve it.



Regards,
Hanxiang Xu
xuhx@xxxxxxxxxxx


// SPDX-License-Identifier: GPL-2.0
/*
 * shpchp_stall.c - controlled-window instrument for the shpchp release-path
 * race (drivers/pci hotplug, audit Case A).
 *
 * Defect under test: shpchp_release_ctlr() destroys the objects the poll
 * timer callback depends on BEFORE the event source is quiesced:
 *
 *   shpchp_release_ctlr()                  shpchp_hpc.c
 *     1. mask per-slot event interrupts    (stops NEW events only)
 *     2. cleanup_slots(ctrl)               destroy_workqueue + kfree(slot)
 *     3. mask global interrupt
 *     4. timer_delete(&ctrl->poll_timer)   NON-sync, does not stop rearm
 *     5. iounmap(ctrl->creg)
 *   shpc_remove(): kfree(ctrl)
 *
 * while the poll callback may still be running (timer softirq):
 *
 *   int_poll_timeout() -> shpc_isr(0, ctrl)
 *     -> shpc_readl(ctrl, INTR_LOC)        [ctrl->creg + readl MMIO]
 *     -> ... handle_* / queue_work(p_slot->wq) ...
 *     -> start_int_poll_timer() -> add_timer(&ctrl->poll_timer)  [rearm]
 *
 * This module stalls the callback at int_poll_timeout() entry (busy-wait in
 * a kprobe pre_handler on the static function, resolved by name). With the
 * stall >= poll period, a userspace `echo <bdf> > .../shpchp/unbind` on
 * another CPU lands inside the stalled window with near certainty, and the
 * callback resumes on freed ctrl/unmapped creg:
 *
 *   - KASAN slab-use-after-free Read   (ctrl->creg field of freed ctrl)
 *     [report form 1 object family]
 *   - kernel page fault on readl() of the unmapped creg (softirq oops)
 *     [report form 2]
 * If the INTR_LOC read had returned before iounmap (natural timing), the
 * unconditional trailing start_int_poll_timer()/add_timer() on the freed
 * ctrl is report form 3's write vector.
 *
 * Kernel sources are NOT modified: this is the same controlled-window
 * methodology as r4_stall.c (kprobe pre_handler + udelay), applied to a
 * static function via kallsyms.
 */
#define pr_fmt(fmt) "SHPCSTALL: " fmt

#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/kprobes.h>
#include <linux/delay.h>
#include <linux/atomic.h>
#include <linux/jiffies.h>
#include <linux/clocksource.h>

static int stall_us = 1200000;
module_param(stall_us, int, 0600);
MODULE_PARM_DESC(stall_us, "busy-wait per callback hit in us (>= poll period makes the stall cover unbind)");

static int hits_max = 8;
module_param(hits_max, int, 0600);
MODULE_PARM_DESC(hits_max, "maximum number of stalled callbacks (0 = unbounded)");

static int mode = 1;
module_param(mode, int, 0600);
MODULE_PARM_DESC(mode, "1=stall callback entry (forms 1+2), 2=stall trailing add_timer from int_poll_timeout (form 3)");

static atomic_t hits = ATOMIC_INIT(0);

static unsigned long last_t;
static unsigned long last_entry_jiffies;

static int stall_pre(struct kprobe *p, struct pt_regs *regs)
{
	int h;

	if (mode == 2) {
		/* record the timer pointer for the add_timer() filter */
		last_t = regs->di;
		return 0;
	}
	if (stall_us <= 0)
		return 0;
	h = atomic_inc_return(&hits);
	if (hits_max > 0 && h > hits_max)
		return 0;
	/* x86-64: first argument (struct timer_list *t) in RDI */
	last_t = regs->di;
	last_entry_jiffies = jiffies;
	if (h <= 4)
		pr_warn("stall #%d: %d us at int_poll_timeout entry t=%px jiffies=%lu\n",
			h, stall_us, (void *)last_t, last_entry_jiffies);
	{
		/* udelay() returned immediately in kprobe context on this
		 * build; busy-wait on the NMI-safe lock-free mono clock. */
		u64 target = ktime_get_mono_fast_ns() + (u64)stall_us * 1000;

		while (ktime_get_mono_fast_ns() < target)
			cpu_relax();
	}
	return 0;
}

static void stall_post(struct kprobe *p, struct pt_regs *regs, unsigned long flags)
{
	if (mode != 1)
		return;
	pr_warn("stall resumed: t=%px elapsed=%lu ms, callback body follows\n",
		(void *)last_t,
		(unsigned long)jiffies_to_msecs(jiffies - last_entry_jiffies));
}

static int isr_pre(struct kprobe *p, struct pt_regs *regs)
{
	/* shpc_isr(irq, dev_id): dev_id = RSI */
	pr_warn("isr entered: irq=%ld ctrl=%px\n", (long)regs->di, (void *)regs->si);
	return 0;
}

static struct kprobe kp_isr = {
	.symbol_name = "shpc_isr",
	.pre_handler = isr_pre,
};

static struct kprobe kp = {
	.symbol_name = "int_poll_timeout",
	.pre_handler = stall_pre,
	.post_handler = stall_post,
};

/*
 * mode 2: stall the REARM path. start_int_poll_timer() is inlined into
 * int_poll_timeout() (it writes ctrl->poll_timer.expires at t+0x10 and calls
 * add_timer()). Stall add_timer() calls whose return address lies inside
 * int_poll_timeout: the callback has already read INTR_LOC while objects were
 * still alive; after the stall the timer gets re-armed on freed memory.
 */
static atomic_t hits_at = ATOMIC_INIT(0);

static int at_pre(struct kprobe *p, struct pt_regs *regs)
{
	u64 target;
	int h;

	if (mode != 2 || stall_us <= 0)
		return 0;
	/* tail call: add_timer() is reached via jmp from int_poll_timeout(),
	 * so filter by the timer pointer recorded on callback entry. */
	if (regs->di != last_t)
		return 0;
	h = atomic_inc_return(&hits_at);
	if (hits_max > 0 && h > hits_max)
		return 0;
	pr_warn("stall-addtimer #%d: timer=%px (== poll_timer from callback)\n",
		h, (void *)regs->di);
	target = ktime_get_mono_fast_ns() + (u64)stall_us * 1000;
	while (ktime_get_mono_fast_ns() < target)
		cpu_relax();
	/* module code IS kasan-instrumented: reading timer->flags on freed
	 * memory must produce a report, proving the post-stall state */
	{
		struct timer_list *t = (struct timer_list *)regs->di;

		pr_warn("stall-addtimer #%d resumed: timer=%px flags=%08x\n",
			h, t, t->flags);
	}
	return 0;
}

static struct kprobe kp_at = {
	.symbol_name = "add_timer",
	.pre_handler = at_pre,
};

/* observe the release path itself */
static int rc_pre(struct kprobe *p, struct pt_regs *regs)
{
	pr_warn("release_ctlr entered: ctrl=%px\n", (void *)regs->di);
	return 0;
}

static struct kprobe kp_rc = {
	.symbol_name = "shpchp_release_ctlr",
	.pre_handler = rc_pre,
};

static int __init shpchp_stall_init(void)
{
	int ret;

	ret = register_kprobe(&kp);
	if (ret < 0) {
		pr_err("register_kprobe(int_poll_timeout) failed: %d\n", ret);
		return ret;
	}
	ret = register_kprobe(&kp_isr);
	if (ret < 0) {
		unregister_kprobe(&kp);
		pr_err("register_kprobe(shpc_isr) failed: %d\n", ret);
		return ret;
	}
	if (mode == 2) {
		ret = register_kprobe(&kp_at);
		if (ret < 0) {
			pr_err("register_kprobe(add_timer) failed: %d\n", ret);
			return ret;
		}
		ret = register_kprobe(&kp_rc);
		if (ret < 0)
			pr_err("register_kprobe(release_ctlr) failed: %d\n", ret);
	}
	pr_info("mode=%d armed\n", mode);
	pr_info("armed at %pS stall_us=%d hits_max=%d\n",
		kp.addr, stall_us, hits_max);
	return 0;
}

static void __exit shpchp_stall_exit(void)
{
	if (mode == 2) {
		unregister_kprobe(&kp_rc);
		unregister_kprobe(&kp_at);
	}
	unregister_kprobe(&kp_isr);
	unregister_kprobe(&kp);
	pr_info("disarmed, hits=%d\n", atomic_read(&hits));
}
module_init(shpchp_stall_init);
module_exit(shpchp_stall_exit);

MODULE_AUTHOR("race verification");
MODULE_DESCRIPTION("Controlled-window instrument: shpchp poll-timer vs release-path race");
MODULE_LICENSE("GPL");