[PATCH v2 0/2] arm64: sdei: Handle exceptions taken inside the event handler
From: Kiryl Shutsemau
Date: Thu Nov 05 2026 - 07:42:47 EST
From: "Kiryl Shutsemau (Meta)" <kas@xxxxxxxxxx>
An SDEI event can land anywhere, and a synchronous exception taken by its
handler breaks the context it interrupted:
- With KPTI, an event that lands in user space runs the handler on the
trampoline vectors, whose current-EL half is empty. The exception
loops there and the CPU is lost.
- The exception overwrites ELR, SPSR, ESR and FAR. If the event landed
in exception entry or return, the interrupted context runs on the
handler's values: a task gets SIGILL, or the kernel oopses on a fault
it never took.
Patch 1 runs the handler on the kernel vectors. Patch 2 saves and
restores the exception registers around it.
Handlers take such exceptions in normal operation now that SDEI delivers
NMI backtraces: show_regs() reads through _nofault() accessors that
fault, and a kprobe or WARN anywhere in the path takes a BRK.
Tested under QEMU TCG with TF-A, 4 CPUs, 20 x sysrq-l under load, with a
test-only copy_from_kernel_nofault() of address 0x10 in
sdei_nmi_handler():
- v7.4-rc1, KPTI on: CPUs stuck in the trampoline vectors, RCU stalls.
- v7.4-rc1, KPTI off: 60 "unsafe: exception during handler" warnings,
SIGILL and SIGSEGV in user tasks, and an oops in execve().
- With the series, KPTI on and off: no warnings, no stalls. The same
with an eprobe on irq_disable that reads address 0x10, which faults
inside irqentry_nmi_enter() when the event lands in user space.
An earlier revision of patch 2 was also run on arm64 hardware with
firmware SDEI, where a plain sysrq-l was enough to trigger the warning
without it.
Changes since v1
================
https://lore.kernel.org/all/20261104180936.2981070-1-kas@xxxxxxxxxx/
- Switch VBAR and save the exception registers in __sdei_handler(),
before irqentry_nmi_enter(), and restore them after
irqentry_nmi_exit(). Both can take an exception too: the eprobe above
wedged v1 with KPTI on (Usama). do_sdei_event() takes the interrupted
VBAR for the IRQ-like resume.
Kiryl Shutsemau (Meta) (2):
arm64: sdei: Run the event handler on the kernel vectors
arm64: sdei: Preserve the interrupted context's exception registers
arch/arm64/include/asm/sdei.h | 3 +-
arch/arm64/kernel/entry-common.c | 40 +++++++++++++++++++++-
arch/arm64/kernel/sdei.c | 59 ++++++++++++++++++++++++++------
3 files changed, 89 insertions(+), 13 deletions(-)
Range-diff against v1:
1: f3e1e7fd09f7 ! 1: 93ef4a287a4c arm64: sdei: Run the event handler on the kernel vectors
@@ Commit message
path does it: a WARN, a kprobe hit, or a fault fixed up by a _nofault()
accessor.
- Install the kernel vectors in do_sdei_event() when VBAR_EL1 holds the
- trampoline vectors, and restore it before returning to firmware,
- including when the handler fails. The IRQ-like resume into user space
- keeps using the interrupted value, as it must go through the trampoline
- to map the kernel.
+ Install the kernel vectors in __sdei_handler() when VBAR_EL1 holds the
+ trampoline vectors, and restore it before returning to firmware. The
+ switch covers irqentry_nmi_enter() and irqentry_nmi_exit() too, as Usama
+ Arif pointed out: their tracepoints and context tracking WARNs can take
+ an exception as well.
+
+ The IRQ-like resume into user space must still go through the
+ trampoline to map the kernel. do_sdei_event() takes the interrupted
+ VBAR_EL1 and keeps basing the resume address on it.
Fixes: 79e9aa59dc29 ("arm64: sdei: Add trampoline code for remapping the kernel")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Kiryl Shutsemau (Meta) <kas@xxxxxxxxxx>
- ## arch/arm64/kernel/sdei.c ##
+ ## arch/arm64/include/asm/sdei.h ##
+@@ arch/arm64/include/asm/sdei.h: asmlinkage unsigned long __sdei_handler(struct pt_regs *regs,
+ struct sdei_registered_event *arg);
+
+ unsigned long do_sdei_event(struct pt_regs *regs,
+- struct sdei_registered_event *arg);
++ struct sdei_registered_event *arg,
++ unsigned long vbar);
+
+ unsigned long sdei_arch_get_entry_point(int conduit);
+ #define sdei_arch_get_entry_point(x) sdei_arch_get_entry_point(x)
+
+ ## arch/arm64/kernel/entry-common.c ##
@@
- #include <asm/sections.h>
#include <asm/stacktrace.h>
#include <asm/sysreg.h>
+ #include <asm/system_misc.h>
+#include <asm/vectors.h>
- #include <asm/vmap_stack.h>
- unsigned long sdei_exit_mode;
-@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- if (arm64_kernel_unmapped_at_el0())
- clobbered_registers++;
+ /*
+ * Where the context being returned to had an active percpu GPR critical
+@@ arch/arm64/kernel/entry-common.c: asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
+ asmlinkage noinstr unsigned long
+ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ {
++ unsigned long vbar;
+ irqentry_state_t state;
+ unsigned long ret;
+
+@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ else if (cpu_has_pan())
+ set_pstate_pan(0);
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
-+ * handle exceptions from EL0: an exception taken by the handler would
-+ * run their empty current-EL half and never come back. Run the handler
-+ * on the kernel vectors. The IRQ-like resume below keeps using the
-+ * interrupted VBAR, so it still goes through the trampoline.
++ * handle exceptions from EL0: an exception taken here would run their
++ * empty current-EL half and never come back. Switch to the kernel
++ * vectors. The IRQ-like resume keeps using the interrupted VBAR, so it
++ * still goes through the trampoline.
+ */
++ vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0() &&
-+ vbar == (unsigned long)this_cpu_read(this_cpu_vector)) {
++ vbar == (unsigned long)__this_cpu_read(this_cpu_vector)) {
+ write_sysreg((unsigned long)vectors, vbar_el1);
+ isb();
+ }
+
- /* Retrieve the missing registers values */
- for (i = 0; i < clobbered_registers; i++) {
- /* from within the handler, this call always succeeds */
-@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- }
-
- err = sdei_event_handler(regs, arg);
-- if (err)
-- return SDEI_EV_FAILED;
-
- if (elr != read_sysreg(elr_el1)) {
- /*
-@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
- pr_warn("unsafe: exception during handler\n");
- }
+ state = irqentry_nmi_enter(regs);
+- ret = do_sdei_event(regs, arg);
++ ret = do_sdei_event(regs, arg, vbar);
+ irqentry_nmi_exit(regs, state);
+ write_sysreg(vbar, vbar_el1);
+
-+ if (err)
-+ return SDEI_EV_FAILED;
-+
- mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
+ return ret;
+ }
+ #endif /* CONFIG_ARM_SDE_INTERFACE */
+
+ ## arch/arm64/kernel/sdei.c ##
+@@ arch/arm64/kernel/sdei.c: unsigned long sdei_arch_get_entry_point(int conduit)
+ * SDEI_EV_HANDLED - success, return to the interrupted context.
+ * SDEI_EV_FAILED - failure, return this error code to firmware.
+ * virtual-address - success, return to this address.
++ *
++ * vbar is the interrupted context's VBAR_EL1, for the IRQ-like resume.
+ */
+ unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
+- struct sdei_registered_event *arg)
++ struct sdei_registered_event *arg,
++ unsigned long vbar)
+ {
+ u32 mode;
+ int i, err = 0;
+ int clobbered_registers = 4;
+ u64 elr = read_sysreg(elr_el1);
+ u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
+- unsigned long vbar = read_sysreg(vbar_el1);
- /*
+ if (arm64_kernel_unmapped_at_el0())
+ clobbered_registers++;
2: ba422c8220e5 ! 2: 07d3078373ee arm64: sdei: Preserve the interrupted context's exception registers
@@ Commit message
kworker that is not running a work item, and a single sysrq-l warned
once per such kworker.
- Save ELR, SPSR, ESR and FAR on entry to do_sdei_event() and restore them
- before returning to firmware.
+ Save ELR, SPSR, ESR and FAR in __sdei_handler() before
+ irqentry_nmi_enter(), which can take an exception too, and restore them
+ after irqentry_nmi_exit().
Once they are restored, a nested exception is harmless if its handler is
safe in NMI context. That holds for EL1 data aborts on translation,
@@ Commit message
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Kiryl Shutsemau (Meta) <kas@xxxxxxxxxx>
+ ## arch/arm64/kernel/entry-common.c ##
+@@ arch/arm64/kernel/entry-common.c: asmlinkage void noinstr __noreturn handle_bad_stack(struct pt_regs *regs)
+ asmlinkage noinstr unsigned long
+ __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ {
+- unsigned long vbar;
++ unsigned long elr, spsr, esr, far, vbar;
+ irqentry_state_t state;
+ unsigned long ret;
+
+@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ else if (cpu_has_pan())
+ set_pstate_pan(0);
+
++ /*
++ * Nothing saved the interrupted context's exception registers on the
++ * way in, and it may be in the middle of exception entry or return
++ * (SDEI ignores DAIF). Save them before anything can take an exception
++ * and overwrite them.
++ */
++ elr = read_sysreg(elr_el1);
++ spsr = read_sysreg(spsr_el1);
++ esr = read_sysreg(esr_el1);
++ far = read_sysreg(far_el1);
++
+ /*
+ * With KPTI, VBAR_EL1 holds the trampoline vectors from the return to
+ * EL0 until the next exception from EL0 maps the kernel. They only
+@@ arch/arm64/kernel/entry-common.c: __sdei_handler(struct pt_regs *regs, struct sdei_registered_event *arg)
+ ret = do_sdei_event(regs, arg, vbar);
+ irqentry_nmi_exit(regs, state);
+
++ /*
++ * Put them back unconditionally. An unchanged ELR does not rule out a
++ * nested exception: one may have returned to the same address.
++ */
++ write_sysreg(elr, elr_el1);
++ write_sysreg(spsr, spsr_el1);
++ write_sysreg(esr, esr_el1);
++ write_sysreg(far, far_el1);
+ write_sysreg(vbar, vbar_el1);
+
+ return ret;
+
## arch/arm64/kernel/sdei.c ##
@@ arch/arm64/kernel/sdei.c: unsigned long sdei_arch_get_entry_point(int conduit)
return 0;
}
+/*
-+ * Exceptions an SDEI handler may take without harm once do_sdei_event() has
++ * Exceptions an SDEI handler may take without harm once __sdei_handler() has
+ * restored the interrupted context's exception registers. Their handlers are
+ * safe in NMI context:
+ * - EL1 data aborts on translation, permission and access-flag faults:
@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *
int i, err = 0;
int clobbered_registers = 4;
u64 elr = read_sysreg(elr_el1);
-+ u64 spsr = read_sysreg(spsr_el1);
-+ u64 esr = read_sysreg(esr_el1);
-+ u64 far = read_sysreg(far_el1);
+ u64 nested_elr, nested_esr, nested_far;
u32 kernel_mode = read_sysreg(CurrentEL) | 1; /* +SPSel */
- unsigned long vbar = read_sysreg(vbar_el1);
+ if (arm64_kernel_unmapped_at_el0())
@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *regs,
+ }
err = sdei_event_handler(regs, arg);
-
-- if (elr != read_sysreg(elr_el1)) {
-- /*
-- * We took a synchronous exception from the SDEI handler.
-- * This could deadlock, and if you interrupt KVM it will
-- * hyp-panic instead.
-- */
-- pr_warn("unsafe: exception during handler\n");
++
+ /*
-+ * SDEI is not an architectural exception: nothing saved the
-+ * interrupted context's exception registers on the way in, and it may
-+ * be in the middle of exception entry or return (SDEI ignores DAIF).
-+ * Any synchronous exception the handler took has overwritten them.
-+ * Only the last one's syndrome survives; report it if it is not one
-+ * known to be harmless.
++ * A synchronous exception taken by the handler has overwritten the
++ * exception registers; __sdei_handler() puts them back. Only the last
++ * one's syndrome survives; report it if it is not one known to be
++ * harmless.
+ *
+ * If the event interrupted a KVM world switch, VBAR points at the KVM
+ * hyp vectors and any such exception is a hyp panic instead.
@@ arch/arm64/kernel/sdei.c: unsigned long __kprobes do_sdei_event(struct pt_regs *
+ if (nested_elr != elr && !sdei_nested_exception_ok(nested_esr)) {
+ pr_warn_once("unexpected exception during handler: ESR %#llx FAR %#llx near %pS\n",
+ nested_esr, nested_far, (void *)nested_elr);
- }
-
-+ /*
-+ * Put them back unconditionally. An unchanged ELR does not rule out a
-+ * nested exception: one may have returned to the same address.
-+ */
-+ write_sysreg(elr, elr_el1);
-+ write_sysreg(spsr, spsr_el1);
-+ write_sysreg(esr, esr_el1);
-+ write_sysreg(far, far_el1);
- write_sysreg(vbar, vbar_el1);
-
++ }
++
if (err)
+ return SDEI_EV_FAILED;
+
+- if (elr != read_sysreg(elr_el1)) {
+- /*
+- * We took a synchronous exception from the SDEI handler.
+- * This could deadlock, and if you interrupt KVM it will
+- * hyp-panic instead.
+- */
+- pr_warn("unsafe: exception during handler\n");
+- }
+-
+ mode = regs->pstate & (PSR_MODE32_BIT | PSR_MODE_MASK);
+
+ /*
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
--
2.54.0