Re: [RFC] x86: usermode IBT and signal handling

[Edgecombe, Rick P]

On Thu, 2026-10-08 at 22:02 +0000, Richard Patel wrote:
> On Wed, Oct 07, 2026 at 03:28:02PM +0000, Edgecombe, Rick P wrote:
> > On Wed, 2026-10-07 at 13:32 +0000, Richard Patel wrote:
> > > 2. Where to preserve WAIT_FOR_ENDBR across signal? Options:
> > >    - shstk signal frame
> > >    - signal frame fpstate (carve out a bit in _fpx_sw_bytes)
> > >    - uc_flags
> > 
> > The shadow stack signal frame is extensible to fit things like this and gives a
> > nice security protection for the bit. But I'd consider pursuing the simplest
> > option first. This option requires making IBT require shadow stack though. It
> > probably is the common case.
> 
> The problem is changing the shadow stack signal frame layout breaks a
> few userland apps that were already written to hardcode it:
> - libgcc's C++ exception unwinder runs into a parse failure if the frame
>   is extended (_URC_FATAL_PHASE2_ERROR)
>   https://github.com/gcc-mirror/gcc/blob/master/libgcc/config/i386/shadow-stack-unwind.h
> - CRIU (process migration) hand-builds such a shstk signal frame
>   https://github.com/checkpoint-restore/criu/blob/criu-dev/compel/arch/x86/src/lib/infect.c
> 
> It might help with logistics to allow apps to evolve shstk and IBT
> independently also.
> 
> > The original patches found a bit in the normal signal frame, but I don't think
> > it ever got fully settled? In any case it would need to be revisited at this
> > point.
> > 
> > But the best option is probably the one with the least resistance. It might be
> > shadow stack, since that is off to the side. We can always harden things with
> > further changes (add to shadow stack), or extend to IBT-only later.
> 
> The original patches used uc_flags but that didn't work with 32-bit
> sigreturn, but that's easy enough to just ban.
> 
> I just didn't like the cosmetics of uc_flags anyways, there's a more
> elegant way to use a reserved fpstate bit.

We'd have to think if it is worth taking from a limited resource of bits for
this if we have another place to put it. Especially since the other place is
more secure. Apps could clear the tracker bit during a signal.

But it depends on the ABI backwards compatibility problems you mentioned.

> 
> > > 3. OK to break legacy 32-bit sigreturn if IBT enabled?
> > 
> > IBT needs userspace enabling to work, and legacy 32 bit userspace is basically,
> > well, legacy. So unless there is any serious user that pops up, we should just
> > not supporting user IBT for 32 bit. This simplifies things. BTW we don't support
> > 32 bit shadow stack.
> 
> Wouldn't you need extra code to prevent IBT from working in 32-bit mode?
> Since the user can just far call into 32-bit without asking the kernel
> AFAIK.

Hmm, yea. We could either ignore it (let the signal handlers, etc just not be
functional). Or block 32 bit syscalls when IBT is on, like discussed on the
shadow stack thread.

> 
> With IBT state in fpstate, IBT would work fine for any variant of
> sigreturn (32-bit and 64-bit) all in the common signal frame restore
> code, no need to modify any syscall handlers.
> 
> So, I'd argue it's simpler to leave it enabled in 32-bit compatibility
> mode even if no one ever used 32-bit IBT. We should certainly prevent
> IBT from being compiled in for true 32-bit kernels though.
> 
> > What happened to the discussion of a PROT_IBT (like PROT_BTI)? I had POCed two
> > ways to do it and one was not that bad. The legacy bitmap thing is not easy to
> > use here, and I don't recommend it. But handling IBT #CPs by looking up the VMA
> > of RIP was pretty compact. If the VMA has !PROT_IBT, then clear the TRACKER bit
> 
> Sorry, I just forgot about them! I like the idea. 
> 
> Do you think we should do PROT_IBT first or prctl or both?
> I'm happy to help upstream the POC, please let me know.

It would need a bit of cleanup to share. It is mixed in with the bitmap version,
which was quite a mess. But the code to do the #CP violation based PROT_IBT is
very small. Here is the jist of it to give some idea. It was based on Yu-cheng's
original user IBT implementation that you linked:

int handle_user_ibt_violation_cp(struct vm_area_struct *vma, unsigned long addr)
{
	if (vma->vm_flags & VM_IBT)
		return 0;

	ibt_get_clear_wait_endbr();

	return 1;
}

int handle_user_ibt_violation(struct vm_area_struct *vma, unsigned long addr)
{
	if (features_enabled(ARCH_IBT_PROT_IBT_BITMAP))
		return handle_user_ibt_violation_bitmap(vma, addr);
	else if(features_enabled(ARCH_IBT_PROT_IBT_CP))
		return handle_user_ibt_violation_cp(vma, addr);

	return 0;
}


diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c
index d2c732a34e5d9..7152f31095912 100644
--- a/arch/x86/kernel/cet.c
+++ b/arch/x86/kernel/cet.c
@@ -44,6 +44,26 @@ static void do_unexpected_cp(struct pt_regs *regs, unsigned
long error_code)
 static DEFINE_RATELIMIT_STATE(cpf_rate, DEFAULT_RATELIMIT_INTERVAL,
                              DEFAULT_RATELIMIT_BURST);
 
+static int do_user_ibt_violation(struct pt_regs *regs, unsigned long
error_code)
+{
+       struct mm_struct *mm = current->mm;
+       struct vm_area_struct *vma;
+       int ret = 0;
+
+       vma = lock_mm_and_find_vma(mm, regs->ip, regs);
+       /* TODO: Hmm, what to do if the VMA is gone? */
+       if (!vma) {
+               printk("vma missing\n");
+               return 0;
+       }
+
+       if (handle_user_ibt_violation(vma, regs->ip))
+               ret = 1;
+
+       mmap_read_unlock(mm);
+       return ret;
+}
+
 static void do_user_cp_fault(struct pt_regs *regs, unsigned long error_code)
 {
        struct task_struct *tsk;
@@ -63,6 +83,9 @@ static void do_user_cp_fault(struct pt_regs *regs, unsigned
long error_code)
        tsk->thread.error_code = error_code;
        tsk->thread.trap_nr = X86_TRAP_CP;
 
+       if (((error_code & CP_EC) == CP_ENDBR) && do_user_ibt_violation(regs,
error_code))
+               goto out;
+
        /* Ratelimit to prevent log spamming. */
        if (show_unhandled_signals && unhandled_signal(tsk, SIGSEGV) &&
            __ratelimit(&cpf_rate)) {
@@ -76,6 +99,8 @@ static void do_user_cp_fault(struct pt_regs *regs, unsigned
long error_code)
        }
 
        force_sig_fault(SIGSEGV, SEGV_CPERR, (void __user *)0);
+
+out:
        cond_local_irq_disable(regs);
 }


> 
> > and proceed. This punishes mixed mode apps indirect calls (not *that* horrible
> > IIRC, I've lost my test results), but doesn't hurt fully enabled apps.
> 
> Could you share your POC?
> I'm curious about the overhead on top of a regular GOT-PLT inter-DSO
> call. There are probably apps that spam those, so if ld.so started
> enabling PROT_IBT automatically, those apps would probably be confused
> by any performance regressions.
> 
> > Now, the security question of whether a mixed mode makes sense, is valid I
> > think. But the issue we struggled with a lot for shadow stack was compatibility
> > of apps made of multiple packages (which is a lot of them), or which have JITs.
> > So I think a mixed mode would be valuable if it can be a stepping stone to a
> > fully locked down mode eventually. For example, start with a more permissive
> > prctl that turns on mixed (PROT_IBT) mode. Distros and other wider enablers can
> > use this without fear of crashing apps with JITs, etc. Emit a pr_info() or
> > something that incentives people to fix their libs. Then later distros can
> > switch to a fully enforced on thing.
> 
> I think pr_info() instead of crashing is a really cool idea! I will port
> this to the prctl() app-wide API. We'd disable IBT and log to dmesg on
> the first violation in log mode. (hopefully loudly enough so that the
> distros' error-reporting telemetry uploads them)
> 
> This allows glibc/distros to roll out IBT immediately.
> 
> > But would be good to hear updated opinions from the distros on this. (maybe I
> > missed it?)
> 
> Wasn't sure who to CC, Florian what do you think?

Are you working on getting IBT ready for a distro? Or just other interest?

> 
> > > - IBT enablement sets both ENDBR_EN and NO_TRACK_EN
> > >   (enforce endbr64 on indirect jumps, allow 'notrack' prefix to opt-out)
> > > - vDSO polishing needed (add missing endbr64 markers, GNU property note)
> > > - signal handler entrypoint does not need 'endbr64' (the signal handler
> > >   entrypoint cannot easily be changed)
> > 
> > I think the signal delivery should manually check for endbr in SW. Using
> > something like the speculate loop in shstk_pop_sigframe(). What is the problem?
> > 
> > At the same time, something small that actually upstream is better than nothing.
> 
> It's trivial to enforce it. But generally the less endbr markers the
> better, because it reduces valid sites for `call rax`. Hijacking a
> sigaction call is much harder than smashing a function pointer.

I wouldn't try to stop an IBT implementation that was missing this to start. But
I think setting a signal handler to random code and triggering a signal would be
pretty trivial forward edge by pass?

> 
> Userland should eventually do a variant of FineIBT where the callee
> checks a cookie that the caller sets, e.g. `mov r13, MAGIC; call rax`.
> If the kernel required endbr for signal delivery, that blows a gap into
> the cookie mechanism. The signal handler would always be a valid target
> because such a cookie can hardly ever be part of uapi (kernel signal
> delivery wouldn't know what cookie the userland handler expects).

Ah! This also brings up the wonderful problem of mixed mode FineIBT, +Joao who
has had a lot of thoughts on this subject. And the signal delivery part too
IIRC.

> 
> Thanks,
> -- Richard