Re: [RFC PATCH v8 08/10] x86/mm/pti: Introduce a kernel/user CR3 software signal

[Frederic Weisbecker]

Le Tue, Mar 24, 2026 at 10:47:59AM +0100, Valentin Schneider a écrit :
> Later commits will rely on being able to check whether a remote CPU is
> using the kernel or the user CR3.
> 
> This software signal needs to be updated before the actual CR3 write, IOW
> it always immediately precedes it:
> 
>   KERNEL_CR3_LOADED := 1
>   SWITCH_TO_KERNEL_CR3
>   [...]
>   KERNEL_CR3_LOADED := 0
>   SWITCH_TO_USER_CR3
> 
> The variable also gets mapped into the user space visible pages.
> I tried really hard not to do that, and at some point had something mostly
> working with having an alias to it through the cpu_entry_area accessed like
> so before the switch to the kernel CR3:
> 
> 	subq $10, %rsp
> 	sgdt (%rsp)
> 	movq 2(%rsp), \scratch_reg /* GDT address */
> 	addq $10, %rsp
> 
> 	movl $1, CPU_ENTRY_AREA_kernel_cr3(\scratch_reg)
> 
> however this explodes when running 64-bit user code that invokes SYSCALL,
> since the scratch reg is %rsp itself, and I figured this was enough headaches.
> 
> This will only be really useful for NOHZ_FULL CPUs, but it should be
> cheaper to unconditionally update a never-used per-CPU variable living in
> its own cacheline than to check a shared cpumask such as
>   housekeeping_cpumask(HK_TYPE_KERNEL_NOISE)
> at every entry.
> 
> Signed-off-by: Valentin Schneider <vschneid@xxxxxxxxxx>
> ---
>  arch/x86/Kconfig                | 14 +++++++++++++
>  arch/x86/entry/calling.h        | 13 ++++++++++++
>  arch/x86/entry/syscall_64.c     |  4 ++++
>  arch/x86/include/asm/tlbflush.h |  3 +++
>  arch/x86/mm/pti.c               | 36 ++++++++++++++++++++++-----------
>  5 files changed, 58 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> index 80527299f859a..f680e83cd5962 100644
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -2192,6 +2192,20 @@ config ADDRESS_MASKING
>  	  The capability can be used for efficient address sanitizers (ASAN)
>  	  implementation and for optimizations in JITs.
>  
> +config TRACK_CR3
> +       def_bool n
> +       prompt "Track which CR3 is in use"
> +       depends on X86_64 && MITIGATION_PAGE_TABLE_ISOLATION && NO_HZ_FULL
> +       help
> +	 This option adds a software signal that allows checking remotely
> +	 whether a CPU is using the user or the kernel page table.
> +
> +	 This allows further optimizations for NOHZ_FULL CPUs.
> +
> +	 This obviously makes the user<->kernel transition overhead even worse.
> +
> +	 If unsure, say N.
> +
>  config HOTPLUG_CPU
>  	def_bool y
>  	depends on SMP
> diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h
> index 77e2d920a6407..4099b7d86efd9 100644
> --- a/arch/x86/entry/calling.h
> +++ b/arch/x86/entry/calling.h
> @@ -9,6 +9,7 @@
>  #include <asm/ptrace-abi.h>
>  #include <asm/msr.h>
>  #include <asm/nospec-branch.h>
> +#include <asm/jump_label.h>
>  
>  /*
>  
> @@ -170,8 +171,17 @@ For 32-bit we have the following conventions - kernel is built with
>  	andq    $(~PTI_USER_PGTABLE_AND_PCID_MASK), \reg
>  .endm
>  
> +.macro NOTE_CR3_SWITCH scratch_reg:req in_kernel:req
> +#ifdef CONFIG_TRACK_CR3
> +	STATIC_BRANCH_FALSE_LIKELY housekeeping_overridden, .Lend_\@
> +	movl \in_kernel, PER_CPU_VAR(kernel_cr3_loaded)

Does this need full ordering of some sort? Like this should be LOCK xadd ?

Thanks.

-- 
Frederic Weisbecker
SUSE Labs