Re: [PATCH v2 -tip] x86/percpu: Use C for arch_raw_cpu_ptr()

[Uros Bizjak]

On Wed, Oct 18, 2023 at 6:13 PM Linus Torvalds
<torvalds@xxxxxxxxxxxxxxxxxxxx> wrote:
>
> On Wed, 18 Oct 2023 at 06:15, Uros Bizjak <ubizjak@xxxxxxxxx> wrote:
> >
> > Attached is the prototype patch that works for me (together with
> > Linus' FPU switching patch).
>
> That looks reasonable, but I think the separate compilation unit is
> unnecessary, and I still absolutely hate how that const_pcpu_hot thing
> is declared twice (your patch does it in both current.h and in
> percpu-hot.c).
>
> How about we just do the whole alias as a linker thing instead? So the
> same way that we just do
>
>     jiffies = jiffies_64;
>
> in our arch/x86/kernel/vmlinux.lds.S file, we could just do
>
>     const_pcpu_hot = pcpu_hot;
>
> in there.
>
> Then, as far as the compiler is concerned, we just have
>
>     DECLARE_PER_CPU_ALIGNED(
>         const struct pcpu_hot __percpu_seg_override,
>         const_pcpu_hot)
>
> and the compiler doesn't know that it's aliased to anything else.

Oh...

... this works, too! Please see the attached patch.

(Note to self: Leave linking stuff to linker) ;)

> And please do that declaration in just *one* place.

Sure. Now the patch looks quite slim, but works as expected, reducing
the number of current_task accesses from 3841 to 3220.

Thanks,
Uros.
diff --git a/arch/x86/include/asm/current.h b/arch/x86/include/asm/current.h
index a1168e7b69e5..36902d61fd55 100644
--- a/arch/x86/include/asm/current.h
+++ b/arch/x86/include/asm/current.h
@@ -36,8 +36,17 @@ static_assert(sizeof(struct pcpu_hot) == 64);
 
 DECLARE_PER_CPU_ALIGNED(struct pcpu_hot, pcpu_hot);
 
+/*
+ *
+ */
+DECLARE_PER_CPU_ALIGNED(const struct pcpu_hot __percpu_seg_override,
+			const_pcpu_hot);
+
 static __always_inline struct task_struct *get_current(void)
 {
+	if (IS_ENABLED(CONFIG_USE_X86_SEG_SUPPORT))
+		return const_pcpu_hot.current_task;
+
 	return this_cpu_read_stable(pcpu_hot.current_task);
 }
 
diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S
index 9cdb1a7332c4..7cbc42619e16 100644
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -46,6 +46,7 @@ ENTRY(phys_startup_64)
 #endif
 
 jiffies = jiffies_64;
+const_pcpu_hot = pcpu_hot;
 
 #if defined(CONFIG_X86_64)
 /*
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index d7779a18b24f..bf9815eaf4aa 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -212,7 +212,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val,
  */
 #define ___ADDRESSABLE(sym, __attrs) \
 	static void * __used __attrs \
-		__UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)&sym;
+	__UNIQUE_ID(__PASTE(__addressable_,sym)) = (void *)(uintptr_t)&sym;
 #define __ADDRESSABLE(sym) \
 	___ADDRESSABLE(sym, __section(".discard.addressable"))