Re: [PATCH v2 2/4] sched: Introduce is_pcpu_safe()

From: Boqun Feng
Date: Mon Aug 09 2021 - 22:42:37 EST


Hi,

On Sat, Aug 07, 2021 at 01:58:05AM +0100, Valentin Schneider wrote:
> Some areas use preempt_disable() + preempt_enable() to safely access
> per-CPU data. The PREEMPT_RT folks have shown this can also be done by
> keeping preemption enabled and instead disabling migration (and acquiring a
> sleepable lock, if relevant).
>
> Introduce a helper which checks whether the current task can safely access
> per-CPU data, IOW if the task's context guarantees the accesses will target
> a single CPU. This accounts for preemption, CPU affinity, and migrate
> disable - note that the CPU affinity check also mandates the presence of
> PF_NO_SETAFFINITY, as otherwise userspace could concurrently render the
> upcoming per-CPU access(es) unsafe.
>
> Signed-off-by: Valentin Schneider <valentin.schneider@xxxxxxx>
> ---
> include/linux/sched.h | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/include/linux/sched.h b/include/linux/sched.h
> index debc960f41e3..b77d65f677f6 100644
> --- a/include/linux/sched.h
> +++ b/include/linux/sched.h
> @@ -1715,6 +1715,16 @@ static inline bool is_percpu_thread(void)
> #endif
> }
>
> +/* Is the current task guaranteed not to be migrated elsewhere? */
> +static inline bool is_pcpu_safe(void)
> +{
> +#ifdef CONFIG_SMP
> + return !preemptible() || is_percpu_thread() || current->migration_disabled;
> +#else
> + return true;
> +#endif
> +}

I wonder whether the following can happen, say thread A is a worker
thread for CPU 1, so it has the flag PF_NO_SETAFFINITY set.

{ percpu variable X on CPU 2 is initially 0 }

thread A
========

<preemption enabled>
if (is_pcpu_safe()) { // nr_cpus_allowed == 1, so return true.
<preempted>
<hot unplug CPU 1>
unbinder_workers(1); // A->cpus_mask becomes cpu_possible_mask
<back to run on CPU 2>
__this_cpu_inc(X);
tmp = X; // tmp == 0
<preempted>
<in thread B>
this_cpu_inc(X); // X becomes 1
<back to run A on CPU 2>
X = tmp + 1; // race!
}

if so, then is_percpu_thread() doesn't indicate is_pcpu_safe()?

Regards,
Boqun

> +
> /* Per-process atomic flags. */
> #define PFA_NO_NEW_PRIVS 0 /* May not gain new privileges. */
> #define PFA_SPREAD_PAGE 1 /* Spread page cache over cpuset */
> --
> 2.25.1
>