Re: Re: [PATCH] sched/mmcid: Bound the CID allocation busy wait

[Jiakai Xu]

> On Thu, Oct 15 2026 at 22:37, Thomas Gleixner wrote:
> > On Fri, Sep 18 2026 at 12:43, Peter Zijlstra wrote:
> > The stall makes no sense because mm_get_cid() is invoked from:
> >
> >   10967                 /* First user ? */
> >   10968                 if (!mm->mm_cid.users) {
> >   10969                         sched_mm_cid_add_user(t, mm);
> >   10970                         t->mm_cid.cid = mm_get_cid(mm);
> >
> > which means this is the first CID user on the MM. So how can the bitmap
> > be exhausted?
> >
> > I have a suspicion but it does not add up (yet).
> >
> > This is a two CPU VM. Assume that the process which invokes execve() has
> > more than two threads. That means the MMCID of the process is in per CPU
> > mode. sched_mm_cid_exit() ignores a mode change when the last user goes
> > away and with two CPUs the threshold to switch back is when the user
> > count goes below _ONE_. So there is probably stale state there, but the
> > last outgoing task should have reclaimed the per CPU CID and dropped it,
> > but the other CPU still has it assigned to it. So that first allocation
> > should succeed...
> >
> > I'll try to write a trivial test program tomorrow which creates enough
> > threads and does an execve().
> 
> Hmm. There is stale state, but there is always _ONE_ CID available
> because the last exiting thread drops the per CPU CID.
> 
> As the mode is TRANSIT the whole scenario resolves when the new exec
> spawns threads or schedules around. That just works by chance, but I
> couldn't find a way to exhaust the bitmap for the initial allocation.
> 
> So I really need to see the reproducer for this splat.

Hi Thomas,

Unfortunately, I don't have a reproducer for this issue.

Best regards,
Jiakai

> 
> The patch below cleans up the stale data and fixes a consistency issue
> vs. the allowed CPUs accounting.
> 
> Thanks,
> 
>         tglx
> ---
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -10880,6 +10880,10 @@ static inline void mm_update_cpus_allowe
>  	 */
>  	mc = &mm->mm_cid;
>  	guard(raw_spinlock)(&mc->lock);
> +	/* Might have raced with exit() */
> +	if (!mm->mm_cid.users)
> +		return;
> +
>  	mm_allowed = mm_cpus_allowed(mm);
>  	weight = cpumask_weighted_or(mm_allowed, mm_allowed, affmsk);
>  	if (weight == mc->nr_cpus_allowed)
> @@ -11019,6 +11023,33 @@ static bool sched_mm_cid_add_user(struct
>  	return mm_update_max_cids(mm);
>  }
>  
> +/*
> + * sched_mm_cid_exit() does not handle the mode switch for the last user
> + * because for normal exit() that's a pointless exercise. But if the
> + * process which issued execve() was multithreaded and the last exiting
> + * thread switched back from per CPU mode then this leaves stale state
> + * around. The indicator for that is mm_cid::mode having MM_CID_TRANSIT
> + * set.
> + */
> +static void sched_mm_cid_cleanup_execve(struct task_struct *t, struct mm_struct *mm)
> +{
> +	if (likely(!mm->mm_cid.mode))
> +		return;
> +
> +	WARN_ON_ONCE(mm->mm_cid.mode & MM_CID_ONCPU);
> +	mm->mm_cid.mode = 0;
> +
> +	unsigned int cpu;
> +
> +	for_each_possible_cpu(cpu)
> +		per_cpu(mm->mm_cid.pcpu->cid, cpu) = 0;
> +
> +	bitmap_zero(mm_cidmask(mm), num_possible_cpus());
> +
> +	mm->mm_cid.nr_cpus_allowed = t->nr_cpus_allowed;
> +	cpumask_copy(mm_cpus_allowed(mm), &t->cpus_mask);
> +}
> +
>  static void sched_mm_cid_fork(struct task_struct *t)
>  {
>  	struct mm_struct *mm = t->mm;
> @@ -11035,6 +11066,7 @@ static void sched_mm_cid_fork(struct tas
>  
>  		/* First user ? */
>  		if (!mm->mm_cid.users) {
> +			sched_mm_cid_cleanup_execve(t, mm);
>  			sched_mm_cid_add_user(t, mm);
>  			t->mm_cid.cid = mm_get_cid(mm);
>  			/* Required for execve() */
> 
>