Re: [PATCH v2 2/2] lib/percpu_counter: fix dying cpu compare race

[Thomas Gleixner]

On Mon, Apr 10 2023 at 22:53, Thomas Gleixner wrote:
> On Thu, Apr 06 2023 at 09:56, Ye Bin wrote:
> cpu_dying_mask is a random number generator w/o cpu_hotplug_lock being
> held. And even with that lock held any cpumask operation on it is silly.
> The mask is a core detail:
>
>   commit e40f74c535b8 "cpumask: Introduce DYING mask"
>     
>     Introduce a cpumask that indicates (for each CPU) what direction the
>     CPU hotplug is currently going. Notably, it tracks rollbacks. Eg. when
>     an up fails and we do a roll-back down, it will accurately reflect the
>     direction.
>
> It does not tell anything to a user which is not aware of the actual
> hotplug state machine state.

Even if the mask is most of the time stable, it's a total disaster
performance wise. The bits in cpu_dying_mask are sticky until the next
online operation.

So for a system which has SMT enabled in BIOS, but SMT is disabled on
the kernel command line or later via the sysfs knob, this means that the
loop in __percpu_counter_sum() will iterate over all shutdown SMT
siblings forever. IOW, it will touch nr_of_shutdown_cpus() cachelines
for absolutely zero reason.

The same applies for the proposed counter.

Thanks,

        tglx