Re: [PATCH] cpu/hotplug: Fix rollback during error-out in takedown_cpu()

[Mukesh Ojha]

On 9/5/2018 5:03 PM, Thomas Gleixner wrote:
> On Tue, 4 Sep 2018, Neeraj Upadhyay wrote:
> > If takedown_cpu() fails during _cpu_down(), st->state is reset,
> > by calling cpuhp_reset_state(). This results in an additional
> > increment of st->state, which results in CPUHP_AP_SMPBOOT_THREADS
> > state being skipped during rollback. Fix this by not calling
> > cpuhp_reset_state() and doing the state reset directly in
> > _cpu_down().
> >
> > Fixes: 4dddfb5faa61 ("smp/hotplug: Rewrite AP state machine core")
> > Signed-off-by: Neeraj Upadhyay <neeraju@xxxxxxxxxxxxxx>
> > ---
> >   kernel/cpu.c | 9 ++++++++-
> >   1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/kernel/cpu.c b/kernel/cpu.c
> > index aa7fe85..9f49edb 100644
> > --- a/kernel/cpu.c
> > +++ b/kernel/cpu.c
> > @@ -970,7 +970,14 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
> >   	 */
> >   	ret = cpuhp_down_callbacks(cpu, st, target);
> >   	if (ret && st->state > CPUHP_TEARDOWN_CPU && st->state < prev_state) {
> > -		cpuhp_reset_state(st, prev_state);
> > +		/*
> > +		 * As st->last is not set, cpuhp_reset_state() increments
> > +		 * st->state, which results in CPUHP_AP_SMPBOOT_THREADS being
> > +		 * skipped during rollback. So, don't use it here.
> > +		 */
> > +		st->rollback = true;
> > +		st->target = prev_state;
> > +		st->bringup = !st->bringup;
> No, this is just papering over the actual problem.
>
> The state inconsistency happens in take_cpu_down() when it returns with a
> failure from __cpu_disable() because that returns with state = TEARDOWN_CPU
> and st->state is then incremented in undo_cpu_down().
>
> That's the real issue and we need to analyze the whole cpu_down rollback
> logic first.

Could this be done like below ?

diff --git a/kernel/cpu.c b/kernel/cpu.c
index aa7fe85..47bce90 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -802,17 +802,18 @@ static int take_cpu_down(void *_param)
ÂÂÂÂÂÂÂ int err, cpu = smp_processor_id();
ÂÂÂÂÂÂÂ int ret;

-ÂÂÂÂÂÂ /* Ensure this CPU doesn't handle any more interrupts. */
-ÂÂÂÂÂÂ err = __cpu_disable();
-ÂÂÂÂÂÂ if (err < 0)
-ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ return err;
-
ÂÂÂÂÂÂÂ /*
ÂÂÂÂÂÂÂÂ * We get here while we are in CPUHP_TEARDOWN_CPU state and we 
must not
ÂÂÂÂÂÂÂÂ * do this step again.
ÂÂÂÂÂÂÂÂ */
ÂÂÂÂÂÂÂ WARN_ON(st->state != CPUHP_TEARDOWN_CPU);
ÂÂÂÂÂÂÂ st->state--;
+
+ÂÂÂÂÂÂ /* Ensure this CPU doesn't handle any more interrupts. */
+ÂÂÂÂÂÂ err = __cpu_disable();
+ÂÂÂÂÂÂ if (err < 0)
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ return err;
+
ÂÂÂÂÂÂÂ /* Invoke the former CPU_DYING callbacks */

Thanks,
Mukesh
>