Re: [PATCH] cpu/hotplug: Allow the CPU in CPU_UP_PREPARE state to be brought up again.

[Boris Ostrovsky]

On 11/24/21 5:54 PM, Thomas Gleixner wrote:
> On Mon, Nov 22 2021 at 16:47, Sebastian Andrzej Siewior wrote:
> > From: "Longpeng(Mike)" <longpeng2@xxxxxxxxxx>
> >
> > A CPU will not show up in virtualized environment which includes an
> > Enclave. The VM splits its resources into a primary VM and a Enclave
> > VM. While the Enclave is active, the hypervisor will ignore all requests
> > to bring up a CPU and this CPU will remain in CPU_UP_PREPARE state.
> > The kernel will wait up to ten seconds for CPU to show up
> > (do_boot_cpu()) and then rollback the hotplug state back to
> > CPUHP_OFFLINE leaving the CPU state in CPU_UP_PREPARE. The CPU state is
> > set back to CPUHP_TEARDOWN_CPU during the CPU_POST_DEAD stage.
> >
> > After the Enclave VM terminates, the primary VM can bring up the CPU
> > again.
> >
> > Allow to bring up the CPU if it is in the CPU_UP_PREPARE state.
> >
> > [bigeasy: Rewrite commit description.]
> >
> > Signed-off-by: Longpeng(Mike) <longpeng2@xxxxxxxxxx>
> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>
> > Link: https://lore.kernel.org/r/20210901051143.2752-1-longpeng2@xxxxxxxxxx
> > ---
> >
> > For XEN: this changes the behaviour as it allows to invoke
> > cpu_initialize_context() again should it have have earlier. I *think*
> > this is okay and would to bring up the CPU again should the memory
> > allocation in cpu_initialize_context() fail.
> Any comment from XEN folks?


If memory allocation in cpu_initialize_context() fails we will not be able to bring up the VCPU because xen_cpu_initialized_map bit at the top of that routine will already have been set. We will BUG in xen_pv_cpu_up() on second (presumably successful) attempt because nothing for that VCPU would be initialized. This can in principle be fixed by moving allocation to the top of the routine and freeing context if the bit in the bitmap is already set.


Having said that, allocation really should not fail: for PV guests we first bring max number of VCPUs up and then offline them down to however many need to run. I think if we fail allocation during boot we are going to have a really bad day anyway.



-boris