Re: [PATCHv3 00/14] x86/tdx: Add kexec support

[Baoquan He]

On 11/21/23 at 11:43am, Kirill A. Shutemov wrote:
> On Tue, Nov 21, 2023 at 02:41:08PM +0800, Baoquan He wrote:
> > > > 
> > > > Still failed. And I found the normal reboot does't work either. I will
> > > > do more testing tomorrow, e.g use the tdx-tools's own rhel9 kernel
> > > > config and rebuild, and update host kernel too.
> > 
> > I did more tests, resuls are summarized as below: 
> > 
> > 1) kexec reboot works, but always fallback to 1 cpu even though multiple
> > cpus are specified;
> 
> That's expected. Until you have new BIOS. See below.
> 
> > 2) kdump kernel need more crashkernel memory to boot up,
> >    crashkernel=512M works well in our case.
> 
> I guess it is due to SWIOTLB memory which requres at least 64M.
> 
> >  But it failed in vmcore
> >    saving process, either makedumpfile or cp can't access the 1st
> >    kernel's old memory;
> 
> Will look into it.
> 
> > 3) Normal reboot always failed;
> 
> It is expected. TD vCPUs are not resettable. So we need to destroy TD and
> construct a new one to emulate "reboot".
> 
> I guess we can try to add some glue on QEMU side to make reboot more
> seamless.
> 
> > My colleague Tao helped to double check this, he got the same testing result.
> > Plesae 
> > 1) what can we do to enable the multiple cpu support for kexec reboot?
> 
> You would need a patched BIOS image. I've hacked one[1] for my testing.
> But it only works if kernel runs in 4-level paging mode (specify no5lvl in
> kernel command line).
> 
> BIOS folks work on proper patch, but it is not ready yet.
> 
> [1] https://gist.github.com/kiryl/e1dc1719e0c990b3ceee5d8de8dbf332

Thanks a lot for explanations. Let's wait for it.

> 
> > 2) anything missing to allow makedumpfile/cp access 1st kernel's memory?
> 
> It worked before for us, but I have not checked for a while.
> 
> I expected you've dropped my "if (crash) return;" debug patch I asked you
> to tested before, right? If not, failure is expected.

Ah, my bad. I didn't actually.

Just now I rebuilt kernel and retested, vmcore saving works. Just when
it finished kdump work and tried to reboot, the rebooting failed because
cpus are not resettable like kexec reboot does. Not sure if this is seen
in your test.
> 
> > 3) not sure if this is particular case on the system we tested on.
> 
> -- 
>   Kiryl Shutsemau / Kirill A. Shutemov
>