Re: kexec reboot failed due to commit 75d090fd167ac

[Ard Biesheuvel]

On Fri, Sep 8, 2023 at 5:58 PM Kees Cook <keescook@xxxxxxxxxxxx> wrote:
>
> On Fri, Sep 08, 2023 at 03:32:33PM +0300, Kirill A. Shutemov wrote:
> > On Fri, Sep 08, 2023 at 02:02:30PM +0800, Aaron Lu wrote:
> > > On Thu, Sep 07, 2023 at 04:14:09PM +0300, Kirill A. Shutemov wrote:
> > > > On Tue, Aug 29, 2023 at 10:04:51PM +0800, Aaron Lu wrote:
> > > > > > Could you show dmesg of the first kernel before kexec?
> > > > >
> > > > > Attached.
> > > > >
> > > > > BTW, kexec is invoked like this:
> > > > > kver=6.4.0-rc5-00009-g75d090fd167a
> > > > > kdir=$HOME/kernels/$kver
> > > > > sudo kexec -l $kdir/vmlinuz-$kver --initrd=$kdir/initramfs-$kver.img --append="root=UUID=4381321e-e01e-455a-9d46-5e8c4c5b2d02 ro net.ifnames=0 acpi_rsdp=0x728e8014 no_hash_pointers sched_verbose selinux=0"
> > > >
> > > > I don't understand why it happens.
> > > >
> > > > Could you check if this patch changes anything:
> > > >
> > > > diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
> > > > index 94b7abcf624b..172c476ff6f3 100644
> > > > --- a/arch/x86/boot/compressed/misc.c
> > > > +++ b/arch/x86/boot/compressed/misc.c
> > > > @@ -456,10 +456,12 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap,
> > > >
> > > >   debug_putstr("\nDecompressing Linux... ");
> > > >
> > > > +#if 0
> > > >   if (init_unaccepted_memory()) {
> > > >           debug_putstr("Accepting memory... ");
> > > >           accept_memory(__pa(output), __pa(output) + needed_size);
> > > >   }
> > > > +#endif
> > > >
> > > >   __decompress(input_data, input_len, NULL, NULL, output, output_len,
> > > >                   NULL, error);
> > > > --
> > >
> > > It solved the problem.
> >
> > Looks like increasing BOOT_INIT_PGT_SIZE fixes the issue. I don't yet
> > understand why and how unaccepted memory is involved. I will look more
> > into it.
> >
> > Enabling CONFIG_RANDOMIZE_BASE also makes the issue go away.
>
> Is this perhaps just luck? I.e. does is break ever on, say, 1000 boot
> attempts? (i.e. maybe some position is bad and KASLR happens to usually
> avoid it?)
>
> > Kees, maybe you have a clue?
>
> The only thing I can think of is that something isn't being counted
> correctly due to the size of code, and it just happens that this commit
> makes the code large enough to exceed some set of mappings?
>
> >
> > diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h
> > index 9191280d9ea3..26ccce41d781 100644
> > --- a/arch/x86/include/asm/boot.h
> > +++ b/arch/x86/include/asm/boot.h
> > @@ -40,7 +40,7 @@
> >  #ifdef CONFIG_X86_64
> >  # define BOOT_STACK_SIZE     0x4000
> >
> > -# define BOOT_INIT_PGT_SIZE  (6*4096)
> > +# define BOOT_INIT_PGT_SIZE  (7*4096)
>
> That's why this might be working, for example? How large is the boot
> image before/after the commit, etc?
>

Not sure why these changes would make a difference here, but choking
on accept_memory() on a non-TDX suggests that init_unaccepted_memory()
is poking into unmapped memory before it even decides that the
unaccepted memory does not exist.

init_unaccepted_memory() has

        ret = efi_get_conf_table(boot_params, &cfg_table_pa, &cfg_table_len);
        if (ret) {
                warn("EFI config table not found.");
                return false;
        }

which looks for <guid, phys_addr> tuples in an array pointed to by the
EFI system table, and if either of those is not mapped, things can be
expected to explode.

The only odd thing there is that this code is invoked after setting up
the 'demand paging' logic in the decompressor.

If you haven't yet, could you please retry the kexec boot with
earlyprintk=tty<insert your UART params here>?