Re: 4.17.x won't boot due to "x86/boot/compressed/64: Handle 5-level paging boot if kernel is above 4G"

[Kirill A. Shutemov]

On Tue, Jul 03, 2018 at 03:44:03PM +0300, Kirill A. Shutemov wrote:
> On Tue, Jul 03, 2018 at 01:24:49PM +0200, Gabriel C wrote:
> > 2018-07-01 23:32 GMT+02:00 Benjamin Gilbert <bgilbert@xxxxxxxxxx>:
> > > On Sun, Jul 01, 2018 at 05:15:59PM -0400, Benjamin Gilbert wrote:
> > >> 4.17 kernels built with the CoreOS Container Linux toolchain and kconfig,
> > >> up to and including 4.17.3, fail to boot on AMD64 running in (at least)
> > >> QEMU/KVM.  No messages are shown post-GRUB; the VM instantly reboots.
> > >> Reverting commit 194a9749c73d ("x86/boot/compressed/64: Handle 5-level
> > >> paging boot if kernel is above 4G") fixes it.  I've attached our kernel
> > >> config for reference, and am happy to test patches, provide sample QCOW
> > >> images, etc.
> > >
> > 
> > Also see https://bugzilla.kernel.org/show_bug.cgi?id=200385 ,
> > 
> > 0a1756bd2897951c03c1cb671bdfd40729ac2177 is acting up
> > too with the same symptoms
> 
> I tracked it down to -flto in LDFLAGS. I'll look more into this.

-flto in LDFLAGS screws up this part of paging_prepare():

	/* Copy trampoline code in place */
	memcpy(trampoline_32bit + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long),
			&trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE);

In particular, relocation for trampoline_32bit_src solved in the wrong
way. Without -flto, we have rip-realtive address load:

  982d30:	48 8d 35 09 cc ff ff 	lea    -0x33f7(%rip),%rsi        # 97f940 <trampoline_32bit_src>

With -flto we have immediate load:

  982cf0:	48 c7 c6 f0 f8 97 00 	mov    $0x97f8f0,%rsi

It only would be okay if bootloader loads kernel at the address we compile
it for. But it's not usually the case.

As result we copy garbage into trampoline and crash when trying to execute
it.

I don't know how to solve it. As far as I know we don't support compiling
kernel with LTO in mainline.

Any suggestions?

Benjamin, do you change LDFLAGS or CFLAGS when compiling the kernel?

-- 
 Kirill A. Shutemov