Re: HVMLite / PVHv2 - using x86 EFI boot entry

[Roger Pau Monné]

On Wed, Apr 13, 2016 at 11:15:15AM +0100, Matt Fleming wrote:
> On Wed, 13 Apr, at 11:02:02AM, Roger Pau Monné wrote:
> > 
> > With my FreeBSD committer hat:
> > 
> > The FreeBSD kernel doesn't contain an EFI entry point, it just contains one 
> > single entry point that's used for both legacy BIOS and EFI. Then the 
> > FreeBSD loader is the one that contains the different entry points. I would 
> > really like to avoid adding an EFI entry point and the PE header to the 
> > FreeBSD kernel. The current trampoline in FreeBSD to tie the Xen entry point 
> > into the native path contains 96 lines of assembly (half of them are 
> > actually comments) and 66 lines of C. I think adding an EFI entry point is 
> > going to add a lot more of code than this, and we would probably need 
> > changes to the build system in order to assembly the PE header and the ELF 
> > headers together.
>  
> What does the boot flow look like for PVH2 on FreeBSD today?
> Presumably it doesn't have the same entry point that Boris proposed
> for Linux?

Yes it does have something quite similar to the entry point that Boris 
proposed for Linux.
 
> Does it go, Hypervisor -> FreeBSD loader -> FreeBSD kernel? Or are you
> able to directly boot the kernel from the hypervisor and skip the
> middle part by having secondary entry point for Xen marked by the ELF
> note?

We skip the bootloader and Xen loads the FreeBSD kernel directly using the 
ELF note that contains the PVH entry point.

I certainly want to be able to run the FreeBSD loader inside of a PVH guest, 
but I plan to simply chainload it from OVMF, so it would look like:

Hypervisor -> OVMF -> FreeBSD EFI loader -> FreeBSD kernel

> > IMHO, if we want to boot PVH using EFI the right solution is to use OVMF (or 
> > any other UEFI firmware) and port it so it's able to run as a PVH guest. I 
> > guess it should even be possible to use it for Dom0, although I think this 
> > is cumbersome.
> 
> There are two levels of EFI boot entry features being discussed,
> 
>  1. Make the OS kernel a PE/COFF executable
>  2. Provide some level of EFI service functionality
> 
> You can adopt 1. without 2, i.e. without actually providing any EFI
> services at all, as long as the Xen hypervisor grows a PE/COFF loader
> (since EFI firmware has to provide you one, for EFI platforms you
> could use the LoadImage() service in the firmware, but for BIOS
> platforms you'd need your own in Xen).

We could use native LoadImage for Dom0 maybe if we are booted on an EFI 
platform, but for DomUs we certainly need to implement our own inside of 
Xen, at which point we could do the same and always use the one inside of 
Xen in order to avoid diverging paths.

TBH, I don't think this is the right solution. We would force every OS 
kernel that wants to be loaded using Xen to become a PE/COFF executable. 
This also includes unikernels like MirageOS, which will be forced to become 
a PE/COFF executable.

Is this header compatible with the ELF header? Con both co-exist in the 
same binary without issues?

> On Linux, this has the advantage of deferring the decompression of the
> bzImage (x86 Linux kernel file format) to the stub on the front of the
> bzImage. And while I realise that the toolstack already has support
> for decompressing bzImages, given what Andrew has said about reducing
> attack surface, having the guest perform the decompression should be a
> win.
> 
> Of course, this is offset somewhat by the fact that you need to audit
> the PE/COFF loader ;) But decompression in general is notoriously
> vulnerable to security issues.
> 
> Using the in-kernel decompressor is how most (all?) Linux boot loaders
> work today, so there's the added benefit of reducing the differences
> between booting on Xen and booting bare metal. For example, you'd
> probably be able to use CONFIG_RANDOMIZE_BASE (ASLR for kernel image)
> for Xen if you use the kernel's decompressor. Xen would also get
> future features in this area for free, and there is a tendency to push
> boot features into the early stub.

All the issues that you mention above are also solved by chainloading OVMF 
instead of directly loading the guest kernel, and it avoids adding a PE/COFF 
loader into Xen.

> For 1. we'd basically be using the PE/COFF file format with the EFI
> ABI as an OS agnostic boot protocol, but not as a full firmware
> runtime environment.

This also means that we will be adding PE/COFF headers to (uni)kernels, but 
we won't still implement full EFI support inside of them, so although it 
would seem like they are capable of being loaded by a native EFI loader, 
they would not.

This seems misleading, and I think it's going to cause grief amongst OS 
developers in general. The current proposed entry point is unique to Xen 
(it's only mentioned in Xen ELF notes), and is certainly not going to cause 
confusion at all.

Also, doesn't this (the fact that Xen will use the EFI entry point 
without a runtime environment) mean that there are going to be diverging 
paths inside of Linux EFI entry point anyway?

At which point, does it really matter that much if this divergence includes 
a new entry point or not?

> 2. is also interesting, though I think less so than 1. I agree that
> making OVMF work as a PVH guest is probably the right way to go, even
> for Dom0, not least because you'd have a much cleaner/less buggy
> implementation than what we see in the real world ;)

I think we all agree that this is not suitable.

Roger.