Re: [PATCH v2 11/15] arm64: add EFI stub

From: Roy Franz
Date: Tue Mar 18 2014 - 17:48:42 EST

On Tue, Mar 18, 2014 at 2:40 PM, Mark Salter <msalter@xxxxxxxxxx> wrote:
> On Tue, 2014-03-18 at 18:28 +0000, Catalin Marinas wrote:
>> On Tue, Mar 18, 2014 at 02:40:29PM +0000, Mark Salter wrote:
>> > On Tue, 2014-03-18 at 12:09 +0000, Catalin Marinas wrote:
>> > > On Thu, Mar 13, 2014 at 10:47:04PM +0000, Leif Lindholm wrote:
>> > > > --- /dev/null
>> > > > +++ b/arch/arm64/kernel/efi-entry.S
>> > > > @@ -0,0 +1,93 @@
>> > > > +/*
>> > > > + * EFI entry point.
>> > > > + *
>> > > > + * Copyright (C) 2013 Red Hat, Inc.
>> > > > + * Author: Mark Salter <msalter@xxxxxxxxxx>
>> > > > + *
>> > > > + * This program is free software; you can redistribute it and/or modify
>> > > > + * it under the terms of the GNU General Public License version 2 as
>> > > > + * published by the Free Software Foundation.
>> > > > + *
>> > > > + */
>> > > > +#include <linux/linkage.h>
>> > > > +#include <linux/init.h>
>> > > > +
>> > > > +#include <asm/assembler.h>
>> > > > +
>> > > > +#define EFI_LOAD_ERROR 0x8000000000000001
>> > > > +
>> > > > + __INIT
>> > > > +
>> > > > + /*
>> > > > + * We arrive here from the EFI boot manager with:
>> > > > + *
>> > > > + * * MMU on with identity-mapped RAM.
>> > > > + * * Icache and Dcache on
>> > > > + *
>> > > > + * We will most likely be running from some place other than where
>> > > > + * we want to be. The kernel image wants to be placed at TEXT_OFFSET
>> > > > + * from start of RAM.
>> > > > + */
>> > > > +ENTRY(efi_stub_entry)
>> > > > + stp x29, x30, [sp, #-32]!
>> > > > +
>> > > > + /*
>> > > > + * Call efi_entry to do the real work.
>> > > > + * x0 and x1 are already set up by firmware. Current runtime
>> > > > + * address of image is calculated and passed via *image_addr.
>> > > > + *
>> > > > + * unsigned long efi_entry(void *handle,
>> > > > + * efi_system_table_t *sys_table,
>> > > > + * unsigned long *image_addr) ;
>> > > > + */
>> > > > + adrp x8, _text
>> > > > + add x8, x8, #:lo12:_text
>> > > > + add x2, sp, 16
>> > > > + str x8, [x2]
>> > > > + bl efi_entry
>> > > > + cmn x0, #1
>> > > > + b.eq efi_load_fail
>> > > > +
>> > > > + /*
>> > > > + * efi_entry() will have relocated the kernel image if necessary
>> > > > + * and we return here with device tree address in x0 and the kernel
>> > > > + * entry point stored at *image_addr. Save those values in registers
>> > > > + * which are preserved by __flush_dcache_all.
>> > > > + */
>> > > > + ldr x1, [sp, #16]
>> > > > + mov x20, x0
>> > > > + mov x21, x1
>> > > > +
>> > > > + /* Turn off Dcache and MMU */
>> > > > + mrs x0, CurrentEL
>> > > > + cmp x0, #PSR_MODE_EL2t
>> > > > + ccmp x0, #PSR_MODE_EL2h, #0x4, ne
>> > > > + b.ne 1f
>> > > > + mrs x0, sctlr_el2
>> > > > + bic x0, x0, #1 << 0 // clear SCTLR.M
>> > > > + bic x0, x0, #1 << 2 // clear SCTLR.C
>> > > > + msr sctlr_el2, x0
>> > > > + isb
>> > > > + b 2f
>> > > > +1:
>> > > > + mrs x0, sctlr_el1
>> > > > + bic x0, x0, #1 << 0 // clear SCTLR.M
>> > > > + bic x0, x0, #1 << 2 // clear SCTLR.C
>> > > > + msr sctlr_el1, x0
>> > > > + isb
>> > > > +2:
>> > > > + bl __flush_dcache_all
>> > >
>> > > In linux-next I'm pushing a patch which no longer exports the
>> > > __flush_dcache_all function. The reason is that it doesn't really work
>> > > if you have a (not fully transparent) external cache like on the Applied
>> > > Micro boards. There other issues when running as a guest as well.
>> > >
>> > > If you know exactly what needs to be flushed here, can you use a range
>> > > (MVA) operation?
>> >
>> > This is just before the EFI stub jumps to kernel proper. The only things
>> > in the dcache would be from identity mapped references to RAM used by
>> > UEFI. The booting.txt doc says dcache should be off and invalidated. I
>> > am just wanting to comply with that. The code here doesn't really know
>> > the extent of DRAM to flush by address.
>>
>> Does UEFI do anything with the caches before invoking the EFI_STUB code?
>> I guess it doesn't since that's just another application for it. Can
>> UEFI flush the caches via exit boot? When is this called?
>>
>> As I said, we have a real problem here since the EFI_STUB call does not
>> have information about the SoC to be able to flush all the caches. But
>> UEFI should know more about the hardware.
>>
>> If UEFI doesn't handle the caches, the only thing left to EFI_STUB is to
>> flush by MVA. We don't need to flush the whole DRAM (and I would even
>> recommend it) but at least the relevant kernel code/data touched with
>> the MMU disabled.
>>
>
> So, it goes like this:
>
> 1) UEFI calls stub with MMU/Caches on. Stub/kernel can be anywhere.
> 2) Stub runs and relocates kernel to the desired runtime location
> but continues to execute from wherever UEFI loaded it until just
> after ExitBootServices().
> 3) After ExitBootServices, efi_entry() returns relocated entry point
> for kernel to efi_stub_entry() in efi-entry.S where the Dcache and
> MMU are turned off, the __flush_dcache_all is called, then the
> code jumps to the kernel proper entry point.
>
> It isn't clear to me if UEFI does cache flushing at ExitBootServices
> time, but even so, at least stack use will get cached between then and
> the kernel entry point. The stub could conceivably get its hands on the
> EFI memmap and invalidate dcache using address ranges from UEFI memory
> descriptors so maybe that is the way we should do it.
>

I looked at the UEFI spec and there is no mention of cache flushing in
ExitBootServices(),
so it seems it is up to the OS to do any cache management.

Roy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/