Re: Unifying x86_64 / Xen init paths and reading hardware_subarch early

[Andy Lutomirski]

On Fri, Jan 15, 2016 at 2:08 PM, Luis R. Rodriguez <mcgrof@xxxxxxxx> wrote:
> I will be respinning the generic Linux linker table solution [0] soon
> based on hpa's feedback again now that I'm back from vacation. As I do
> that though I wanted to highlight a feature I'm throwing into the
> linker table solution which I am not sure many have paid close
> attention to but I think is important to Xen. I'm making use of the
> zero page hardware_subarch to enable us to detect if we're a specific
> hypervisor solution *as early as is possible*. This has a few
> implications, short term it is designed to provides a proactive
> technical solution to bugs such as the cr4 shadow crash (see
> 5054daa285beaf706f051fbd395dc36c9f0f907f) and ensure that *new* x86
> features get a proper Xen implementation proactively *or* at the very
> least get annotated as unsupported properly, instead of having them
> crash and later finding out. A valid example here is Kasan, which to
> this day lacks proper Xen support. In the future, if the generic
> linker table solution gets merged, it would mean developers would have
> to *think* about if they support Xen or not at development time. It
> does this in a not-disruptive way to Xen / x86_64 but most
> *importantly* it does not extend pvops! This should avoid issues in
> cases of developer / maintainer bandwidth, should some new features be
> pushed onto Linux for x86_64 but a respective Xen solution is not
> addressed, and that was not caught early in patch review, such as with
> Kasan.
>
> [0] https://lkml.kernel.org/r/1450217797-19295-1-git-send-email-mcgrof@xxxxxxxxxxxxxxxx
>
> Two things I'd like to request a bit of help with and review / consideration:
>
> 1) I'd like some advice on a curious problem I've stumbled on. I'd
> like to access hardware_subarch super early, and in my review with at
> least two x86 folks this *should* work:
>
> diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
> index c913b7eb5056..9168842821c8 100644
> --- a/arch/x86/kernel/head64.c
> +++ b/arch/x86/kernel/head64.c
> @@ -141,6 +141,7 @@ static void __init copy_bootdata(char *real_mode_data)
>
>  asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data)
>  {
> + struct boot_params *params = (struct boot_params *)__va(real_mode_data);
>   int i;

This is a mess :-p

If you want to access real_mode_data before load_idt, you'll need to do:

    for (i = 0; i < sizeof(boot_params); i += 4096)
        early_make_pgtable((unsigned long)params + i);

Of course, it's entirely possible that that will blow up if you try to
do it on Xen.

I think this would all be easier to understand if you try to separate
out the ideas of linker tables from the idea of rearranging early
init.  AFAICT the linker table thing is just an implementation detail.

If I understand right, you're trying to unify the Xen and native
startup as much as possible.  Why not add little shims, though?
Create a new start_kernel_common(int subarch, ...) where subarch
indicates native vs Xen and have its callers tell it which mode it's
in?

--Andy