Re: Unifying x86_64 / Xen init paths and reading hardware_subarch early

[H. Peter Anvin]

On January 15, 2016 4:43:04 PM PST, "Luis R. Rodriguez" <mcgrof@xxxxxxxx> wrote:
>On Fri, Jan 15, 2016 at 03:47:25PM -0800, Andy Lutomirski wrote:
>> On Fri, Jan 15, 2016 at 2:08 PM, Luis R. Rodriguez <mcgrof@xxxxxxxx>
>wrote:
>> > I will be respinning the generic Linux linker table solution [0]
>soon
>> > based on hpa's feedback again now that I'm back from vacation. As I
>do
>> > that though I wanted to highlight a feature I'm throwing into the
>> > linker table solution which I am not sure many have paid close
>> > attention to but I think is important to Xen. I'm making use of the
>> > zero page hardware_subarch to enable us to detect if we're a
>specific
>> > hypervisor solution *as early as is possible*. This has a few
>> > implications, short term it is designed to provides a proactive
>> > technical solution to bugs such as the cr4 shadow crash (see
>> > 5054daa285beaf706f051fbd395dc36c9f0f907f) and ensure that *new* x86
>> > features get a proper Xen implementation proactively *or* at the
>very
>> > least get annotated as unsupported properly, instead of having them
>> > crash and later finding out. A valid example here is Kasan, which
>to
>> > this day lacks proper Xen support. In the future, if the generic
>> > linker table solution gets merged, it would mean developers would
>have
>> > to *think* about if they support Xen or not at development time. It
>> > does this in a not-disruptive way to Xen / x86_64 but most
>> > *importantly* it does not extend pvops! This should avoid issues in
>> > cases of developer / maintainer bandwidth, should some new features
>be
>> > pushed onto Linux for x86_64 but a respective Xen solution is not
>> > addressed, and that was not caught early in patch review, such as
>with
>> > Kasan.
>> >
>> > [0]
>https://lkml.kernel.org/r/1450217797-19295-1-git-send-email-mcgrof@xxxxxxxxxxxxxxxx
>> >
>> > Two things I'd like to request a bit of help with and review /
>consideration:
>> >
>> > 1) I'd like some advice on a curious problem I've stumbled on. I'd
>> > like to access hardware_subarch super early, and in my review with
>at
>> > least two x86 folks this *should* work:
>> >
>> > diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c
>> > index c913b7eb5056..9168842821c8 100644
>> > --- a/arch/x86/kernel/head64.c
>> > +++ b/arch/x86/kernel/head64.c
>> > @@ -141,6 +141,7 @@ static void __init copy_bootdata(char
>*real_mode_data)
>> >
>> >  asmlinkage __visible void __init x86_64_start_kernel(char *
>real_mode_data)
>> >  {
>> > + struct boot_params *params = (struct boot_params
>*)__va(real_mode_data);
>> >   int i;
>> 
>> This is a mess :-p
>
>Agreed. Doing what I can without extending pvops though ;)
>
>> If you want to access real_mode_data before load_idt, you'll need to
>do:
>> 
>>     for (i = 0; i < sizeof(boot_params); i += 4096)
>>         early_make_pgtable((unsigned long)params + i);
>
>Thanks I'll give this a shot.
>
>> Of course, it's entirely possible that that will blow up if you try
>to
>> do it on Xen.
>
>I'll check, if its safe and if the subarch strategy is desirable to
>help with
>unifying init, then great. Otherwise we'd need to figure this out.
>
>> I think this would all be easier to understand if you try to separate
>> out the ideas of linker tables from the idea of rearranging early
>> init.
>
>Oh absolutely. The goal to unify init *or* to access subarch earlier
>provides
>slightly different gains and possiblities. This is why I am addressing
>this
>separately. Its important to highlight the prospects though given I
>think a few
>folks may not have realized what might be possible here...
>
>>  AFAICT the linker table thing is just an implementation detail.
>
>Indeed, but just as a linker table is one thing, the *use* of the
>linker table
>for x86 early init is another. Its a good example how how to use the
>linker
>tables though.
>
>The things I make mention of here are just possible *enhancements* of
>that work
>provided the subarch can be read earlier.  Another possibility which I
>also had
>not mentioned is the ability to also free annotated code on x86 init
>which we
>*know* for sure we don't need, much as __init code after we boot, only
>this
>could be done later at run time. That's also best technically
>considered later
>but perhaps worth mentioning now as a future possibility.
>
>Although the linker table series does not address unifying init, in
>this thread
>we are talking about the prospect of being able to do that in the
>future. Its
>best to consider this early than late.
>
>> If I understand right, you're trying to unify the Xen and native
>> startup as much as possible. 
>
>That ultimately is a possibility. The original patches don't do that
>though.
>They just pave the way with linker tables as baby steps.
>
>Without access to the subarch so early unifying init is not possible
>with a
>linker table solution though. As the series was posted though its late
>use
>(after load_idt()) still holds promise to help annotate subarch
>support, which
>in turn also helps provide dependency maps. This should help with a
>proactive
>solution to ensure x86 developers think about x86 early requirements.
>Only
>that gain would be restricted to after load_idt().
>
>> Why not add little shims, though?
>> Create a new start_kernel_common(int subarch, ...) where subarch
>> indicates native vs Xen and have its callers tell it which mode it's
>> in?
>
>That's possible too, if you think that's best. However the subarch is
>already
>part of the zero page, so figured why not just make use of it. Also if
>you make
>use of it, it may even be possible to unify x86 init even on the asm
>front if
>we ultimately even wanted that. One of the gains of striving to unify
>inits
>*and* having a subarch dependency annotated for features is its a
>proactive
>measure to ensure developers think of the different x86 requirements.
>If we
>use the shim we may be able to unify some code but there would still be
>some area not addressed. My litmus test was to try to strive to address
>features such as Kasan, and that required even earlier subarch access.
>
>  Luis

You can't assume setup_data is page aligned.
-- 
Sent from my Android device with K-9 Mail. Please excuse brevity and formatting.