Re: KASAN vs. boot-time switching between 4- and 5-level paging

[Dmitry Vyukov]

On Mon, Jul 10, 2017 at 2:33 PM, Kirill A. Shutemov
<kirill@xxxxxxxxxxxxx> wrote:
> On Thu, Jun 01, 2017 at 05:56:30PM +0300, Andrey Ryabinin wrote:
>> On 05/29/2017 03:46 PM, Andrey Ryabinin wrote:
>> > On 05/29/2017 02:45 PM, Andrey Ryabinin wrote:
>> >>>>>> Looks like KASAN will be a problem for boot-time paging mode switching.
>> >>>>>> It wants to know CONFIG_KASAN_SHADOW_OFFSET at compile-time to pass to
>> >>>>>> gcc -fasan-shadow-offset=. But this value varies between paging modes...
>> >>>>>>
>> >>>>>> I don't see how to solve it. Folks, any ideas?
>> >>>>>
>> >>>>> +kasan-dev
>> >>>>>
>> >>>>> I wonder if we can use the same offset for both modes. If we use
>> >>>>> 0xFFDFFC0000000000 as start of shadow for 5 levels, then the same
>> >>>>> offset that we use for 4 levels (0xdffffc0000000000) will also work
>> >>>>> for 5 levels. Namely, ending of 5 level shadow will overlap with 4
>> >>>>> level mapping (both end at 0xfffffbffffffffff), but 5 level mapping
>> >>>>> extends towards lower addresses. The current 5 level start of shadow
>> >>>>> is actually close -- 0xffd8000000000000 and it seems that the required
>> >>>>> space after it is unused at the moment (at least looking at mm.txt).
>> >>>>> So just try to move it to 0xFFDFFC0000000000?
>> >>>>>
>> >>>>
>> >>>> Yeah, this should work, but note that 0xFFDFFC0000000000 is not PGDIR aligned address. Our init code
>> >>>> assumes that kasan shadow stars and ends on the PGDIR aligned address.
>> >>>> Fortunately this is fixable, we'd need two more pages for page tables to map unaligned start/end
>> >>>> of the shadow.
>> >>>
>> >>> I think we can extend the shadow backwards (to the current address),
>> >>> provided that it does not affect shadow offset that we pass to
>> >>> compiler.
>> >>
>> >> I thought about this. We can round down shadow start to 0xffdf000000000000, but we can't
>> >> round up shadow end, because in that case shadow would end at 0xffffffffffffffff.
>> >> So we still need at least one more page to cover unaligned end.
>> >
>> > Actually, I'm wrong here. I assumed that we would need an additional page to store p4d entries,
>> > but in fact we don't need it, as such page should already exist. It's the same last pgd where kernel image
>> > is mapped.
>> >
>>
>>
>> Something like bellow might work. It's just a proposal to demonstrate the idea, so some code might look ugly.
>> And it's only build-tested.
>
> [Sorry for loong delay.]
>
> The patch works for me for legacy boot. But it breaks EFI boot with
> 5-level paging. And I struggle to understand why.
>
> What I see is many page faults at mm/kasan/kasan.c:758 --
> "DEFINE_ASAN_LOAD_STORE(4)". Handling one of them I get double-fault at
> arch/x86/kernel/head_64.S:298 -- "pushq %r14", which ends up with triple
> fault.
>
> Any ideas?


Just playing the role of the rubber duck:
 - what is the fault address?
 - is it within the shadow range?
 - was the shadow mapped already?


> If you want to play with this by yourself, qemu supports la57 -- use
> -cpu "qemu64,+la57".
>
> --
>  Kirill A. Shutemov