Re: [PATCH v4 3/3] powerpc/32: Add KASAN support

[Christophe Leroy]

Le 12/02/2019 Ã 02:08, Daniel Axtens a ÃcritÂ:
> Andrey Ryabinin <aryabinin@xxxxxxxxxxxxx> writes:
>
> > On 2/11/19 3:25 PM, Andrey Konovalov wrote:
> > > On Sat, Feb 9, 2019 at 12:55 PM christophe leroy
> > > <christophe.leroy@xxxxxx> wrote:
> > > > Hi Andrey,
> > > >
> > > > Le 08/02/2019 Ã 18:40, Andrey Konovalov a Ãcrit :
> > > > > On Fri, Feb 8, 2019 at 6:17 PM Christophe Leroy <christophe.leroy@xxxxxx> wrote:
> > > > > > Hi Daniel,
> > > > > >
> > > > > > Le 08/02/2019 Ã 17:18, Daniel Axtens a Ãcrit :
> > > > > > > Hi Christophe,
> > > > > > >
> > > > > > > I've been attempting to port this to 64-bit Book3e nohash (e6500),
> > > > > > > although I think I've ended up with an approach more similar to Aneesh's
> > > > > > > much earlier (2015) series for book3s.
> > > > > > >
> > > > > > > Part of this is just due to the changes between 32 and 64 bits - we need
> > > > > > > to hack around the discontiguous mappings - but one thing that I'm
> > > > > > > particularly puzzled by is what the kasan_early_init is supposed to do.
> > > > > >
> > > > > > It should be a problem as my patch uses a 'for_each_memblock(memory,
> > > > > > reg)' loop.
> > > > > >
> > > > > > > > +void __init kasan_early_init(void)
> > > > > > > > +{
> > > > > > > > +    unsigned long addr = KASAN_SHADOW_START;
> > > > > > > > +    unsigned long end = KASAN_SHADOW_END;
> > > > > > > > +    unsigned long next;
> > > > > > > > +    pmd_t *pmd = pmd_offset(pud_offset(pgd_offset_k(addr), addr), addr);
> > > > > > > > +    int i;
> > > > > > > > +    phys_addr_t pa = __pa(kasan_early_shadow_page);
> > > > > > > > +
> > > > > > > > +    BUILD_BUG_ON(KASAN_SHADOW_START & ~PGDIR_MASK);
> > > > > > > > +
> > > > > > > > +    if (early_mmu_has_feature(MMU_FTR_HPTE_TABLE))
> > > > > > > > +            panic("KASAN not supported with Hash MMU\n");
> > > > > > > > +
> > > > > > > > +    for (i = 0; i < PTRS_PER_PTE; i++)
> > > > > > > > +            __set_pte_at(&init_mm, (unsigned long)kasan_early_shadow_page,
> > > > > > > > +                         kasan_early_shadow_pte + i,
> > > > > > > > +                         pfn_pte(PHYS_PFN(pa), PAGE_KERNEL_RO), 0);
> > > > > > > > +
> > > > > > > > +    do {
> > > > > > > > +            next = pgd_addr_end(addr, end);
> > > > > > > > +            pmd_populate_kernel(&init_mm, pmd, kasan_early_shadow_pte);
> > > > > > > > +    } while (pmd++, addr = next, addr != end);
> > > > > > > > +}
> > > > > > >
> > > > > > > As far as I can tell it's mapping the early shadow page, read-only, over
> > > > > > > the KASAN_SHADOW_START->KASAN_SHADOW_END range, and it's using the early
> > > > > > > shadow PTE array from the generic code.
> > > > > > >
> > > > > > > I haven't been able to find an answer to why this is in the docs, so I
> > > > > > > was wondering if you or anyone else could explain the early part of
> > > > > > > kasan init a bit better.
> > > > > >
> > > > > > See https://www.kernel.org/doc/html/latest/dev-tools/kasan.html for an
> > > > > > explanation of the shadow.
> > > > > >
> > > > > > When shadow is 0, it means the memory area is entirely accessible.
> > > > > >
> > > > > > It is necessary to setup a shadow area as soon as possible because all
> > > > > > data accesses check the shadow area, from the begining (except for a few
> > > > > > files where sanitizing has been disabled in Makefiles).
> > > > > >
> > > > > > Until the real shadow area is set, all access are granted thanks to the
> > > > > > zero shadow area beeing for of zeros.
> > > > >
> > > > > Not entirely correct. kasan_early_init() indeed maps the whole shadow
> > > > > memory range to the same kasan_early_shadow_page. However as kernel
> > > > > loads and memory gets allocated this shadow page gets rewritten with
> > > > > non-zero values by different KASAN allocator hooks. Since these values
> > > > > come from completely different parts of the kernel, but all land on
> > > > > the same page, kasan_early_shadow_page's content can be considered
> > > > > garbage. When KASAN checks memory accesses for validity it detects
> > > > > these garbage shadow values, but doesn't print any reports, as the
> > > > > reporting routine bails out on the current->kasan_depth check (which
> > > > > has the value of 1 initially). Only after kasan_init() completes, when
> > > > > the proper shadow memory is mapped, current->kasan_depth gets set to 0
> > > > > and we start reporting bad accesses.
> > > >
> > > > That's surprising, because in the early phase I map the shadow area
> > > > read-only, so I do not expect it to get modified unless RO protection is
> > > > failing for some reason.
> > >
> > > Actually it might be that the allocator hooks don't modify shadow at
> > > this point, as the allocator is not yet initialized. However stack
> > > should be getting poisoned and unpoisoned from the very start. But the
> > > generic statement that early shadow gets dirtied should be correct.
> > > Might it be that you don't use stack instrumentation?
> >
> > Yes, stack instrumentation is not used here, because shadow offset which we pass to
> > the -fasan-shadow-offset= cflag is not specified here. So the logic in scrpits/Makefile.kasan
> > just fallbacks to CFLAGS_KASAN_MINIMAL, which is outline and without stack instrumentation.
> >
> > Christophe, you can specify KASAN_SHADOW_OFFSET either in Kconfig (e.g. x86_64) or
> > in Makefile (e.g. arm64). And make early mapping writable, because compiler generated code will write
> > to shadow memory in function prologue/epilogue.
>
> Hmm. Is this limitation just that compilers have not implemented
> out-of-line support for stack instrumentation, or is there a deeper
> reason that stack/global instrumentation relies upon inline
> instrumentation?

No, it looks like as soon as we define KASAN_SHADOW_OFFSET in Makefile 
in addition to asm/kasan.h, stack instrumentation works with out-of-line.

I'll send series v5 soon.

Christophe

> I ask because it's very common on ppc64 to have the virtual address
> space split up into discontiguous blocks. I know this means we lose
> inline instrumentation, but I didn't realise we'd also lose stack and
> global instrumentation...
>
> I wonder if it would be worth, in the distant future, trying to
> implement a smarter scheme in compilers where we could insert more
> complex inline mapping schemes.
>
> Regards,
> Daniel