Re: [PATCH v5 1/4] riscv: Move kernel mapping to vmalloc zone

From: Benjamin Herrenschmidt
Date: Tue Jul 21 2020 - 19:13:47 EST


On Tue, 2020-07-21 at 12:05 -0700, Palmer Dabbelt wrote:
>
> * We waste vmalloc space on 32-bit systems, where there isn't a lot of it.
> * On 64-bit systems the VA space around the kernel is precious because it's the
> only place we can place text (modules, BPF, whatever).

Why ? Branch distance limits ? You can't use trampolines ?

> If we start putting
> the kernel in the vmalloc space then we either have to pre-allocate a bunch
> of space around it (essentially making it a fixed mapping anyway) or it
> becomes likely that we won't be able to find space for modules as they're
> loaded into running systems.

I dislike the kernel being in the vmalloc space (see my other email)
but I don't understand the specific issue with modules.

> * Relying on a relocatable kernel for sv48 support introduces a fairly large
> performance hit.

Out of curiosity why would relocatable kernels introduce a significant
hit ? Where about do you see the overhead coming from ?

> Roughly, my proposal would be to:
>
> * Leave the 32-bit memory map alone. On 32-bit systems we can load modules
> anywhere and we only have one VA width, so we're not really solving any
> problems with these changes.
> * Staticly allocate a 2GiB portion of the VA space for all our text, as its own
> region. We'd link/relocate the kernel here instead of around PAGE_OFFSET,
> which would decouple the kernel from the physical memory layout of the system.
> This would have the side effect of sorting out a bunch of bootloader headaches
> that we currently have.
> * Sort out how to maintain a linear map as the canonical hole moves around
> between the VA widths without adding a bunch of overhead to the virt2phys and
> friends. This is probably going to be the trickiest part, but I think if we
> just change the page table code to essentially lie about VAs when an sv39
> system runs an sv48+sv39 kernel we could make it work -- there'd be some
> logical complexity involved, but it would remain fast.
>
> This doesn't solve the problem of virtually relocatable kernels, but it does
> let us decouple that from the sv48 stuff. It also lets us stop relying on a
> fixed physical address the kernel is loaded into, which is another thing I
> don't like.
>
> I know this may be a more complicated approach, but there aren't any sv48
> systems around right now so I just don't see the rush to support them,
> particularly when there's a cost to what already exists (for those who haven't
> been watching, so far all the sv48 patch sets have imposed a significant
> performance penalty on all systems).