Re: [External] Re: [PATCH v2] riscv: mm: Fix alignment of phys_ram_base

[Alexandre Ghiti]

Hi Xu,

On 06/12/2024 04:11, Xu Lu wrote:
> Hi Alex,
>
> On Thu, Dec 5, 2024 at 10:34 PM Alexandre Ghiti <alex@xxxxxxxx> wrote:
> > Hi Xu,
> >
> > On 03/12/2024 15:49, Xu Lu wrote:
> > > This commit fixes the alignment of phys_ram_base in RISC-V.
> > >
> > > In sparse vmemmap model, the virtual address of vmemmap is calculated as:
> > > ((struct page *)VMEMMAP_START - (phys_ram_base >> PAGE_SHIFT)).
> > > And the struct page's va can be calculated with an offset:
> > > (vmemmap + (pfn)).
> > >
> > > However, when initializing struct pages, kernel actually starts from the
> > > first page from the same section that phys_ram_base belongs to. If the
> > > first page's physical address is not (phys_ram_base >> PAGE_SHIFT), then
> > > we get an va below VMEMMAP_START when calculating va for it's struct page.
> > >
> > > For example, if phys_ram_base starts from 0x82000000 with pfn 0x82000, the
> > > first page in the same section is actually pfn 0x80000. During
> > > init_unavailable_range(), we will initialize struct page for pfn 0x80000
> > > with virtual address ((struct page *)VMEMMAP_START - 0x2000), which is
> > > below VMEMMAP_START as well as PCI_IO_END.
> > >
> > > This commit fixes this bug by aligning phys_ram_base with SECTION_SIZE.
> > >
> > > Fixes: c3bcc65d4d2e ("riscv: Start of DRAM should at least be aligned on PMD size for the direct mapping")
> > > Signed-off-by: Xu Lu <luxu.kernel@xxxxxxxxxxxxx>
> > > ---
> > >    arch/riscv/mm/init.c | 15 ++++++++++++++-
> > >    1 file changed, 14 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c
> > > index 0e8c20adcd98..974cafa7c85e 100644
> > > --- a/arch/riscv/mm/init.c
> > > +++ b/arch/riscv/mm/init.c
> > > @@ -33,6 +33,9 @@
> > >    #include <asm/pgtable.h>
> > >    #include <asm/sections.h>
> > >    #include <asm/soc.h>
> > > +#ifdef CONFIG_SPARSEMEM
> > > +#include <asm/sparsemem.h>
> > > +#endif
> > >    #include <asm/tlbflush.h>
> > >
> > >    #include "../kernel/head.h"
> > > @@ -59,6 +62,12 @@ EXPORT_SYMBOL(pgtable_l4_enabled);
> > >    EXPORT_SYMBOL(pgtable_l5_enabled);
> > >    #endif
> > >
> > > +#ifdef CONFIG_SPARSEMEM
> > > +#define RISCV_MEMSTART_ALIGN (1UL << SECTION_SIZE_BITS)
> > > +#else
> > > +#define RISCV_MEMSTART_ALIGN PMD_SIZE
> > > +#endif
> > > +
> > >    phys_addr_t phys_ram_base __ro_after_init;
> > >    EXPORT_SYMBOL(phys_ram_base);
> > >
> > > @@ -239,9 +248,13 @@ static void __init setup_bootmem(void)
> > >        /*
> > >         * Make sure we align the start of the memory on a PMD boundary so that
> > >         * at worst, we map the linear mapping with PMD mappings.
> > > +      *
> > > +      * Also, make sure we align the start of the memory on a SECTION boundary
> > > +      * when CONFIG_SPARSEMEM_VMEMMAP is enabled to ensure the correctness of
> > > +      * pfn_to_page().
> > >         */
> > >        if (!IS_ENABLED(CONFIG_XIP_KERNEL))
> > > -             phys_ram_base = memblock_start_of_DRAM() & PMD_MASK;
> > > +             phys_ram_base = round_down(memblock_start_of_DRAM(), RISCV_MEMSTART_ALIGN);
> > >
> > >        /*
> > >         * In 64-bit, any use of __va/__pa before this point is wrong as we
> >
> > That's a good catch indeed. But I'm wondering if it would be more
> > correct to fix the macro vmemmap instead of phys_ram_base since
> > phys_ram_base is supposed to hold the real base of the system memory,
> > which would be wrong with your patch. I mean something like that instead
> > (or similar, I haven't tested):
> >
> > #define vmemmap         ((struct page *)VMEMMAP_START -
> > (round_down(memblock_start_of_DRAM(), RISCV_MEMSTART_ALIGN) >> PAGE_SHIFT))
> Thanks for your comment.
>
> Good idea. I have thought about this. But I wasn't sure if it's OK to
> introduce extra calculation whenever pfn_to_page() and page_to_pfn()
> is called. So I referred to ARM which aligns memstart_addr with
> SECTION size too and then made a similar modification.
>
> If it is not appropriate to change the semantics of phys_ram_base, how
> about introducing a new variable vmemmap_start_addr and use it to
> calculate vmemmap:
>
> #define vmemmap         ((struct page *)VMEMMAP_START -
> (vmemmap_start_addr >> PAGE_SHIFT))


I agree, that's a good idea to introduce vmemmap_start_addr.

Thanks,

Alex


> Best Regards,
>
> Xu Lu
>
> > And the fixes tag should be:
> >
> > Fixes: a11dd49dcb93 ("riscv: Sparse-Memory/vmemmap out-of-bounds fix")
> >
> > Thanks,
> >
> > Alex
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@xxxxxxxxxxxxxxxxxxx
> http://lists.infradead.org/mailman/listinfo/linux-riscv