Re: [PATCH v4 2/2] RISC-V: Implement sparsemem

[Greentime Hu]

Hi Logan,

Logan Gunthorpe <logang@xxxxxxxxxxxx> æ 2019å8æ12æ éä äå11:52åéï
>
>
>
> On 2019-08-11 10:01 p.m., Greentime Hu wrote:
> > Hi Logan,
> >
> > Logan Gunthorpe <logang@xxxxxxxxxxxx> æ 2019å8æ10æ éå äå3:03åéï
> >>
> >>
> >>
> >> On 2019-08-09 11:01 a.m., Greentime Hu wrote:
> >>> Hi Logan,
> >>>
> >>> Logan Gunthorpe <logang@xxxxxxxxxxxx> æ 2019å8æ9æ éä äå11:47åéï
> >>>>
> >>>>
> >>>>
> >>>> On 2019-08-08 10:23 p.m., Greentime Hu wrote:
> >>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >>>>> index 3f12b069af1d..208b3e14ccd8 100644
> >>>>> --- a/arch/riscv/Kconfig
> >>>>> +++ b/arch/riscv/Kconfig
> >>>>> @@ -116,7 +116,8 @@ config PGTABLE_LEVELS
> >>>>>         default 2
> >>>>>
> >>>>>  config HAVE_ARCH_PFN_VALID
> >>>>> -       def_bool y
> >>>>> +       bool
> >>>>> +       default !SPARSEMEM_VMEMMAP
> >>>>>
> >>>>>  menu "Platform type"
> >>>>>
> >>>>> diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h
> >>>>> index 8ddb6c7fedac..6991f7a5a4a7 100644
> >>>>> --- a/arch/riscv/include/asm/page.h
> >>>>> +++ b/arch/riscv/include/asm/page.h
> >>>>> @@ -93,16 +93,20 @@ extern unsigned long min_low_pfn;
> >>>>>  #define virt_to_pfn(vaddr)     (phys_to_pfn(__pa(vaddr)))
> >>>>>  #define pfn_to_virt(pfn)       (__va(pfn_to_phys(pfn)))
> >>>>>
> >>>>> +#if !defined(CONFIG_SPARSEMEM_VMEMMAP)
> >>>>> +#define pfn_valid(pfn) \
> >>>>> +       (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))
> >>>>>  #define virt_to_page(vaddr)    (pfn_to_page(virt_to_pfn(vaddr)))
> >>>>>  #define page_to_virt(page)     (pfn_to_virt(page_to_pfn(page)))
> >>>>> +#else
> >>>>> +#define virt_to_page(vaddr)    ((struct page *)((((u64)vaddr -
> >>>>> va_pa_offset) / PAGE_SIZE) * sizeof(struct page) + VMEMMAP_START))
> >>>>> +#define page_to_virt(pg)       ((void *)(((((u64)pg - VMEMMAP_START) /
> >>>>> sizeof(struct page)) * PAGE_SIZE) + va_pa_offset))
> >>>>> +#endif
> >>>>
> >>>> This doesn't make sense to me at all. It should always use pfn_to_page()
> >>>> for virt_to_page() and the generic pfn_to_page()/page_to_pfn()
> >>>> implementations essentially already do what you are doing in a cleaner
> >>>> way. So I'd be really surprised if this does anything at all.
> >>>>
> >>>
> >>> Thank you for point me out that. I just checked the generic
> >>> implementation and I should use that one.
> >>> Sorry I didn't check the generic one and just implement it again.
> >>> I think the only patch we need is the first part to use generic
> >>> pfn_valid(). I just tested it and yes it can boot successfully in dts
> >>> with hole.
> >>>
> >>> It will fail in this check ((pfn)-pfn_base) < max_mapnr.
> >>
> >> Sounds to me like max_mapnr is not set correctly. See the code in
> >> setup_bootmem(). Seems like 'mem_size' should be set to the largest
> >> memory block, not just the one that contains the kernel...
> >>
> >>
> >>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >>> index 3f12b069af1d..208b3e14ccd8 100644
> >>> --- a/arch/riscv/Kconfig
> >>> +++ b/arch/riscv/Kconfig
> >>> @@ -116,7 +116,8 @@ config PGTABLE_LEVELS
> >>>         default 2
> >>>
> >>>  config HAVE_ARCH_PFN_VALID
> >>> -       def_bool y
> >>> +       bool
> >>> +       default !SPARSEMEM_VMEMMAP
> >>>
> >>>  menu "Platform type"
> >>>
> >>> diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h
> >>> index 8ddb6c7fedac..80d28fa1e2eb 100644
> >>> --- a/arch/riscv/include/asm/page.h
> >>> +++ b/arch/riscv/include/asm/page.h
> >>> @@ -100,8 +100,10 @@ extern unsigned long min_low_pfn;
> >>>  #define page_to_bus(page)      (page_to_phys(page))
> >>>  #define phys_to_page(paddr)    (pfn_to_page(phys_to_pfn(paddr)))
> >>>
> >>> +#if !defined(CONFIG_SPARSEMEM_VMEMMAP)
> >>>  #define pfn_valid(pfn) \
> >>>         (((pfn) >= pfn_base) && (((pfn)-pfn_base) < max_mapnr))
> >>> +#endif
> >>>
> >>>  #define ARCH_PFN_OFFSET                (pfn_base)
> >>
> >>
> >> This patch still makes no sense. I'm not sure why we have an arch
> >> specific pfn_valid() because it's very similar to the generic one. But
> >> my guess is there's a reason for it and it's not doing what it is
> >> supposed when you remove it for the sparsemem case.
> >
> > It will use another pfn_valid() implementation in
> > include/linux/mmzone.h if CONFIG_SPARSEMEM and
> > !CONFIG_HAVE_ARCH_PFN_VALID
> > It will be this one.
> >
> > static inline int pfn_valid(unsigned long pfn)
> > {
> >         if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS)
> >                 return 0;
> >         return valid_section(__nr_to_section(pfn_to_section_nr(pfn)));
> > }
>
> Ah, ok I see. "page.h" is only included in no-mmu arches. Which explains
> why riscv re-implements that macro. Couple follow up questions then:
>
> * Did you test the memory-with-hole scenario without the sparsemem
> patches? It seems pfn_valid() will be wrong regardless of sparse/flat mem.
>
> * Any chance we can just use the generic pfn_valid() function in all
> cases not just sparsemem? Can you test that?
>

I think  flat mem doesn't support memory-with-hole scenario.
In mm/Kconfig, it says
"
          For systems that have holes in their physical address
          spaces and for features like NUMA and memory hotplug,
          choose "Sparse Memory"
"
IMHO, the memory-with-hole scenario should only be tested for sparse
mem but flat mem.

The generic pfn_valid() is just for non-mmu arches. Every architecture
with mmu defines their own pfn_valid().
This is supposed to be another separate patch that do we need to
implement a generic pfn_valid().