Re: [PATCH v3 0/2] Hugetlbfs support for riscv

[Alex Ghiti]

On 7/4/19 7:35 AM, Paul Walmsley wrote:
> On Thu, 4 Jul 2019, Alexandre Ghiti wrote:
>
> > On 7/4/19 12:57 AM, Paul Walmsley wrote:
> > > On Mon, 1 Jul 2019, Alexandre Ghiti wrote:
> > >
> > > > - libhugetlbfs testsuite on riscv64/2M:
> > > >     - brk_near_huge triggers an assert in malloc.c, does not on x86.
> > > I was able to reproduce the 2MB megapages test results on rv64 QEMU.  On a
> > > HiFive Unleashed, though, a few more tests fail:
> [ ... ]
>
> > > - One of the heapshrink tests fails ("Heap did not shrink")
> > >
> > >     # LD_PRELOAD="obj64/libhugetlbfs_privutils.so obj64/libhugetlbfs.so
> > > tests/obj64/libheapshrink.so" HUGETLB_MORECORE_SHRINK=yes
> > > HUGETLB_MORECORE=yes tests/obj64/heapshrink
> > >     Starting testcase "tests/obj64/heapshrink", pid 753
> > >     FAIL    Heap did not shrink
> > >     #
> > >
> > > Some of these may be related to the top-down mmap work, but there might be
> > > more work to do on actual hardware.
> >
> > I don't think this is related to top-down mmap layout, this test only
> > mmaps a huge page. It might be interesting to see more verbose messages
> > adding HUGETLB_VERBOSE=99 when launching the test.
> Here is the HUGETLB_VERBOSE=99 output from the above heapshrink test on an
> FU540:
>
> libhugetlbfs [(none):86]: INFO: Found pagesize 2048 kB
> libhugetlbfs [(none):86]: INFO: Parsed kernel version: [5] . [2] . [0]  [pre-release: 6]
> libhugetlbfs [(none):86]: INFO: Feature private_reservations is present in this kernel
> libhugetlbfs [(none):86]: INFO: Feature noreserve_safe is present in this kernel
> libhugetlbfs [(none):86]: INFO: Feature map_hugetlb is present in this kernel
> libhugetlbfs [(none):86]: INFO: Kernel has MAP_PRIVATE reservations.  Disabling heap prefaulting.
> libhugetlbfs [(none):86]: INFO: Kernel supports MAP_HUGETLB
> libhugetlbfs [(none):86]: INFO: HUGETLB_SHARE=0, sharing disabled
> libhugetlbfs [(none):86]: INFO: HUGETLB_NO_RESERVE=no, reservations enabled
> libhugetlbfs [(none):86]: INFO: No segments were appropriate for remapping
> libhugetlbfs [(none):86]: INFO: setup_morecore(): heapaddr = 0x2aaac00000
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(1052672) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaac00000, mapsize = 0, delta=1052672
> libhugetlbfs [(none):86]: INFO: Attempting to map 2097152 bytes
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaac00000
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(0) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=-1044480
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
> Starting testcase "tests/obj64/heapshrink", pid 86
> libhugetlbfs [(none):86]: INFO: hugetlbfs_morecore(33558528) = ...
> libhugetlbfs [(none):86]: INFO: heapbase = 0x2aaac00000, heaptop = 0x2aaad01000, mapsize = 200000, delta=32514048
> libhugetlbfs [(none):86]: INFO: Attempting to map 33554432 bytes
> libhugetlbfs [(none):86]: INFO: ... = 0x2aaad01000
> FAIL    Heap did not shrink
>
>
> This is with this hugepage configuration:
>
> # /usr/local/bin/hugeadm --pool-list
>        Size  Minimum  Current  Maximum  Default
>     2097152       64       64       64        *
> #

Ok thanks for that, but it does not say much :)

While trying to understand why it may fail on HW, I actually failed to 
reproduce the results on qemu (I did not
check the results for v3 and I recently switched from yocto to buildroot 
so I lost my configuration...).

What configuration do you use to reproduce the results on qemu ?

FYI, while playing around, I noticed that with qemu v4.0.0, 
icache_hygiene stalls whereas with
v3.1.0, it does not but I did not investigate though.

Thanks,

Alex


> - Paul