MAP_POPULATE vs. MADV_HUGEPAGES

From: Avi Kivity
Date: Wed Mar 15 2017 - 12:51:18 EST


A user is trying to allocate 1TB of anonymous memory in parallel on 48 cores (4 NUMA nodes). The kernel ends up spinning in isolate_freepages_block().


I thought to help it along by using MAP_POPULATE, but then my MADV_HUGEPAGE won't be seen until after mmap() completes, with pages already populated. Are MAP_POPULATE and MADV_HUGEPAGE mutually exclusive?


Is my only option to serialize those memory allocations, and fault in those pages manually? Or perhaps use mlock()?