Re: [patch] mm, thp: always direct reclaim for MADV_HUGEPAGE even when deferred

[Michal Hocko]

On Tue 27-12-16 13:36:54, David Rientjes wrote:
[...]
> All madvised VMAs stall now because THEY WANT TO STALL.  It is 
> unbelievable that you would claim otherwise or think that you know better 
> than the application writer about their application.

I do care more about _users_ and their _experience_ than what
application _writers_ think is the best. This is the whole point
of giving the defrag tunable. madvise(MADV_HUGEPAGE) is just a hint to
the system that using transparent hugepages is _preferable_, not
mandatory. We have an option to allow stalls for those vmas to increase
the allocation success rate. We also have tunable to completely ignore
it. And we should also have an option to not stall.

Your interpretation of what is the expected stalling behavior of
MADV_HUGEPAGE doesn't match what the man page says:

: MADV_HUGEPAGE (since Linux 2.6.38)
: Enable Transparent Huge Pages (THP) for pages in the range specified
: by addr and length.  Currently, Transparent Huge Pages work only with
: private anonymous pages (see mmap(2)).  The kernel will regularly
: scan the areas marked as huge page candidates to replace them with
: huge pages.  The kernel will also allocate huge pages directly
: when the region is naturally aligned to the huge page size (see
: posix_memalign(2)).
: 
: This feature is primarily aimed at applications that use large mappings
: of data and access large regions of that memory at a time (e.g.,
: virtualization systems such as QEMU).  It can very easily waste memory
: (e.g., a 2MB mapping that only ever accesses 1 byte will result in 2MB
: of wired memory instead of one 4KB page).  See the Linux kernel source
: file Documentation/vm/transhuge.txt for more details.
: 
: The MADV_HUGEPAGE and MADV_NOHUGEPAGE operations are available only if
: the kernel was configured with CONFIG_TRANSPARENT_HUGEPAGE.

while it clearly contradicts what "defer" option is documented to
provide in  Documentation/vm/transhuge.txt:

: "defer" means that an application will wake kswapd in the background
: to reclaim pages and wake kcompact to compact memory so that THP is
: available in the near future. It's the responsibility of khugepaged
: to then install the THP pages later.

I am not going to reply to other your claims because I would just repeat
myself.
-- 
Michal Hocko
SUSE Labs