Re: [PATCH 1/1] mm: thp: Redefine default THP defrag behaviour disable it by default

[Andrea Arcangeli]

On Thu, Feb 25, 2016 at 05:12:19PM +0000, Mel Gorman wrote:
> some cases, this will reduce THP usage but the benefit of THP is hard to
> measure and not a universal win where as a stall to reclaim/compaction is

It depends on the workload: with virtual machines THP is essential
from the start without having to wait half a khugepaged cycle in
average, especially on large systems. We see this effect for example
in postcopy live migraiton where --postcopy-after-precopy is essential
to reach peak performance during database workloads in guest,
immediately after postcopy completes. With --postcopy-after-precopy
only those pages that may be triggering userfaults will need to be
collapsed with khugepaged and all the rest that was previously passed
over with precopy has an high probability to be immediately THP backed
also thanks to defrag/direct-compaction. Failing at starting
the destination node largely THP backed is very visible in benchmark
(even if a full precopy pass is done first). Later on the performance
increases again as khugepaged fixes things, but it takes some time.

So unless we've a very good kcompatd or a workqueue doing the job of
providing enough THP for page faults, I'm skeptical of this. If
something I'd rather set defrag to "madvise" as qemu and other loads
that critically need THP or they run literally half as slow (for
example with enterprise workloads in the guest) will still be able not
to rely solely on khugepaged which can takes some time to act. So your
benchmark would run the same but the VM could still start fully THP
backed.

Another problem is that khugepaged isn't able to collapse shared
readonly anon pages, mostly because of the rmap complexities.  I agree
with Kirill we should be looking into how make this work, although I
doubt the simpler refcounting is going to help much in this regard as
the problem is in dealing with rmap, not so much with refcounts.