Re: [PATCH 0/6] Enable parallel page migration

[Michal Hocko]

On Thu 09-03-17 15:09:04, Mel Gorman wrote:
> On Wed, Mar 08, 2017 at 09:34:27PM +0530, Anshuman Khandual wrote:
> > > Any comments, suggestions are welcome.
> > 
> > Hello Vlastimil/Michal/Minchan/Mel/Dave,
> > 
> > Apart from the comments from Naoya on a different thread posted by Zi
> > Yan, I did not get any more review comments on this series. Could you
> > please kindly have a look on the over all design and its benefits from
> > page migration performance point of view and let me know your views.
> > Thank you.
> > 
> 
> I didn't look into the patches in detail except to get a general feel
> for how it works and I'm not convinced that it's a good idea at all.
> 
> I accept that memory bandwidth utilisation may be higher as a result but
> consider the impact. THP migrations are relatively rare and when they
> occur, it's in the context of a single thread. To parallelise the copy,
> an allocation, kmap and workqueue invocation are required. There may be a
> long delay before the workqueue item can start which may exceed the time
> to do a single copy if the CPUs on a node are saturated. Furthermore, a
> single thread can preempt operations of other unrelated threads and incur
> CPU cache pollution and future misses on unrelated CPUs. It's compounded by
> the fact that a high priority system workqueue is used to do the operation,
> one that is used for CPU hotplug operations and rolling back when a netdevice
> fails to be registered. It treats a hugepage copy as an essential operation
> that can preempt all other work which is very questionable.
> 
> The series leader has no details on a workload that is bottlenecked by
> THP migrations and even if it is, the primary question should be *why*
> THP migrations are so frequent and alleviating that instead of
> preempting multiple CPUs to do the work.

FWIW I very much agree here and the follow up reply. Making migration
itself parallel is a hard task. You should start simple and optimize the
current code first and each step accompany with numbers. Parallel
migration should be the very last step - if it is needed at all of
course. I am quite skeptical that a reasonable parallel load balancing
is achievable without a large maintenance cost and/or predictable
behavior. 
-- 
Michal Hocko
SUSE Labs