Re: [PATCH 00/27] Move LRU page reclaim from zones to nodes v7

[Balbir Singh]

On 24/06/16 17:50, Mel Gorman wrote:
> On Fri, Jun 24, 2016 at 04:35:45PM +1000, Balbir Singh wrote:
>>> 1. The residency of a page partially depends on what zone the page was
>>>    allocated from.  This is partially combatted by the fair zone allocation
>>>    policy but that is a partial solution that introduces overhead in the
>>>    page allocator paths.
>>>
>>> 2. Currently, reclaim on node 0 behaves slightly different to node 1. For
>>>    example, direct reclaim scans in zonelist order and reclaims even if
>>>    the zone is over the high watermark regardless of the age of pages
>>>    in that LRU. Kswapd on the other hand starts reclaim on the highest
>>>    unbalanced zone. A difference in distribution of file/anon pages due
>>>    to when they were allocated results can result in a difference in 
>>>    again. While the fair zone allocation policy mitigates some of the
>>>    problems here, the page reclaim results on a multi-zone node will
>>>    always be different to a single-zone node.
>>>    it was scheduled on as a result.
>>>
>>> 3. kswapd and the page allocator scan zones in the opposite order to
>>>    avoid interfering with each other but it's sensitive to timing.  This
>>>    mitigates the page allocator using pages that were allocated very recently
>>>    in the ideal case but it's sensitive to timing. When kswapd is allocating
>>>    from lower zones then it's great but during the rebalancing of the highest
>>>    zone, the page allocator and kswapd interfere with each other. It's worse
>>>    if the highest zone is small and difficult to balance.
>>>
>>> 4. slab shrinkers are node-based which makes it harder to identify the exact
>>>    relationship between slab reclaim and LRU reclaim.
>>>
>>
>> Sorry, I am late in reading the thread and the patches, but I am trying to understand
>> the key benefits?
> 
> The key benefits were outlined at the beginning of the changelog. The
> one that is missing is the large overhead from the fair zone allocation
> policy which can be removed safely by the feature. The benefit to page
> allocator micro-benchmarks is outlined in the series introduction.

I did look at them, but between 1 to 4, it seemed like the largest benefit
was mm cleanup and better behaviour of reclaim on node 0.

> 
>> I know that
>> zones have grown to be overloaded to mean many things now. What is the contention impact
>> of moving the LRU from zone to nodes?
> 
> Expected to be minimal. On NUMA machines, most nodes have only one zone.
> On machines with multiple zones, the lock per zone is not that fine-grained
> given the size of the zones on large memory configurations.
> 

Makes sense

Thanks,
Balbir Singh.