Re: [PATCH 4/4] mm: Increase SWAP_CLUSTER_MAX to batch TLB flushes

[Andrew Morton]

On Mon,  6 Jul 2015 14:39:56 +0100 Mel Gorman <mgorman@xxxxxxx> wrote:

> Pages that are unmapped for reclaim must be flushed before being freed to
> avoid corruption due to a page being freed and reallocated while a stale
> TLB entry exists. When reclaiming mapped pages, the requires one IPI per
> SWAP_CLUSTER_MAX. This patch increases SWAP_CLUSTER_MAX to 256 so more
> pages can be flushed with a single IPI. This number was selected because
> it reduced IPIs for TLB shootdowns by 40% on a workload that is dominated
> by mapped pages.
> 
> Note that it is expected that doubling SWAP_CLUSTER_MAX would not always
> halve the IPIs as it is workload dependent. Reclaim efficiency was not 100%
> on this workload which was picked for being IPI-intensive and was closer to
> 35%. More importantly, reclaim does not always isolate in SWAP_CLUSTER_MAX
> pages. The LRU lists for a zone may be small, the priority can be low
> and even when reclaiming a lot of pages, the last isolation may not be
> exactly SWAP_CLUSTER_MAX.
> 
> There are a few potential issues with increasing SWAP_CLUSTER_MAX.
> 
> 1. LRU lock hold times increase slightly because more pages are being
>    isolated.
> 2. There are slight timing changes due to more pages having to be
>    processed before they are freed. There is a slight risk that more
>    pages than are necessary get reclaimed.
> 3. There is a risk that too_many_isolated checks will be easier to
>    trigger resulting in a HZ/10 stall.
> 4. The rotation rate of active->inactive is slightly faster but there
>    should be fewer rotations before the lists get balanced so it
>    shouldn't matter.
> 5. More pages are reclaimed in a single pass if zone_reclaim_mode is
>    active but that thing sucks hard when it's enabled no matter what
> 6. More pages are isolated for compaction so page hold times there
>    are longer while they are being copied
> 
> It's unlikely any of these will be problems but worth keeping in mind if
> there are any reclaim-related bug reports in the near future.

Yes, this may well cause small&subtle changes which will take some time
to be noticed.

What is the overall effect on the performance improvement if this patch
is omitted?

I wonder if we should leave small systems or !SMP systems or
CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH=n systems with
SWAP_CLUSTER_MAX=32.  If not, why didn't we change this years ago ;)


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/