Re: [PATCH] mm: fadvise: avoid expensive remote LRU cache draining after FADV_DONTNEED

[Vlastimil Babka]

On 12/10/2016 06:26 PM, Johannes Weiner wrote:
> When FADV_DONTNEED cannot drop all pages in the range, it observes
> that some pages might still be on per-cpu LRU caches after recent
> instantiation and so initiates remote calls to all CPUs to flush their
> local caches. However, in most cases, the fadvise happens from the
> same context that instantiated the pages, and any pre-LRU pages in the
> specified range are most likely sitting on the local CPU's LRU cache,
> and so in many cases this results in unnecessary remote calls, which,
> in a loaded system, can hold up the fadvise() call significantly.

Got any numbers for this part?

> Try to avoid the remote call by flushing the local LRU cache before
> even attempting to invalidate anything. It's a cheap operation, and
> the local LRU cache is the most likely to hold any pre-LRU pages in
> the specified fadvise range.

Anyway it looks like things can't be worse after this patch, so...

> Signed-off-by: Johannes Weiner <hannes@xxxxxxxxxxx>

Acked-by: Vlastimil Babka <vbabka@xxxxxxx>

> ---
>  mm/fadvise.c | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/mm/fadvise.c b/mm/fadvise.c
> index 6c707bfe02fd..a43013112581 100644
> --- a/mm/fadvise.c
> +++ b/mm/fadvise.c
> @@ -139,7 +139,20 @@ SYSCALL_DEFINE4(fadvise64_64, int, fd, loff_t, offset, loff_t, len, int, advice)
>  		}
>
>  		if (end_index >= start_index) {
> -			unsigned long count = invalidate_mapping_pages(mapping,
> +			unsigned long count;
> +
> +			/*
> +			 * It's common to FADV_DONTNEED right after
> +			 * the read or write that instantiates the
> +			 * pages, in which case there will be some
> +			 * sitting on the local LRU cache. Try to
> +			 * avoid the expensive remote drain and the
> +			 * second cache tree walk below by flushing
> +			 * them out right away.
> +			 */
> +			lru_add_drain();
> +
> +			count = invalidate_mapping_pages(mapping,
>  						start_index, end_index);
>
>  			/*