Re: [PATCH 06/10] x86,mm: use broadcast TLB flushing for page reclaim TLB flushing

[Peter Zijlstra]

On Sat, Dec 21, 2024 at 11:06:38PM -0500, Rik van Riel wrote:
> In the page reclaim code, we only track the CPU(s) where the TLB needs
> to be flushed, rather than all the individual mappings that may be getting
> invalidated.
> 
> Use broadcast TLB flushing when that is available.
> 
> Signed-off-by: Rik van Riel <riel@xxxxxxxxxxx>
> ---
>  arch/x86/mm/tlb.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
> index bf85cd0590d5..9422b10edec1 100644
> --- a/arch/x86/mm/tlb.c
> +++ b/arch/x86/mm/tlb.c
> @@ -1313,6 +1313,12 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
>  
>  	int cpu = get_cpu();
>  
> +	if (static_cpu_has(X86_FEATURE_INVLPGB)) {
> +		invlpgb_flush_all_nonglobals();
> +		tlbsync();
> +		goto out_put_cpu;
> +	}

Urgh, move this before the get_cpu(), and write it like:

	if (static_cpu_has(X86_FEATURE_INVLPGB)) {
		guard(preempt)();
		invlpgb_flush_all_nonglobals();
		tlbsync();
		return;	
	}

?


> +
>  	info = get_flush_tlb_info(NULL, 0, TLB_FLUSH_ALL, 0, false,
>  				  TLB_GENERATION_INVALID);
>  	/*
> @@ -1332,6 +1338,7 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch)
>  	cpumask_clear(&batch->cpumask);
>  
>  	put_flush_tlb_info();
> +out_put_cpu:
>  	put_cpu();
>  }
>  
> -- 
> 2.47.1
>