Re: [PATCH] x86,mm: delay TLB flush after clearing accessed bit

[Rik van Riel]

On 04/01/2014 09:20 AM, Ingo Molnar wrote:
> 
> * Rik van Riel <riel@xxxxxxxxxx> wrote:
> 
>>>>  int ptep_clear_flush_young(struct vm_area_struct *vma,
>>>>  			   unsigned long address, pte_t *ptep)
>>>>  {
>>>> -	int young;
>>>> +	int young, cpu;
>>>>  
>>>>  	young = ptep_test_and_clear_young(vma, address, ptep);
>>>> -	if (young)
>>>> -		flush_tlb_page(vma, address);
>>>> +	if (young) {
>>>> +		for_each_cpu(cpu, vma->vm_mm->cpu_vm_mask_var)
>>>> +			tlb_set_force_flush(cpu);
>>>
>>> Hm, just to play the devil's advocate - what happens when we have 
>>> a va that is used on a few dozen, a few hundred or a few thousand 
>>> CPUs? Will the savings be dwarved by the O(nr_cpus_used) loop 
>>> overhead?
>>>
>>> Especially as this is touching cachelines on other CPUs and likely 
>>> creating the worst kind of cachemisses. That can really kill 
>>> performance.
>>
>> flush_tlb_page does the same O(nr_cpus_used) loop, but it sends an 
>> IPI to each CPU every time, instead of dirtying a cache line once 
>> per pageout run (or until the next context switch).
>>
>> Does that address your concern?
> 
> That depends on the platform - which could implement flush_tlb_page() 
> as a broadcast IPI - but yes, it was bad before as well, now it became 
> more visible and I noticed it :)
> 
> Wouldn't it be more scalable to use a generation count as a timestamp, 
> and set that in the mm? mm that last flushed before that timestamp 
> need to flush, or so. That gets rid of the mask logic and the loop, 
> AFAICS.

More scalable in the page eviction code, sure.

However, that would cause the context switch code to load an
additional cache line, so I am not convinced that is a good
tradeoff...

-- 
All rights reversed
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/