Re: [PATCH 06/17] arm: mmu_gather rework
From: Russell King
Date: Mon Feb 28 2011 - 07:01:16 EST
On Mon, Feb 28, 2011 at 12:44:47PM +0100, Peter Zijlstra wrote:
> Right, so the normal case is:
The fullmm argument is important here as it specifies the mode.
> for (; vma; vma = vma->vm_next)
> tlb_start_vma() -> flush cache range
> ptep_get_and_clear_full() -> batch/track external tlbs
> tlb_remove_tlb_entry() -> batch/track external tlbs
> tlb_remove_page() -> track range/batch page
> tlb_end_vma() -> flush tlb range
tlb_finish_mmu() -> nothing
> [ for architectures that have hardware page table walkers
> concurrent faults can still load the page tables ]
> while (vma)
tlb_finish_mmu() -> flush tlb mm
> free vmas
So this is all fine. Note that we *don't* use the range stuff here.
> Now, if we want to track ranges _and_ have hardware page table walkers
> (ARM seems to be one such), we must flush TLBs at tlb_end_vma() because
> flush_tlb_range() requires a vma pointer (ARM and TILE actually use more
> than ->vm_mm), and on tlb_finish_mmu() issue a full mm wide invalidate
> because the hardware walker could have re-populated the cache after
> clearing the PTEs but before freeing the page tables.
No. The hardware walker won't re-populate the TLB after the page table
entries have been cleared - where would it get this information from if
not from the page tables?
> What ARM does is it retains the last vma pointer and tracks
> pte_free_tlb() range and uses that in tlb_finish_mmu(), which is a tad
It may be hacky but then the TLB shootdown interface is hacky too. We
don't keep the vma around to re-use after tlb_end_vma() - if you think
that then you misunderstand what's going on. The vma pointer is kept
around as a cheap way of allowing tlb_finish_mmu() to distinguish
between the unmap_region() mode and the shift_arg_pages() mode.
> Mostly because of shift_arg_pages(), where we have:
tlb_finish_mmu() does nothing without the ARM change.
tlb_finish_mmu() -> flush_tlb_mm() with the ARM change.
And this is where the bug was - these page table entries could find
their way into the TLB and persist after they've been torn down.
> For which ARM now punts and does a full tlb invalidate (no vma pointer).
> But also because it drags along that vma pointer, which might not at all
> match the range its actually going to invalidate (and hence its vm_flags
> might not accurately reflect things -- at worst more expensive than
Where do you get that from? Where exactly in the above code would the
VMA pointer get set? In this case, it will be NULL, so we do a
flush_tlb_mm() for this case. We have to - we don't have any VMA to
deal with at this point.
> The reason I wanted flush_tlb_range() to take an mm_struct and not the
> current vm_area_struct is because we can avoid doing the
> flush_tlb_range() from tlb_end_vma() and delay the thing until
> tlb_finish_mmu() without having to resort to such games as above. We
> could simply track the full range over all VMAs and free'd page-tables
> and do one range invalidate.
No. That's stupid. Consider the case where you have to loop one page
at a time over the range (we do on ARM.) If we ended up with your
suggestion above, that means we could potentially have to loop 4K at a
time over 3GB of address space. That's idiotic when we have an
instruction which can flush the entire TLB for a particular thread.
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/