Re: [RFC 05/13] x86/mm: Add barriers and document switch_mm-vs-flush synchronization

[Wanpeng Li]

Hi Andy,
2016-01-09 7:15 GMT+08:00 Andy Lutomirski <luto@xxxxxxxxxx>:
> When switch_mm activates a new pgd, it also sets a bit that tells
> other CPUs that the pgd is in use so that tlb flush IPIs will be
> sent.  In order for that to work correctly, the bit needs to be
> visible prior to loading the pgd and therefore starting to fill the
> local TLB.
>
> Document all the barriers that make this work correctly and add a
> couple that were missing.
>
> Cc: stable@xxxxxxxxxxxxxxx
> Signed-off-by: Andy Lutomirski <luto@xxxxxxxxxx>
> ---
>  arch/x86/include/asm/mmu_context.h | 33 ++++++++++++++++++++++++++++++++-
>  arch/x86/mm/tlb.c                  | 29 ++++++++++++++++++++++++++---
>  2 files changed, 58 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
> index 379cd3658799..1edc9cd198b8 100644
> --- a/arch/x86/include/asm/mmu_context.h
> +++ b/arch/x86/include/asm/mmu_context.h
> @@ -116,8 +116,34 @@ static inline void switch_mm(struct mm_struct *prev, struct mm_struct *next,
>  #endif
>                 cpumask_set_cpu(cpu, mm_cpumask(next));
>
> -               /* Re-load page tables */
> +               /*
> +                * Re-load page tables.
> +                *
> +                * This logic has an ordering constraint:
> +                *
> +                *  CPU 0: Write to a PTE for 'next'
> +                *  CPU 0: load bit 1 in mm_cpumask.  if nonzero, send IPI.
> +                *  CPU 1: set bit 1 in next's mm_cpumask
> +                *  CPU 1: load from the PTE that CPU 0 writes (implicit)
> +                *
> +                * We need to prevent an outcome in which CPU 1 observes
> +                * the new PTE value and CPU 0 observes bit 1 clear in
> +                * mm_cpumask.  (If that occurs, then the IPI will never
> +                * be sent, and CPU 0's TLB will contain a stale entry.)

I misunderstand this comments, CPU0 write to a PTE for 'next', and
CPU0 observes bit 1 clear in mm_cpumask, so CPU0 won't kick IPI to
CPU1, why CPU0's TLB will contain a stale entry instead of CPU1?

Regards,
Wanpeng Li