Re: question on lazy tlb flush

[Zhenzhong Duan]

On 2019/5/29 22:17, Rik van Riel wrote:
> On Wed, 2019-05-29 at 12:54 +0800, Zhenzhong Duan wrote:
> > Hi Maintainers,
> >
> > A question raised when I learned below code.  Appreciate any help me
> > understand the code.
> >
> > void native_flush_tlb_others(const struct cpumask *cpumask,
> >                                const struct flush_tlb_info *info)
> >
> > {
> >
> > ...
> >
> >           /*
> >            * If no page tables were freed, we can skip sending IPIs to
> >            * CPUs in lazy TLB mode. They will flush the CPU themselves
> >            * at the next context switch.
> >            *
> >            * However, if page tables are getting freed, we need to
> > send the
> >            * IPI everywhere, to prevent CPUs in lazy TLB mode from
> > tripping
> >            * up on the new contents of what used to be page tables,
> > while
> >            * doing a speculative memory access.
> >            */
> >           if (info->freed_tables)
> >                   smp_call_function_many(cpumask,
> > flush_tlb_func_remote,
> >                                  (void *)info, 1);
> >           else
> >                   on_each_cpu_cond_mask(tlb_is_not_lazy,
> > flush_tlb_func_remote,
> >                                   (void *)info, 1, GFP_ATOMIC,
> > cpumask);
> >
> > }
> >
> > I just didn't understand how a kernel thread could trip up on the
> > new
> > contents of what used to be page tables. I presume the freed page
> > tables
> > are user mapping?
> That is correct, and you ask a very good question.
>
> The software does indeed not access userspace memory
> addresses from kernel threads.
>
> However, the CPU itself can do speculative loads from
> userspace memory addresses, even when the software thread
> is running exclusively in kernel mode.
>
> Add to that the fact that the page table pages can get
> reused for something else after they have been freed, and
> that the CPU can cache intermediate translations (eg. PUD
> and PMD level things get cached in the TLB), and you might
> end up with a speculative load poking at a PCI register,
> or something else that trips up the machine.
>
> For that reason, when page table pages get freed, we need
> to flush the TLBs of all users, inluding lazy TLB kernel
> threads.

Understood. Thanks for your detailed explanation :)

Zhenzhong