Re: [PATCH 3.10 060/268] xen/gntdev: Use VM_MIXEDMAP instead of VM_IO to avoid NUMA balancing

[Willy Tarreau]

On Mon, Jun 19, 2017 at 07:57:27PM -0700, Hugh Dickins wrote:
> On Mon, 19 Jun 2017, Willy Tarreau wrote:
> 
> > From: Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>
> > 
> > commit 30faaafdfa0c754c91bac60f216c9f34a2bfdf7e upstream.
> > 
> > Commit 9c17d96500f7 ("xen/gntdev: Grant maps should not be subject to
> > NUMA balancing") set VM_IO flag to prevent grant maps from being
> > subjected to NUMA balancing.
> > 
> > It was discovered recently that this flag causes get_user_pages() to
> > always fail with -EFAULT.
> > 
> > check_vma_flags
> > __get_user_pages
> > __get_user_pages_locked
> > __get_user_pages_unlocked
> > get_user_pages_fast
> > iov_iter_get_pages
> > dio_refill_pages
> > do_direct_IO
> > do_blockdev_direct_IO
> > do_blockdev_direct_IO
> > ext4_direct_IO_read
> > generic_file_read_iter
> > aio_run_iocb
> > 
> > (which can happen if guest's vdisk has direct-io-safe option).
> > 
> > To avoid this let's use VM_MIXEDMAP flag instead --- it prevents
> > NUMA balancing just as VM_IO does and has no effect on
> > check_vma_flags().
> 
> This is only valid if kernel/sched/fair.c is checking for VM_MIXEDMAP,
> and that came in v4.1's 8e76d4eecf7a ("sched, numa: do not hint for
> NUMA balancing on VM_MIXEDMAP mappings"), which I don't see in your
> tree nor in this series (please double check, I may have missed it).
> 
> It would be good to have that one in too, and it was marked for
> stable; but maybe it didn't apply, because of depending on another
> commit adding the is_vm_hugetlb_page(vma) check there?  Which I
> expect would also be good to have, but I haven't looked it up.
> 
> Maybe drop this one for this round, and gather up its dependencies
> for the next round.

Yep that's what I'm going to do, thanks for the details. I've found
that I'll simply have to pick 6b79c57b92 and 8e76d4eecf. I won't
take the onse adding vma_policy_mof() that late in the cycle as
apparently it's only about a performance regression.

> Ben's 3.16 tree appeared to be in the same position,
> I didn't look at the EOL 3.18.

I picked this one by reviewing what patches from 3.12 were missing in
3.10 and 3.12 had the same issue. We've probably lost this along the
chain of backports.

> (I've not yet checked through backports of the "larger stack guard gap"
> - thank you all for those - will do so, but won't get through them
> tonight - I must look into DaveJ's trinity VM_BUG_ON now.)

No pb, you're welcome. At least they didn't prevent Guenter's machines
from booting on 82 platforms, which is a good start ;-)

Willy