Re: [regression?] Re: [PATCH v6 06/12] mm/gup: track FOLL_PIN pages

[Alex Williamson]

On Tue, 28 Apr 2020 14:49:57 -0300
Jason Gunthorpe <jgg@xxxxxxxx> wrote:

> On Tue, Apr 28, 2020 at 10:54:55AM -0600, Alex Williamson wrote:
> >  static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
> >  {
> >  	struct vfio_pci_device *vdev = device_data;
> > @@ -1253,8 +1323,14 @@ static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma)
> >  	vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
> >  	vma->vm_pgoff = (pci_resource_start(pdev, index) >> PAGE_SHIFT) + pgoff;
> >  
> > +	vma->vm_ops = &vfio_pci_mmap_ops;
> > +
> > +#if 1
> > +	return 0;
> > +#else
> >  	return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
> > -			       req_len, vma->vm_page_prot);
> > +			       vma->vm_end - vma->vm_start, vma->vm_page_prot);  
> 
> The remap_pfn_range here is what tells get_user_pages this is a
> non-struct page mapping:
> 
> 	vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP;
> 
> Which has to be set when the VMA is created, they shouldn't be
> modified during fault.

Aha, thanks Jason!  So fundamentally, pin_user_pages_remote() should
never have been faulting in this vma since the pages are non-struct
page backed.  Maybe I was just getting lucky before this commit.  For a
VM_PFNMAP, vaddr_get_pfn() only needs pin_user_pages_remote() to return
error and the vma information that we setup in vfio_pci_mmap().  We
only need the fault handler to trigger for user access, which is what I
see with this change.  That should work for me.

> Also the vma code above looked a little strange to me, if you do send
> something like this cc me and I can look at it. I did some work like
> this for rdma a while ago..

Cool, I'll do that.  I'd like to be able to zap the vmas from user
access at a later point and I have doubts that I'm holding the
refs/locks that I need to for that.  Thanks,

Alex