Re: [RFC PATCH 4/7] drm/ttm: Support huge pagefaults

[Thomas HellstrÃm (VMware)]

On 11/27/19 10:12 AM, Christian KÃnig wrote:
> Am 27.11.19 um 09:31 schrieb Thomas HellstrÃm (VMware):
> > From: Thomas Hellstrom <thellstrom@xxxxxxxxxx>
> >
> > Support huge (PMD-size and PUD-size) page-table entries by providing a
> > huge_fault() callback.
> > We still support private mappings and write-notify by splitting the huge
> > page-table entries on write-access.
> >
> > Note that for huge page-faults to occur, either the kernel needs to be
> > compiled with trans-huge-pages always enabled, or the kernel needs to be
> > compiled with trans-huge-pages enabled using madvise, and the user-space
> > app needs to call madvise() to enable trans-huge pages on a per-mapping
> > basis.
> >
> > Furthermore huge page-faults will not occur unless buffer objects and
> > user-space addresses are aligned on huge page size boundaries.
> >
> > Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> > Cc: Michal Hocko <mhocko@xxxxxxxx>
> > Cc: "Matthew Wilcox (Oracle)" <willy@xxxxxxxxxxxxx>
> > Cc: "Kirill A. Shutemov" <kirill.shutemov@xxxxxxxxxxxxxxx>
> > Cc: Ralph Campbell <rcampbell@xxxxxxxxxx>
> > Cc: "JÃrÃme Glisse" <jglisse@xxxxxxxxxx>
> > Cc: "Christian KÃnig" <christian.koenig@xxxxxxx>
> > Signed-off-by: Thomas Hellstrom <thellstrom@xxxxxxxxxx>
> > ---
> > Â drivers/gpu/drm/ttm/ttm_bo_vm.c | 139 +++++++++++++++++++++++++++++++-
> > Â include/drm/ttm/ttm_bo_api.hÂÂÂ |ÂÂ 3 +-
> > Â 2 files changed, 138 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c 
> > b/drivers/gpu/drm/ttm/ttm_bo_vm.c
> > index 2098f8d4dfc5..8d6089880e39 100644
> > --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
> > +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
> > @@ -150,6 +150,84 @@ vm_fault_t ttm_bo_vm_reserve(struct 
> > ttm_buffer_object *bo,
> > Â }
> > Â EXPORT_SYMBOL(ttm_bo_vm_reserve);
> > Â +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> > +/**
> > + * ttm_bo_vm_insert_huge - Insert a pfn for PUD or PMD faults
> > + * @vmf: Fault data
> > + * @bo: The buffer object
> > + * @page_offset: Page offset from bo start
> > + * @fault_page_size: The size of the fault in pages.
> > + * @pgprot: The page protections.
> > + * Does additional checking whether it's possible to insert a PUD or 
> > PMD
> > + * pfn and performs the insertion.
> > + *
> > + * Return: VM_FAULT_NOPAGE on successful insertion, 
> > VM_FAULT_FALLBACK if
> > + * a huge fault was not possible, and a VM_FAULT_ERROR code otherwise.
> > + */
> > +static vm_fault_t ttm_bo_vm_insert_huge(struct vm_fault *vmf,
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ struct ttm_buffer_object *bo,
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ pgoff_t page_offset,
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ pgoff_t fault_page_size,
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ pgprot_t pgprot)
> > +{
> > +ÂÂÂ pgoff_t i;
> > +ÂÂÂ vm_fault_t ret;
> > +ÂÂÂ unsigned long pfn;
> > +ÂÂÂ pfn_t pfnt;
> > +ÂÂÂ struct ttm_tt *ttm = bo->ttm;
> > +ÂÂÂ bool write = vmf->flags & FAULT_FLAG_WRITE;
> > +
> > +
> > +ÂÂÂ /* Fault should not cross bo boundary */
> > +ÂÂÂ page_offset &= ~(fault_page_size - 1);
> > +ÂÂÂ if (page_offset + fault_page_size > bo->num_pages)
> > +ÂÂÂÂÂÂÂ goto out_fallback;
> > +
> > +ÂÂÂ if (bo->mem.bus.is_iomem)
> > +ÂÂÂÂÂÂÂ pfn = ttm_bo_io_mem_pfn(bo, page_offset);
> > +ÂÂÂ else
> > +ÂÂÂÂÂÂÂ pfn = page_to_pfn(ttm->pages[page_offset]);
> > +
> > +ÂÂÂ /* pfn must be fault_page_size aligned. */
> > +ÂÂÂ if ((pfn & (fault_page_size - 1)) != 0)
> > +ÂÂÂÂÂÂÂ goto out_fallback;
> > +
> > +ÂÂÂ /* IO memory is OK now, TT memory must be contigous. */
>
> That won't work correctly, IO mem might not be contiguous either.
>
> We either need to call ttm_bo_io_mem_pfn() multiple times and check 
> that the addresses are linear or return the length additional to the pfn.

Yes, you're right. Will fix that up.

Thanks,

Thomas



> Regards,
> Christian.