答复: [PATCH V2] vfio dma_map/unmap: optimized for hugetlbfs pages

[Maoming (maoming, Cloud Infrastructure Service Product Dept.)]

-----邮件原件-----
发件人: Peter Xu [mailto:peterx@xxxxxxxxxx] 
发送时间: 2020年8月26日 4:59
收件人: Maoming (maoming, Cloud Infrastructure Service Product Dept.) <maoming.maoming@xxxxxxxxxx>
抄送: linux-kernel@xxxxxxxxxxxxxxx; kvm@xxxxxxxxxxxxxxx; alex.williamson@xxxxxxxxxx; cohuck@xxxxxxxxxx; Zhoujian (jay) <jianjay.zhou@xxxxxxxxxx>; Huangweidong (C) <weidong.huang@xxxxxxxxxx>; aarcange@xxxxxxxxxx
主题: Re: [PATCH V2] vfio dma_map/unmap: optimized for hugetlbfs pages

On Fri, Aug 14, 2020 at 10:37:29AM +0800, Ming Mao wrote:
> +static long hugetlb_page_vaddr_get_pfn(unsigned long vaddr, long npage,
> +						unsigned long pfn)
> +{
> +	long hugetlb_residual_npage;
> +	long contiguous_npage;
> +	struct page *head = compound_head(pfn_to_page(pfn));
> +
> +	/*
> +	 * If pfn is valid,
> +	 * hugetlb_residual_npage is greater than or equal to 1.
> +	 */
> +	hugetlb_residual_npage = hugetlb_get_residual_pages(vaddr,
> +						compound_order(head));
> +	if (hugetlb_residual_npage < 0)
> +		return -1;
> +
> +	/* The page of vaddr has been gotten by vaddr_get_pfn */
> +	contiguous_npage = min_t(long, (hugetlb_residual_npage - 1), npage);
> +	if (!contiguous_npage)
> +		return 0;
> +	/*
> +	 * Unlike THP, the splitting should not happen for hugetlb pages.
> +	 * Since PG_reserved is not relevant for compound pages, and the pfn of
> +	 * PAGE_SIZE page which in hugetlb pages is valid,
> +	 * it is not necessary to check rsvd for hugetlb pages.
> +	 * We do not need to alloc pages because of vaddr and we can finish all
> +	 * work by a single operation to the head page.
> +	 */
> +	atomic_add(contiguous_npage, compound_pincount_ptr(head));
> +	page_ref_add(head, contiguous_npage);
> +	mod_node_page_state(page_pgdat(head), NR_FOLL_PIN_ACQUIRED, 
> +contiguous_npage);

I think I asked this question in v1, but I didn't get any answer... So I'm trying again...

Could I ask why manual referencing of pages is done here rather than using
pin_user_pages_remote() just like what we've done with vaddr_get_pfn(), and let
try_grab_page() to do the page reference and accountings?

I feel like this at least is against the FOLL_PIN workflow of gup, because those FOLL_PIN paths were bypassed, afaict.


Hi,
My apologies for not answering your question.
As I understand, pin_user_pages_remote() might spend much time.
Because all PAGE_SIZE-pages in a hugetlb page are pinned one by one in pin_user_pages_remote() and try_grab_page().
So I think maybe we can use these simple code to do all work.
Am I wrong? And is there something else we can use? For example :pin_user_pages_fast()


> +
> +	return contiguous_npage;
> +}

--
Peter Xu