Re: [RFC PATCH 2/4] iommu/vt-d: Add first level page table interfaces

[Lu Baolu]

Hi,

On 9/29/19 1:25 PM, Peter Xu wrote:
> On Sat, Sep 28, 2019 at 04:23:16PM +0800, Lu Baolu wrote:
> > Hi Peter,
> >
> > On 9/27/19 1:34 PM, Peter Xu wrote:
> > > Hi, Baolu,
> > >
> > > On Fri, Sep 27, 2019 at 10:27:24AM +0800, Lu Baolu wrote:
> > > > > > > > +	spin_lock(&(domain)->page_table_lock);				\
> > > > > > >
> > > > > > > Is this intended to lock here instead of taking the lock during the
> > > > > > > whole page table walk?  Is it safe?
> > > > > > >
> > > > > > > Taking the example where nm==PTE: when we reach here how do we
> > > > > > > guarantee that the PMD page that has this PTE is still valid?
> > > > > >
> > > > > > We will always keep the non-leaf pages in the table,
> > > > >
> > > > > I see.  Though, could I ask why?  It seems to me that the existing 2nd
> > > > > level page table does not keep these when unmap, and it's not even use
> > > > > locking at all by leveraging cmpxchg()?
> > > >
> > > > I still need some time to understand how cmpxchg() solves the race issue
> > > > when reclaims pages. For example.
> > > >
> > > > Thread A				Thread B
> > > > -A1: check all PTE's empty		-B1: up-level PDE valid
> > > > -A2: clear the up-level PDE
> > > > -A3: reclaim the page			-B2: populate the PTEs
> > > >
> > > > Both (A1,A2) and (B1,B2) should be atomic. Otherwise, race could happen.
> > >
> > > I'm not sure of this, but IMHO it is similarly because we need to
> > > allocate the iova ranges from iova allocator first, so thread A (who's
> > > going to unmap pages) and thread B (who's going to map new pages)
> > > should never have collapsed regions if happening concurrently.  I'm
> >
> > Although they don't collapse, they might share a same pmd entry. If A
> > cleared the pmd entry and B goes ahead with populating the pte's. It
> > will crash.
>
> My understanding is that if A was not owning all the pages on that PMD
> entry then it will never free the page that was backing that PMD
> entry.  Please refer to the code in dma_pte_clear_level() where it
> has:
>
>          /* If range covers entire pagetable, free it */
>          if (start_pfn <= level_pfn &&
>                  last_pfn >= level_pfn + level_size(level) - 1) {
>                  ...
>          } else {
>                  ...
>          }
>
> Note that when going into the else block, the PMD won't be freed but
> only the PTEs that upon the PMD will be cleared.

Exactly! Thanks for pointing this out.

I will do the same thing in v2.

> In the case you mentioned above, IMHO it should go into that else
> block.  Say, thread A must not contain the whole range of that PMD
> otherwise thread B won't get allocated with pages within that range
> covered by the same PMD.
>
> Thanks,

Best regards,
Baolu