Re: [Xen-devel] [PATCH] grant-table: don't set m2p override ifkmap_ops is not set

[Matt Wilson]

On Tue, Nov 05, 2013 at 02:09:53PM -0800, Anthony Liguori wrote:
> On Tue, Nov 5, 2013 at 1:16 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@xxxxxxxxxx> wrote:
> > On Tue, Nov 05, 2013 at 12:53:17PM -0800, Anthony Liguori wrote:
> >> Matt Wilson <msw@xxxxxxxxx> writes:
> >>
> >> > On Tue, Nov 05, 2013 at 05:03:58PM +0100, Roger Pau Monné wrote:
> >> >> On 05/11/13 16:08, Ian Campbell wrote:
> >> >> > On Tue, 2013-11-05 at 16:01 +0100, Roger Pau Monné wrote:
> >> >> >> On 05/11/13 15:56, Konrad Rzeszutek Wilk wrote:
> >> >> >>> On Tue, Nov 05, 2013 at 03:47:08PM +0100, Roger Pau Monné wrote:
> >> >> >>>> On 05/11/13 13:36, David Vrabel wrote:
> >> >> >>>>> On 05/11/13 11:24, Roger Pau Monne wrote:
> >> >> >>>>>> IMHO there's no reason to set a m2p override if the mapping is done in
> >> >> >>>>>> kernel space, so only set the m2p override when kmap_ops is set.
> >> >> >>>>>
> >> >> >>>>> Can you provide a more detailed reasoning about why this is safe?
> >> >> >>>>
> >> >> >>>> To tell the truth, I don't understand why we need to use the m2p
> >> >> >>>> override for kernel space only mappings, my understanding is that this
> >> >> >>>> m2p override is needed for user space mappings only (where we actually
> >> >> >>>> end up doing two mappings, one in kernel space and one in user space).
> >> >> >>>> For kernel space I don't see why we need to do anything else than
> >> >> >>>> setting the right p2m translation.
> >> >> >>>
> >> >> >>> We needed the m2p when doing DMA operations. As the driver would
> >> >> >>> want the bus address (so p2m) and then when unmapping the DMA we
> >> >> >>> only get the bus address  - so we needed to do a m2p lookup.
> >> >> >>
> >> >> >> OK, we need a m2p (that we already have in machine_to_phys_mapping),
> >> >> >> what I don't understand is why we need the m2p override.
> >> >> >
> >> >> > The m2p is a host global table.
> >> >> >
> >> >> > For a foreign page grant mapped into the current domain the m2p will
> >> >> > give you the foreign (owner) domain's p from the m, not the local one.
> >> >>
> >> >> Yes, you are completely right, then I have to figure out why blkback
> >> >> works fine with this patch applied (or at least it seems to work fine).
> >> >
> >> > blkback also works for me when testing a similar patch. I'm still
> >> > confused. One thing with your proposed patch: I'm not sure that you're
> >> > putting back the correct mfn.
> >>
> >> It's perfectly fine to store a foreign pfn in the m2p table.  The m2p
> >> override table is used by the grant device to allow a reverse lookup of
> >> the real mfn to a pfn even if it's foreign.
> >>
> >> blkback doesn't actually need this though.  This was introduced in:
> >>
> >> commit 5dc03639cc903f887931831d69895facb5260f4b
> >> Author: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
> >> Date:   Tue Mar 1 16:46:45 2011 -0500
> >>
> >>     xen/blkback: Utilize the M2P override mechanism for GNTMAP_host_map
> >>
> >> Purely as an optimization.  In practice though due to lock contention it
> >> slows things down.
> >>
> >> I think an alternative would be to use a read/write lock instead of just
> >> a spinlock since it's the read path that is the most hot.
> >
> > The m2p hash table can also be expanded to lower the contention.
> 
> That was the first thing I tried :-)
> 
> The issue isn't lookup cost as much as the number of acquisitions and
> the cost of adding/removing entries.
> 
> >> I haven't tested that yet though.
> >
> > Looking forward to your patches :-)
> 
> I'm really just being thorough in suggesting the rwlock.  I actually
> think that not using the m2p override table is the right long term
> fix.

I tried a rwlock a while back. It didn't help much because we actually
have a problem with concurrent adds/removes as much as lookups.

--msw
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/