RE: [Xen-devel] [RFC] virtio_ring: check dma_mem for xen_domain

[Peng Fan]

Hi stefano,

> -----Original Message-----
> From: Stefano Stabellini [mailto:sstabellini@xxxxxxxxxx]
> Sent: 2019å1æ24æ 7:44
> To: hch@xxxxxxxxxxxxx
> Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>; Peng Fan
> <peng.fan@xxxxxxx>; mst@xxxxxxxxxx; jasowang@xxxxxxxxxx;
> xen-devel@xxxxxxxxxxxxxxxxxxxx; linux-remoteproc@xxxxxxxxxxxxxxx;
> linux-kernel@xxxxxxxxxxxxxxx; virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx;
> luto@xxxxxxxxxx; jgross@xxxxxxxx; boris.ostrovsky@xxxxxxxxxx;
> bjorn.andersson@xxxxxxxxxx; jliang@xxxxxxxxxx
> Subject: Re: [Xen-devel] [RFC] virtio_ring: check dma_mem for xen_domain
> 
> On Wed, 23 Jan 2019, hch@xxxxxxxxxxxxx wrote:
> > On Wed, Jan 23, 2019 at 01:04:33PM -0800, Stefano Stabellini wrote:
> > > If vring_use_dma_api is actually supposed to return true when
> > > dma_dev->dma_mem is set, then both Peng's patch and the patch I
> > > wrote are not fixing the real issue here.
> > >
> > > I don't know enough about remoteproc to know where the problem
> > > actually lies though.
> >
> > The problem is the following:
> >
> > Devices can declare a specific memory region that they want to use
> > when the driver calls dma_alloc_coherent for the device, this is done
> > using the shared-dma-pool DT attribute, which comes in two variants
> > that would be a little to much to explain here.
> >
> > remoteproc makes use of that because apparently the device can only
> > communicate using that region.  But it then feeds back memory obtained
> > with dma_alloc_coherent into the virtio code.  For that it calls
> > vmalloc_to_page on the dma_alloc_coherent, which is a huge no-go for
> > the ÄMA API and only worked accidentally on a few platform, and
> > apparently arm64 just changed a few internals that made it stop
> > working for remoteproc.
> >
> > The right answer is to not use the DMA API to allocate memory from a
> > device-speficic region, but to tie the driver directly into the DT
> > reserved memory API in a way that allows it to easilt obtain a struct
> > device for it.
> 
> If I understand correctly, Peng should be able to reproduce the problem on
> native Linux without any Xen involvement simply by forcing
> vring_use_dma_api to return true. Peng, can you confirm?

It is another issue without xen involvement, 
There is an thread talking this: https://patchwork.kernel.org/patch/10742923/

Without xen, vring_use_dma_api will return false.
With xen, if vring_use_dma_api returns true, it will dma_map_xx and trigger dump.

Thanks,
Peng.

> 
> And the right fix is not to call vmalloc_to_page on a dma_alloc_coherent
> buffer -- I don't know about the recent changes on arm64, but that's not going
> to work with arm32 either AFAIK. Given that I don't have a repro, I'll leave it to
> Peng and/or others to send the appropriate patch for remoteproc.
> 
> 
> > This is orthogonal to another issue, and that is that hardware virtio
> > devices really always need to use the DMA API, otherwise we'll bypass
> > such features as the device specific DMA pools, DMA offsets, cache
> > flushing, etc, etc.
> 
> I understand, I'll drop my patch.