Re: [RFC PATCH 00/22] Enhance VHOST to enable SoC-to-SoC communication

[Jason Wang]

On 2020/7/7 äå10:45, Kishon Vijay Abraham I wrote:
> Hi Jason,
>
> On 7/7/2020 3:17 PM, Jason Wang wrote:
> > On 2020/7/6 äå5:32, Kishon Vijay Abraham I wrote:
> > > Hi Jason,
> > >
> > > On 7/3/2020 12:46 PM, Jason Wang wrote:
> > > > On 2020/7/2 äå9:35, Kishon Vijay Abraham I wrote:
> > > > > Hi Jason,
> > > > >
> > > > > On 7/2/2020 3:40 PM, Jason Wang wrote:
> > > > > > On 2020/7/2 äå5:51, Michael S. Tsirkin wrote:
> > > > > > > On Thu, Jul 02, 2020 at 01:51:21PM +0530, Kishon Vijay Abraham I wrote:
> > > > > > > > This series enhances Linux Vhost support to enable SoC-to-SoC
> > > > > > > > communication over MMIO. This series enables rpmsg communication between
> > > > > > > > two SoCs using both PCIe RC<->EP and HOST1-NTB-HOST2
> > > > > > > >
> > > > > > > > 1) Modify vhost to use standard Linux driver model
> > > > > > > > 2) Add support in vring to access virtqueue over MMIO
> > > > > > > > 3) Add vhost client driver for rpmsg
> > > > > > > > 4) Add PCIe RC driver (uses virtio) and PCIe EP driver (uses vhost) for
> > > > > > > >  ÂÂÂÂÂ rpmsg communication between two SoCs connected to each other
> > > > > > > > 5) Add NTB Virtio driver and NTB Vhost driver for rpmsg communication
> > > > > > > >  ÂÂÂÂÂ between two SoCs connected via NTB
> > > > > > > > 6) Add configfs to configure the components
> > > > > > > >
> > > > > > > > UseCase1 :
> > > > > > > >
> > > > > > > >  ÂÂÂ VHOST RPMSGÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ VIRTIO RPMSG
> > > > > > > >  ÂÂÂÂÂÂÂÂ +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +
> > > > > > > >  ÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > > +-----v------+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +------v-------+
> > > > > > > > |ÂÂ LinuxÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ LinuxÂÂÂ |
> > > > > > > > | Endpoint | | Root Complex |
> > > > > > > > |ÂÂÂÂÂÂÂÂÂÂÂ <----------------->ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > > |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > > |ÂÂÂ SOC1ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ SOC2ÂÂÂÂ |
> > > > > > > > +------------+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +--------------+
> > > > > > > >
> > > > > > > > UseCase 2:
> > > > > > > >
> > > > > > > >  ÂÂÂÂÂÂÂ VHOST RPMSGÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ VIRTIO RPMSG
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂ +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂ +------v------+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +------v------+
> > > > > > > >  ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂ |ÂÂÂ HOST1ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ HOST2ÂÂÂ |
> > > > > > > >  ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂ +------^------+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +------^------+
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > > +---------------------------------------------------------------------+
> > > > > > > > |Â +------v------+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +------v------+Â |
> > > > > > > > |Â |ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |Â |
> > > > > > > > |Â |ÂÂÂÂ EPÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ EPÂÂÂÂÂ |Â |
> > > > > > > > |Â | CONTROLLER1 |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | CONTROLLER2 |Â |
> > > > > > > > |Â |ÂÂÂÂÂÂÂÂÂÂÂÂ <----------------------------------->ÂÂÂÂÂÂÂÂÂÂÂÂ |Â |
> > > > > > > > |Â |ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |Â |
> > > > > > > > |Â |ÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |Â |
> > > > > > > > |Â |ÂÂÂÂÂÂÂÂÂÂÂÂ |Â SoC With Multiple EP InstancesÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ |Â |
> > > > > > > > |Â |ÂÂÂÂÂÂÂÂÂÂÂÂ |Â (Configured using NTB Function)Â |ÂÂÂÂÂÂÂÂÂÂÂÂ |Â |
> > > > > > > > |Â +-------------+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ +-------------+Â |
> > > > > > > > +---------------------------------------------------------------------+
> > > > > > > >
> > > > > > > > Software Layering:
> > > > > > > >
> > > > > > > > The high-level SW layering should look something like below. This series
> > > > > > > > adds support only for RPMSG VHOST, however something similar should be
> > > > > > > > done for net and scsi. With that any vhost device (PCI, NTB, Platform
> > > > > > > > device, user) can use any of the vhost client driver.
> > > > > > > >
> > > > > > > >
> > > > > > > >  ÂÂÂÂÂÂ +----------------+Â +-----------+Â +------------+Â +----------+
> > > > > > > >  ÂÂÂÂÂÂ |Â RPMSG VHOSTÂÂ |Â | NET VHOST |Â | SCSI VHOST |Â |ÂÂÂ XÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂ +-------^--------+Â +-----^-----+Â +-----^------+Â +----^-----+
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > > +-----------v-----------------v--------------v--------------v----------+
> > > > > > > > |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ VHOST COREÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > > +--------^---------------^--------------------^------------------^-----+
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > >  ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
> > > > > > > > +--------v-------+Â +----v------+Â +----------v----------+Â +----v-----+
> > > > > > > > |Â PCI EPF VHOST |Â | NTB VHOST |Â |PLATFORM DEVICE VHOST|Â |ÂÂÂ XÂÂÂÂ |
> > > > > > > > +----------------+Â +-----------+Â +---------------------+Â +----------+
> > > > > > > >
> > > > > > > > This was initially proposed here [1]
> > > > > > > >
> > > > > > > > [1] ->
> > > > > > > > https://lore.kernel.org/r/2cf00ec4-1ed6-f66e-6897-006d1a5b6390@xxxxxx
> > > > > > > I find this very interesting. A huge patchset so will take a bit
> > > > > > > to review, but I certainly plan to do that. Thanks!
> > > > > > Yes, it would be better if there's a git branch for us to have a look.
> > > > > I've pushed the branch
> > > > > https://github.com/kishon/linux-wip.git vhost_rpmsg_pci_ntb_rfc
> > > > Thanks
> > > >
> > > >
> > > > > > Btw, I'm not sure I get the big picture, but I vaguely feel some of the
> > > > > > work is
> > > > > > duplicated with vDPA (e.g the epf transport or vhost bus).
> > > > > This is about connecting two different HW systems both running Linux and
> > > > > doesn't necessarily involve virtualization.
> > > > Right, this is something similar to VOP
> > > > (Documentation/misc-devices/mic/mic_overview.rst). The different is the
> > > > hardware I guess and VOP use userspace application to implement the device.
> > > I'd also like to point out, this series tries to have communication between two
> > > SoCs in vendor agnostic way. Since this series solves for 2 usecases (PCIe
> > > RC<->EP and NTB), for the NTB case it directly plugs into NTB framework and any
> > > of the HW in NTB below should be able to use a virtio-vhost communication
> > >
> > > #ls drivers/ntb/hw/
> > > amd epf idt intel mscc
> > >
> > > And similarly for the PCIe RC<->EP communication, this adds a generic endpoint
> > > function driver and hence any SoC that supports configurable PCIe endpoint can
> > > use virtio-vhost communication
> > >
> > > # ls drivers/pci/controller/dwc/*ep*
> > > drivers/pci/controller/dwc/pcie-designware-ep.c
> > > drivers/pci/controller/dwc/pcie-uniphier-ep.c
> > > drivers/pci/controller/dwc/pci-layerscape-ep.c
> >
> > Thanks for those backgrounds.
> >
> >
> > > > >  ÂÂ So there is no guest or host as in
> > > > > virtualization but two entirely different systems connected via PCIe cable,
> > > > > one
> > > > > acting as guest and one as host. So one system will provide virtio
> > > > > functionality reserving memory for virtqueues and the other provides vhost
> > > > > functionality providing a way to access the virtqueues in virtio memory.
> > > > > One is
> > > > > source and the other is sink and there is no intermediate entity. (vhost was
> > > > > probably intermediate entity in virtualization?)
> > > > (Not a native English speaker) but "vhost" could introduce some confusion for
> > > > me since it was use for implementing virtio backend for userspace drivers. I
> > > > guess "vringh" could be better.
> > > Initially I had named this vringh but later decided to choose vhost instead of
> > > vringh. vhost is still a virtio backend (not necessarily userspace) though it
> > > now resides in an entirely different system. Whatever virtio is for a frontend
> > > system, vhost can be that for a backend system. vring can be for accessing
> > > virtqueue and can be used either in frontend or backend.
> >
> > Ok.
> >
> >
> > > > > > Have you considered to implement these through vDPA?
> > > > > IIUC vDPA only provides an interface to userspace and an in-kernel rpmsg
> > > > > driver
> > > > > or vhost net driver is not provided.
> > > > >
> > > > > The HW connection looks something like https://pasteboard.co/JfMVVHC.jpg
> > > > > (usecase2 above),
> > > > I see.
> > > >
> > > >
> > > > >  ÂÂ all the boards run Linux. The middle board provides NTB
> > > > > functionality and board on either side provides virtio/vhost functionality and
> > > > > transfer data using rpmsg.
> > > > So I wonder whether it's worthwhile for a new bus. Can we use the existed
> > > > virtio-bus/drivers? It might work as, except for the epf transport, we can
> > > > introduce a epf "vhost" transport driver.
> > > IMHO we'll need two buses one for frontend and other for backend because the
> > > two components can then co-operate/interact with each other to provide a
> > > functionality. Though both will seemingly provide similar callbacks, they are
> > > both provide symmetrical or complimentary funcitonality and need not be same or
> > > identical.
> > >
> > > Having the same bus can also create sequencing issues.
> > >
> > > If you look at virtio_dev_probe() of virtio_bus
> > >
> > > device_features = dev->config->get_features(dev);
> > >
> > > Now if we use same bus for both front-end and back-end, both will try to
> > > get_features when there has been no set_features. Ideally vhost device should
> > > be initialized first with the set of features it supports. Vhost and virtio
> > > should use "status" and "features" complimentarily and not identically.
> >
> > Yes, but there's no need for doing status/features passthrough in epf vhost
> > drivers.b
> >
> >
> > > virtio device (or frontend) cannot be initialized before vhost device (or
> > > backend) gets initialized with data such as features. Similarly vhost (backend)
> > > cannot access virqueues or buffers before virtio (frontend) sets
> > > VIRTIO_CONFIG_S_DRIVER_OK whereas that requirement is not there for virtio as
> > > the physical memory for virtqueues are created by virtio (frontend).
> >
> > epf vhost drivers need to implement two devices: vhost(vringh) device and
> > virtio device (which is a mediated device). The vhost(vringh) device is doing
> > feature negotiation with the virtio device via RC/EP or NTB. The virtio device
> > is doing feature negotiation with local virtio drivers. If there're feature
> > mismatch, epf vhost drivers and do mediation between them.
> Here epf vhost should be initialized with a set of features for it to negotiate
> either as vhost device or virtio device no? Where should the initial feature
> set for epf vhost come from?


I think it can work as:

1) Having an initial features (hard coded in the code) set X in epf vhost
2) Using this X for both virtio device and vhost(vringh) device
3) local virtio driver will negotiate with virtio device with feature set Y
4) remote virtio driver will negotiate with vringh device with feature set Z
5) mediate between feature Y and feature Z since both Y and Z are a 
subset of X


> > > > It will have virtqueues but only used for the communication between itself and
> > > > uppter virtio driver. And it will have vringh queues which will be probe by
> > > > virtio epf transport drivers. And it needs to do datacopy between virtqueue and
> > > > vringh queues.
> > > >
> > > > It works like:
> > > >
> > > > virtio drivers <- virtqueue/virtio-bus -> epf vhost drivers <- vringh
> > > > queue/epf>
> > > >
> > > > The advantages is that there's no need for writing new buses and drivers.
> > > I think this will work however there is an addtional copy between vringh queue
> > > and virtqueue,
> >
> > I think not? E.g in use case 1), if we stick to virtio bus, we will have:
> >
> > virtio-rpmsg (EP) <- virtio ring(1) -> epf vhost driver (EP) <- virtio ring(2)
> > -> virtio pci (RC) <-> virtio rpmsg (RC)
> IIUC epf vhost driver (EP) will access virtio ring(2) using vringh?


Yes.


> And virtio
> ring(2) is created by virtio pci (RC).


Yes.


> > What epf vhost driver did is to read from virtio ring(1) about the buffer len
> > and addr and them DMA to Linux(RC)?
> okay, I made some optimization here where vhost-rpmsg using a helper writes a
> buffer from rpmsg's upper layer directly to remote Linux (RC) as against here
> were it has to be first written to virtio ring (1).
>
> Thinking how this would look for NTB
> virtio-rpmsg (HOST1) <- virtio ring(1) -> NTB(HOST1) <-> NTB(HOST2)  <- virtio
> ring(2) -> virtio-rpmsg (HOST2)
>
> Here the NTB(HOST1) will access the virtio ring(2) using vringh?


Yes, I think so it needs to use vring to access virtio ring (1) as well.


> Do you also think this will work seamlessly with virtio_net.c, virtio_blk.c?


Yes.


> I'd like to get clarity on two things in the approach you suggested, one is
> features (since epf vhost should ideally be transparent to any virtio driver)


We can have have an array of pre-defined features indexed by virtio 
device id in the code.


> and the other is how certain inputs to virtio device such as number of buffers
> be determined.


We can start from hard coded the value like 256, or introduce some API 
for user to change the value.


> Thanks again for your suggestions!


You're welcome.

Note that I just want to check whether or not we can reuse the virtio 
bus/driver. It's something similar to what you proposed in Software 
Layering but we just replace "vhost core" with "virtio bus" and move the 
vhost core below epf/ntb/platform transport.

Thanks


> Regards
> Kishon
>
> > > in some cases adds latency because of forwarding interrupts
> > > between vhost and virtio driver, vhost drivers providing features (which means
> > > it has to be aware of which virtio driver will be connected).
> > > virtio drivers (front end) generally access the buffers from it's local memory
> > > but when in backend it can access over MMIO (like PCI EPF or NTB) or userspace.
> > > > Does this make sense?
> > > Two copies in my opinion is an issue but lets get others opinions as well.
> >
> > Sure.
> >
> >
> > > Thanks for your suggestions!
> >
> > You're welcome.
> >
> > Thanks
> >
> >
> > > Regards
> > > Kishon
> > >
> > > > Thanks
> > > >
> > > >
> > > > > Thanks
> > > > > Kishon
> > > > >
> > > > > > Thanks
> > > > > >
> > > > > >
> > > > > > > > Kishon Vijay Abraham I (22):
> > > > > > > >  ÂÂÂÂ vhost: Make _feature_ bits a property of vhost device
> > > > > > > >  ÂÂÂÂ vhost: Introduce standard Linux driver model in VHOST
> > > > > > > >  ÂÂÂÂ vhost: Add ops for the VHOST driver to configure VHOST device
> > > > > > > >  ÂÂÂÂ vringh: Add helpers to access vring in MMIO
> > > > > > > >  ÂÂÂÂ vhost: Add MMIO helpers for operations on vhost virtqueue
> > > > > > > >  ÂÂÂÂ vhost: Introduce configfs entry for configuring VHOST
> > > > > > > >  ÂÂÂÂ virtio_pci: Use request_threaded_irq() instead of request_irq()
> > > > > > > >  ÂÂÂÂ rpmsg: virtio_rpmsg_bus: Disable receive virtqueue callback when
> > > > > > > >  ÂÂÂÂÂÂ reading messages
> > > > > > > >  ÂÂÂÂ rpmsg: Introduce configfs entry for configuring rpmsg
> > > > > > > >  ÂÂÂÂ rpmsg: virtio_rpmsg_bus: Add Address Service Notification support
> > > > > > > >  ÂÂÂÂ rpmsg: virtio_rpmsg_bus: Move generic rpmsg structure to
> > > > > > > >  ÂÂÂÂÂÂ rpmsg_internal.h
> > > > > > > >  ÂÂÂÂ virtio: Add ops to allocate and free buffer
> > > > > > > >  ÂÂÂÂ rpmsg: virtio_rpmsg_bus: Use virtio_alloc_buffer() and
> > > > > > > >  ÂÂÂÂÂÂ virtio_free_buffer()
> > > > > > > >  ÂÂÂÂ rpmsg: Add VHOST based remote processor messaging bus
> > > > > > > >  ÂÂÂÂ samples/rpmsg: Setup delayed work to send message
> > > > > > > >  ÂÂÂÂ samples/rpmsg: Wait for address to be bound to rpdev for sending
> > > > > > > >  ÂÂÂÂÂÂ message
> > > > > > > >  ÂÂÂÂ rpmsg.txt: Add Documentation to configure rpmsg using configfs
> > > > > > > >  ÂÂÂÂ virtio_pci: Add VIRTIO driver for VHOST on Configurable PCIe Endpoint
> > > > > > > >  ÂÂÂÂÂÂ device
> > > > > > > >  ÂÂÂÂ PCI: endpoint: Add EP function driver to provide VHOST interface
> > > > > > > >  ÂÂÂÂ NTB: Add a new NTB client driver to implement VIRTIO functionality
> > > > > > > >  ÂÂÂÂ NTB: Add a new NTB client driver to implement VHOST functionality
> > > > > > > >  ÂÂÂÂ NTB: Describe the ntb_virtio and ntb_vhost client in the documentation
> > > > > > > >
> > > > > > > >  ÂÂÂ Documentation/driver-api/ntb.rstÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 11 +
> > > > > > > >  ÂÂÂ Documentation/rpmsg.txtÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 56 +
> > > > > > > >  ÂÂÂ drivers/ntb/KconfigÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 18 +
> > > > > > > >  ÂÂÂ drivers/ntb/MakefileÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 2 +
> > > > > > > >  ÂÂÂ drivers/ntb/ntb_vhost.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 776 +++++++++++
> > > > > > > >  ÂÂÂ drivers/ntb/ntb_virtio.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 853 ++++++++++++
> > > > > > > >  ÂÂÂ drivers/ntb/ntb_virtio.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 56 +
> > > > > > > >  ÂÂÂ drivers/pci/endpoint/functions/KconfigÂÂÂÂÂÂÂ |ÂÂ 11 +
> > > > > > > >  ÂÂÂ drivers/pci/endpoint/functions/MakefileÂÂÂÂÂÂ |ÂÂÂ 1 +
> > > > > > > >  ÂÂÂ .../pci/endpoint/functions/pci-epf-vhost.cÂÂÂ | 1144 ++++++++++++++++
> > > > > > > >  ÂÂÂ drivers/rpmsg/KconfigÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 10 +
> > > > > > > >  ÂÂÂ drivers/rpmsg/MakefileÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 3 +-
> > > > > > > >  ÂÂÂ drivers/rpmsg/rpmsg_cfs.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 394 ++++++
> > > > > > > >  ÂÂÂ drivers/rpmsg/rpmsg_core.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 7 +
> > > > > > > >  ÂÂÂ drivers/rpmsg/rpmsg_internal.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 136 ++
> > > > > > > >  ÂÂÂ drivers/rpmsg/vhost_rpmsg_bus.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | 1151 +++++++++++++++++
> > > > > > > >  ÂÂÂ drivers/rpmsg/virtio_rpmsg_bus.cÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 184 ++-
> > > > > > > >  ÂÂÂ drivers/vhost/KconfigÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 1 +
> > > > > > > >  ÂÂÂ drivers/vhost/MakefileÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 2 +-
> > > > > > > >  ÂÂÂ drivers/vhost/net.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 10 +-
> > > > > > > >  ÂÂÂ drivers/vhost/scsi.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 24 +-
> > > > > > > >  ÂÂÂ drivers/vhost/test.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 17 +-
> > > > > > > >  ÂÂÂ drivers/vhost/vdpa.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 2 +-
> > > > > > > >  ÂÂÂ drivers/vhost/vhost.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 730 ++++++++++-
> > > > > > > >  ÂÂÂ drivers/vhost/vhost_cfs.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 341 +++++
> > > > > > > >  ÂÂÂ drivers/vhost/vringh.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 332 +++++
> > > > > > > >  ÂÂÂ drivers/vhost/vsock.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 20 +-
> > > > > > > >  ÂÂÂ drivers/virtio/KconfigÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 9 +
> > > > > > > >  ÂÂÂ drivers/virtio/MakefileÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 1 +
> > > > > > > >  ÂÂÂ drivers/virtio/virtio_pci_common.cÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 25 +-
> > > > > > > >  ÂÂÂ drivers/virtio/virtio_pci_epf.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |Â 670 ++++++++++
> > > > > > > >  ÂÂÂ include/linux/mod_devicetable.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 6 +
> > > > > > > >  ÂÂÂ include/linux/rpmsg.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 6 +
> > > > > > > >  ÂÂÂ {drivers/vhost => include/linux}/vhost.hÂÂÂÂÂ |Â 132 +-
> > > > > > > >  ÂÂÂ include/linux/virtio.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 3 +
> > > > > > > >  ÂÂÂ include/linux/virtio_config.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 42 +
> > > > > > > >  ÂÂÂ include/linux/vringh.hÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ 46 +
> > > > > > > >  ÂÂÂ samples/rpmsg/rpmsg_client_sample.cÂÂÂÂÂÂÂÂÂÂ |ÂÂ 32 +-
> > > > > > > >  ÂÂÂ tools/virtio/virtio_test.cÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ 2 +-
> > > > > > > >  ÂÂÂ 39 files changed, 7083 insertions(+), 183 deletions(-)
> > > > > > > >  ÂÂÂ create mode 100644 drivers/ntb/ntb_vhost.c
> > > > > > > >  ÂÂÂ create mode 100644 drivers/ntb/ntb_virtio.c
> > > > > > > >  ÂÂÂ create mode 100644 drivers/ntb/ntb_virtio.h
> > > > > > > >  ÂÂÂ create mode 100644 drivers/pci/endpoint/functions/pci-epf-vhost.c
> > > > > > > >  ÂÂÂ create mode 100644 drivers/rpmsg/rpmsg_cfs.c
> > > > > > > >  ÂÂÂ create mode 100644 drivers/rpmsg/vhost_rpmsg_bus.c
> > > > > > > >  ÂÂÂ create mode 100644 drivers/vhost/vhost_cfs.c
> > > > > > > >  ÂÂÂ create mode 100644 drivers/virtio/virtio_pci_epf.c
> > > > > > > >  ÂÂÂ rename {drivers/vhost => include/linux}/vhost.h (66%)
> > > > > > > >
> > > > > > > > --
> > > > > > > > 2.17.1