Re: [RFC] Proposal of QEMU PCI Endpoint test environment

[Shunsuke Mie]

On 2023/09/21 18:11, Kishon Vijay Abraham I wrote:
> +Vaishnav
>
> Hi Shunsuke,
>
> On 8/18/2023 7:16 PM, Shunsuke Mie wrote:
> > Hi all,
> >
> > We are proposing to add a new test syste to Linux for PCIe Endpoint. 
> > That
> > can be run on QEMU without real hardware. At present, partially we have
> > confirmed that pci-epf-test is working, but it is not yet complete.
> > However, we would appreciate your comments on the architecture design.
> >
> > # Background
> > The background is as follows.
> >
> > PCI Endpoint function driver is implemented using the PCIe Endpoint
> > framework, but it requires physical boards for testing, and it is 
> > difficult
> > to test sufficiently. In order to find bugs and hardware-dependent
> > implementations early, continuous testing is required. Since it is
> > difficult to automate tests that require hardware, this RFC proposes a
> > virtual environment for testing PCI endpoint function drivers.
>
> This would be quite useful and thank you for attempting it! I would 
> like to compare other mechanisms available in-addition to QEMU before 
> going with the QEMU approach.

I got it. I'll make a table to compare some methods that includes 
greybus to realize this emulation environment.


Best,

Shunsuke

> Though I don't understand this fully, Looking at 
> https://osseu2023.sched.com/event/1OGk8/emulating-devices-in-linux-using-greybus-subsystem-vaishnav-mohandas-achath-texas-instruments, 
> Vaishnav seems to solve the same problem using greybus for multiple 
> type s of devices.
>
> Vaishnav, we'd wait for your OSS presentation but do you have any 
> initial thoughts on how greybus could be used to test PCIe endpoint 
> drivers?
>
> Thanks,
> Kishon
>
> > # Architecture
> > The overview of the architecture is as follows.
> >
> >    Guest 1                        Guest 2
> > +-------------------------+    +----------------------------+
> > | Linux kernel            |    | Linux kernel               |
> > |                         |    |                            |
> > | PCI EP function driver  |    |                            |
> > | (e.g. pci-epf-test)     |    |                            |
> > |-------------------------|    | PCI Device Driver          |
> > | (2) QEMU EPC Driver     |    | (e.g. pci_endpoint_test)   |
> > +-------------------------+    +----------------------------+
> > +-------------------------+    +----------------------------+
> > | QEMU                    |    | QEMU                       |
> > |-------------------------|    |----------------------------|
> > | (1) QEMU PCI EPC Device *----* (3) QEMU EPF Bridge Device |
> > +-------------------------+    +----------------------------+
> >
> > At present, it is designed to work guests only on the same host, and
> > communication is done through Unix domain sockets.
> >
> > The three parts shown in the figure were introduced this time.
> >
> > (1) QEMU PCI Endpoint Controller(EPC) Device
> > PCI Endpoint Controller implemented as QEMU PCI device.
> > (2) QEMU PCI Endpoint Controller(EPC) Driver
> > Linux kernel driver that drives the device (1). It registers a epc 
> > device
> > to linux kernel and handling each operations for the epc device.
> > (3) QEMU PCI Endpoint function(EPF) Bridge Device
> > QEMU PCI device that cooperates with (1) and performs accesses to pci
> > configuration space, BAR and memory space to communicate each guests, 
> > and
> > generates interruptions to the guest 1.
> >
> > Each projects are:
> > (1), (3) https://github.com/ShunsukeMie/qemu/tree/epf-bridge/v1 
> > <https://github.com/ShunsukeMie/qemu/tree/epf-bridge/v1>
> > files: hw/misc/{qemu-epc.{c,h}, epf-bridge.c}
> > (2) https://github.com/ShunsukeMie/linux-virtio-rdma/tree/qemu-epc 
> > <https://github.com/ShunsukeMie/linux-virtio-rdma/tree/qemu-epc>
> > files: drivers/pci/controller/pcie-qemu-ep.c
> >
> > # Protocol
> >
> > PCI, PCIe has a layer structure that includes Physical, Data Lane and
> > Transaction. The communicates between the bridge(3) and controller (1)
> > mimic the Transaction. Specifically, a protocol is implemented for
> > exchanging fd for communication protocol version check and 
> > communication,
> > in addition to the interaction equivalent to PCIe Transaction Layer 
> > Packet
> > (Read and Write of I/O, Memory, Configuration space and Message). In my
> > mind, we need to discuss the communication mor.
> >
> > We also are planning to post the patch set after the code is 
> > organized and
> > the protocol discussion is matured.
> >
> > Best regards,
> > Shunsuke