Re: [PATCH RFC v1 0/3] Introduce vfio-pci-core subsystem

[Jason Gunthorpe]

On Mon, Jan 18, 2021 at 02:38:06PM +0100, Cornelia Huck wrote:

> > These devices will be seen on the Auxiliary bus as:
> > mlx5_core.vfio_pci.2048 -> ../../../devices/pci0000:00/0000:00:02.0/0000:05:00.0/0000:06:00.0/0000:07:00.0/mlx5_core.vfio_pci.2048
> > mlx5_core.vfio_pci.2304 -> ../../../devices/pci0000:00/0000:00:02.0/0000:05:00.0/0000:06:00.0/0000:07:00.1/mlx5_core.vfio_pci.2304
> > 
> > 2048 represents BDF 08:00.0 and 2304 represents BDF 09:00.0 in decimal
> > view. In this manner, the administrator will be able to locate the
> > correct vfio-pci module it should bind the desired BDF to (by finding
> > the pointer to the module according to the Auxiliary driver of that
> > BDF).
> 
> I'm not familiar with that auxiliary framework (it seems to be fairly
> new?); 

Auxillary bus is for connecting two parts of the same driver that
reside in different modules/subystems. Here it is connecting the vfio
part to the core part of mlx5 (running on the PF).

> but can you maybe create an auxiliary device unconditionally and
> contain all hardware-specific things inside a driver for it? Or is
> that not flexible enough?

The goal is to make a vfio device, auxiliary bus is only in the
picture because a VF device under vfio needs to interact with the PF
mlx5_core driver, auxiliary bus provides that connection.

You can say that all the HW specific things are in the mlx5_vfio_pci
driver. It is an unusual driver because it must bind to both the PCI
VF with a pci_driver and to the mlx5_core PF using an
auxiliary_driver. This is needed for the object lifetimes to be
correct.

The PF is providing services to control a full VF which mlx5_vfio_pci
converts into VFIO API.

> >  drivers/vfio/pci/Kconfig            |   22 +-
> >  drivers/vfio/pci/Makefile           |   16 +-
> >  drivers/vfio/pci/mlx5_vfio_pci.c    |  253 +++
> >  drivers/vfio/pci/vfio_pci.c         | 2386 +--------------------------
> >  drivers/vfio/pci/vfio_pci_core.c    | 2311 ++++++++++++++++++++++++++
> 
> Especially regarding this diffstat... 

It is a bit misleading because it doesn't show the rename

Jason