[PATCH v4 00/15] Tidy up vfio_device life cycle

From: Kevin Tian
Date: Tue Sep 20 2022 - 23:26:32 EST


The idea is to let vfio core manage the vfio_device life cycle instead
of duplicating the logic cross drivers. Besides cleaner code in driver
side this also allows adding struct device to vfio_device as the first
step toward adding cdev uAPI in the future. Another benefit is that
user can now look at sysfs to decide whether a device is bound to
vfio [1], e.g.:

/sys/devices/pci0000\:6f/0000\:6f\:01.0/vfio-dev/vfio0

Though most drivers can fit the new model naturally:

- vfio_alloc_device() to allocate and initialize vfio_device
- vfio_put_device() to release vfio_device
- dev_ops->init() for driver private initialization
- dev_ops->release() for driver private cleanup

vfio-ccw is the only exception due to a life cycle mess that its private
structure mixes both parent and mdev info hence must be alloc/freed
outside of the life cycle of vfio device.

Per prior discussions this won't be fixed in short term by IBM folks [2].

Instead of waiting this series introduces a few tricks to move forward:

- vfio_init_device() to initialize a pre-allocated device structure;

- require *EVERY* driver to implement @release and free vfio_device
inside. Then vfio-ccw can use a completion mechanism to delay the
free to css driver;

The second trick is not a real burden to other drivers because they
all require a @release for private cleanup anyway. Later once the ccw
mess is fixed a simple cleanup can be done by moving free from @release
to vfio core.

Thanks
Kevin

[1] https://listman.redhat.com/archives/libvir-list/2022-August/233482.html
[2] https://lore.kernel.org/all/0ee29bd6583f17f0ee4ec0769fa50e8ea6703623.camel@xxxxxxxxxxxxx/

v4:
- fix use-after-free issue in @release of mtty/mbochs and also change
mdpy/ap to free vfio-device as the last thing in @release (Alex)
- revert the rename from 'vfio' to 'vfio_group' in procfs (Alex)

v3:
- https://lore.kernel.org/lkml/20220909102247.67324-1-kevin.tian@xxxxxxxxx/
- rebase to vfio-next after resolving conflicts with Yishai's series
- add missing fixes for two checkpatch errors
- fix grammar issues (Eric Auger)
- add more r-b's

v2:
- https://lore.kernel.org/lkml/20220901143747.32858-1-kevin.tian@xxxxxxxxx/
- rebase to 6.0-rc3
- fix build warnings (lkp)
- patch1: remove unnecessary forward reference (Jason)
- patch10: leave device_set released by vfio core (Jason)
- patch13: add Suggested-by
- patch15: add ABI file sysfs-devices-vfio-dev (Alex)
- patch15: rename 'vfio' to 'vfio_group' in procfs (Jason)

v1: https://lore.kernel.org/lkml/20220827171037.30297-1-kevin.tian@xxxxxxxxx/

Kevin Tian (6):
vfio: Add helpers for unifying vfio_device life cycle
drm/i915/gvt: Use the new device life cycle helpers
vfio/platform: Use the new device life cycle helpers
vfio/amba: Use the new device life cycle helpers
vfio/ccw: Use the new device life cycle helpers
vfio: Rename vfio_device_put() and vfio_device_try_get()

Yi Liu (9):
vfio/pci: Use the new device life cycle helpers
vfio/mlx5: Use the new device life cycle helpers
vfio/hisi_acc: Use the new device life cycle helpers
vfio/mdpy: Use the new device life cycle helpers
vfio/mtty: Use the new device life cycle helpers
vfio/mbochs: Use the new device life cycle helpers
vfio/ap: Use the new device life cycle helpers
vfio/fsl-mc: Use the new device life cycle helpers
vfio: Add struct device to vfio_device

.../ABI/testing/sysfs-devices-vfio-dev | 8 +
MAINTAINERS | 1 +
drivers/gpu/drm/i915/gvt/gvt.h | 5 +-
drivers/gpu/drm/i915/gvt/kvmgt.c | 52 ++++--
drivers/gpu/drm/i915/gvt/vgpu.c | 33 ++--
drivers/s390/cio/vfio_ccw_ops.c | 52 +++++-
drivers/s390/cio/vfio_ccw_private.h | 3 +
drivers/s390/crypto/vfio_ap_ops.c | 50 +++---
drivers/vfio/fsl-mc/vfio_fsl_mc.c | 85 +++++----
.../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 80 ++++-----
drivers/vfio/pci/mlx5/main.c | 50 ++++--
drivers/vfio/pci/vfio_pci.c | 20 +--
drivers/vfio/pci/vfio_pci_core.c | 23 ++-
drivers/vfio/platform/vfio_amba.c | 72 ++++++--
drivers/vfio/platform/vfio_platform.c | 66 +++++--
drivers/vfio/platform/vfio_platform_common.c | 71 +++-----
drivers/vfio/platform/vfio_platform_private.h | 18 +-
drivers/vfio/vfio_main.c | 164 +++++++++++++++---
include/linux/vfio.h | 28 ++-
include/linux/vfio_pci_core.h | 6 +-
samples/vfio-mdev/mbochs.c | 73 +++++---
samples/vfio-mdev/mdpy.c | 81 +++++----
samples/vfio-mdev/mtty.c | 67 ++++---
23 files changed, 728 insertions(+), 380 deletions(-)
create mode 100644 Documentation/ABI/testing/sysfs-devices-vfio-dev


base-commit: f39856aacb078c1c93acef011a37121b17d54fe0
--
2.21.3