Re: [PATCH v2] PCI: hotplug: Add OCTEON PCI hotplug controller driver

[Ilpo Järvinen]

On Fri, 23 Aug 2024, Shijith Thotton wrote:

> This patch introduces a PCI hotplug controller driver for the OCTEON
> PCIe device, a multi-function PCIe device where the first function acts
> as a hotplug controller. It is equipped with MSI-x interrupts to notify
> the host of hotplug events from the OCTEON firmware.
> 
> The driver facilitates the hotplugging of non-controller functions
> within the same device. During probe, non-controller functions are
> removed and registered as PCI hotplug slots. The slots are added back
> only upon request from the device firmware. The driver also allows the
> enabling and disabling of the slots via sysfs slot entries, provided by
> the PCI hotplug framework.
> 
> Signed-off-by: Shijith Thotton <sthotton@xxxxxxxxxxx>
> Co-developed-by: Vamsi Attunuru <vattunuru@xxxxxxxxxxx>
> Signed-off-by: Vamsi Attunuru <vattunuru@xxxxxxxxxxx>
> ---
> 
> This patch introduces a PCI hotplug controller driver for OCTEON PCIe hotplug
> controller. The OCTEON PCIe device is a multi-function device where the first
> function acts as a PCI hotplug controller.
> 
>               +--------------------------------+
>               |           Root Port            |
>               +--------------------------------+
>                               |
>                              PCIe
>                               |
> +---------------------------------------------------------------+
> |              OCTEON PCIe Multifunction Device                 |
> +---------------------------------------------------------------+
>             |                    |              |            |
>             |                    |              |            |
> +---------------------+  +----------------+  +-----+  +----------------+
> |      Function 0     |  |   Function 1   |  | ... |  |   Function 7   |
> | (Hotplug controller)|  | (Hotplug slot) |  |     |  | (Hotplug slot) |
> +---------------------+  +----------------+  +-----+  +----------------+
>             |
>             |
> +-------------------------+
> |   Controller Firmware   |
> +-------------------------+
> 
> The hotplug controller driver facilitates the hotplugging of non-controller
> functions in the same device. During the probe of the driver, the non-controller
> function are removed and registered as PCI hotplug slots. They are added back
> only upon request from the device firmware. The driver also allows the user to
> enable/disable the functions using sysfs slot entries provided by PCI hotplug
> framework.
> 
> This solution adopts a hardware + software approach for several reasons:
> 
> 1. To reduce hardware implementation cost. Supporting complete hotplug
>    capability within the card would require a PCI switch implemented within.
> 
> 2. In the multi-function device, non-controller functions can act as emulated
>    devices. The firmware can dynamically enable or disable them at runtime.
> 
> 3. Not all root ports support PCI hotplug. This approach provides greater
>    flexibility and compatibility across different hardware configurations.
> 
> The hotplug controller function is lightweight and is equipped with MSI-x
> interrupts to notify the host about hotplug events. Upon receiving an
> interrupt, the hotplug register is read, and the required function is enabled
> or disabled.
> 
> This driver will be beneficial for managing PCI hotplug events on the OCTEON
> PCIe device without requiring complex hardware solutions.
> 
> Changes in v2:
> - Added missing include files.
> - Used dev_err_probe() for error handling.
> - Used guard() for mutex locking.
> - Splited cleanup actions and added per-slot cleanup action.
> - Fixed coding style issues.
> - Added co-developed-by tag.
> 
>  MAINTAINERS                    |   6 +
>  drivers/pci/hotplug/Kconfig    |  10 +
>  drivers/pci/hotplug/Makefile   |   1 +
>  drivers/pci/hotplug/octep_hp.c | 412 +++++++++++++++++++++++++++++++++
>  4 files changed, 429 insertions(+)
>  create mode 100644 drivers/pci/hotplug/octep_hp.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 42decde38320..7b5a618eed1c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -13677,6 +13677,12 @@ R:	schalla@xxxxxxxxxxx
>  R:	vattunuru@xxxxxxxxxxx
>  F:	drivers/vdpa/octeon_ep/
>  
> +MARVELL OCTEON HOTPLUG CONTROLLER DRIVER
> +R:	Shijith Thotton <sthotton@xxxxxxxxxxx>
> +R:	Vamsi Attunuru <vattunuru@xxxxxxxxxxx>
> +S:	Supported
> +F:	drivers/pci/hotplug/octep_hp.c
> +
>  MATROX FRAMEBUFFER DRIVER
>  L:	linux-fbdev@xxxxxxxxxxxxxxx
>  S:	Orphan
> diff --git a/drivers/pci/hotplug/Kconfig b/drivers/pci/hotplug/Kconfig
> index 1472aef0fb81..2e38fd25f7ef 100644
> --- a/drivers/pci/hotplug/Kconfig
> +++ b/drivers/pci/hotplug/Kconfig
> @@ -173,4 +173,14 @@ config HOTPLUG_PCI_S390
>  
>  	  When in doubt, say Y.
>  
> +config HOTPLUG_PCI_OCTEONEP
> +	bool "OCTEON PCI device Hotplug controller driver"
> +	depends on HOTPLUG_PCI
> +	help
> +	  Say Y here if you have an OCTEON PCIe device with a hotplug
> +	  controller. This driver enables the non-controller functions of the
> +	  device to be registered as hotplug slots.
> +
> +	  When in doubt, say N.
> +
>  endif # HOTPLUG_PCI
> diff --git a/drivers/pci/hotplug/Makefile b/drivers/pci/hotplug/Makefile
> index 240c99517d5e..40aaf31fe338 100644
> --- a/drivers/pci/hotplug/Makefile
> +++ b/drivers/pci/hotplug/Makefile
> @@ -20,6 +20,7 @@ obj-$(CONFIG_HOTPLUG_PCI_RPA)		+= rpaphp.o
>  obj-$(CONFIG_HOTPLUG_PCI_RPA_DLPAR)	+= rpadlpar_io.o
>  obj-$(CONFIG_HOTPLUG_PCI_ACPI)		+= acpiphp.o
>  obj-$(CONFIG_HOTPLUG_PCI_S390)		+= s390_pci_hpc.o
> +obj-$(CONFIG_HOTPLUG_PCI_OCTEONEP)	+= octep_hp.o
>  
>  # acpiphp_ibm extends acpiphp, so should be linked afterwards.
>  
> diff --git a/drivers/pci/hotplug/octep_hp.c b/drivers/pci/hotplug/octep_hp.c
> new file mode 100644
> index 000000000000..3ac90ffff564
> --- /dev/null
> +++ b/drivers/pci/hotplug/octep_hp.c
> @@ -0,0 +1,412 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright (C) 2024 Marvell. */
> +
> +#include <linux/cleanup.h>
> +#include <linux/container_of.h>
> +#include <linux/delay.h>
> +#include <linux/dev_printk.h>
> +#include <linux/init.h>
> +#include <linux/interrupt.h>
> +#include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/kernel.h>
> +#include <linux/list.h>
> +#include <linux/module.h>
> +#include <linux/mutex.h>
> +#include <linux/pci.h>
> +#include <linux/pci_hotplug.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +#include <linux/workqueue.h>
> +
> +#define OCTEP_HP_INTR_OFFSET(x) (0x20400 + ((x) << 4))
> +#define OCTEP_HP_INTR_VECTOR(x) (16 + (x))
> +#define OCTEP_HP_DRV_NAME "octep_hp"
> +
> +/*
> + * Type of MSI-X interrupts.
> + * The macros OCTEP_HP_INTR_VECTOR and OCTEP_HP_INTR_OFFSET are used to
> + * generate the vector and offset for an interrupt type.
> + */
> +enum octep_hp_intr_type {
> +	OCTEP_HP_INTR_INVALID = -1,
> +	OCTEP_HP_INTR_ENA,
> +	OCTEP_HP_INTR_DIS,

Making these numbers explicit (since they cannot be just any numbers) fell 
through cracks.


-- 
 i.