RE: [EXTERNAL] Re: [PATCH v10 1/1] misc: mrvl-cn10k-dpi: add Octeon CN10K DPI administrative driver

From: Vamsi Krishna Attunuru
Date: Wed Jul 10 2024 - 09:20:53 EST



>-----Original Message-----
>From: Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx>
>Sent: Wednesday, July 10, 2024 6:28 PM
>To: Vamsi Krishna Attunuru <vattunuru@xxxxxxxxxxx>
>Cc: arnd@xxxxxxxx; Jerin Jacob <jerinj@xxxxxxxxxxx>; Srujana Challa
><schalla@xxxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx
>Subject: [EXTERNAL] Re: [PATCH v10 1/1] misc: mrvl-cn10k-dpi: add Octeon
>CN10K DPI administrative driver
>
>On Sat, Jul 06, 2024 at 08: 30: 09AM -0700, Vamsi Attunuru wrote: > Adds a
>misc driver for Marvell CN10K DPI(DMA Engine) device's physical > function
>which initializes DPI DMA hardware's global configuration and > enables
>hardware mailbox
>On Sat, Jul 06, 2024 at 08:30:09AM -0700, Vamsi Attunuru wrote:
>> Adds a misc driver for Marvell CN10K DPI(DMA Engine) device's physical
>> function which initializes DPI DMA hardware's global configuration and
>> enables hardware mailbox channels between physical function (PF) and
>> it's virtual functions (VF). VF device drivers (User space drivers)
>> use this hw mailbox to communicate any required device configuration
>> on it's respective VF device. Accordingly, this DPI PF driver
>> provisions the VF device resources.
>>
>> At the hardware level, the DPI physical function (PF) acts as a
>> management interface to setup the VF device resources, VF devices are
>> only provisioned to handle or control the actual DMA Engine's data transfer
>capabilities.
>>
>> Signed-off-by: Vamsi Attunuru <vattunuru@xxxxxxxxxxx>
>> Reviewed-by: Srujana Challa <schalla@xxxxxxxxxxx>
>> ---
>> Changes V9 -> V10
>> - Added checks to ensure reserved fields are set to 0
>>
>> Changes V8 -> V9:
>> - Addressed minor comments
>>
>> Changes V7 -> V8:
>> - Used bit shift operations to access mbox msg fields
>> - Removed bitfields in mailbox msg structure
>>
>> Changes V6 -> V7:
>> - Updated documentation with required references
>> - Addressed V6 review comments
>>
>> Changes V5 -> V6:
>> - Updated documentation
>> - Fixed data types in uapi file
>>
>> Changes V4 -> V5:
>> - Fixed license and data types in uapi file
>>
>> Changes V3 -> V4:
>> - Moved ioctl definations to .h file
>> - Fixed structure alignements which are passed in ioctl
>>
>> Changes V2 -> V3:
>> - Added ioctl operation to the fops
>> - Used managed version of kzalloc & request_irq
>> - Addressed miscellaneous comments
>>
>> Changes V1 -> V2:
>> - Fixed return values and busy-wait loops
>> - Merged .h file into .c file
>> - Fixed directory structure
>> - Removed module params
>> - Registered the device as misc device
>>
>> Documentation/misc-devices/index.rst | 1 +
>> Documentation/misc-devices/mrvl_cn10k_dpi.rst | 52 ++
>> .../userspace-api/ioctl/ioctl-number.rst | 1 +
>> MAINTAINERS | 5 +
>> drivers/misc/Kconfig | 14 +
>> drivers/misc/Makefile | 1 +
>> drivers/misc/mrvl_cn10k_dpi.c | 676 ++++++++++++++++++
>> include/uapi/misc/mrvl_cn10k_dpi.h | 39 +
>> 8 files changed, 789 insertions(+)
>>
>> diff --git a/Documentation/misc-devices/index.rst
>> b/Documentation/misc-devices/index.rst
>> index 2d0ce9138588..8c5b226d8313 100644
>> --- a/Documentation/misc-devices/index.rst
>> +++ b/Documentation/misc-devices/index.rst
>> @@ -21,6 +21,7 @@ fit into other categories.
>> isl29003
>> lis3lv02d
>> max6875
>> + mrvl_cn10k_dpi
>> oxsemi-tornado
>> pci-endpoint-test
>> spear-pcie-gadget
>> diff --git a/Documentation/misc-devices/mrvl_cn10k_dpi.rst
>> b/Documentation/misc-devices/mrvl_cn10k_dpi.rst
>> new file mode 100644
>> index 000000000000..a75e372723d8
>> --- /dev/null
>> +++ b/Documentation/misc-devices/mrvl_cn10k_dpi.rst
>> @@ -0,0 +1,52 @@
>> +.. SPDX-License-Identifier: GPL-2.0
>> +
>> +===============================================
>> +Marvell CN10K DMA packet interface (DPI) driver
>> +===============================================
>> +
>> +Overview
>> +========
>> +
>> +DPI is a DMA packet interface hardware block in Marvell's CN10K silicon.
>> +DPI hardware comprises a physical function (PF), its virtual
>> +functions, mailbox logic, and a set of DMA engines & DMA command
>queues.
>> +
>> +DPI PF function is an administrative function which services the
>> +mailbox requests from its VF functions and provisions DMA engine
>> +resources to it's VF functions.
>> +
>> +mrvl_cn10k_dpi.ko misc driver loads on DPI PF device and services the
>> +mailbox commands submitted by the VF devices and accordingly
>> +initializes the DMA engines and VF device's DMA command queues. Also,
>> +driver creates /dev/mrvl-cn10k-dpi node to set DMA engine and PEM
>> +(PCIe interface) port attributes like fifo length, molr, mps & mrrs.
>> +
>> +DPI PF driver is just an administrative driver to setup its VF
>> +device's queues and provisions the hardware resources, it cannot
>> +initiate any DMA operations. Only VF devices are provisioned with DMA
>capabilities.
>> +
>> +Driver location
>> +===============
>> +
>> +drivers/misc/mrvl_cn10k_dpi.c
>> +
>> +Driver IOCTLs
>> +=============
>> +
>> +:c:macro::`DPI_MPS_MRRS_CFG`
>> +ioctl that sets max payload size & max read request size parameters
>> +of a pem port to which DMA engines are wired.
>> +
>> +
>> +:c:macro::`DPI_ENGINE_CFG`
>> +ioctl that sets DMA engine's fifo sizes & max outstanding load
>> +request thresholds.
>> +
>> +User space code example
>> +=======================
>> +
>> +DPI VF devices are probed and accessed from user space applications
>> +using vfio-pci driver. Below is a sample dpi dma application to
>> +demonstrate on how applications use mailbox and ioctl services from DPI
>PF kernel driver.
>> +
>> +https://urldefense.proofpoint.com/v2/url?u=https-
>3A__github.com_Marve
>> +llEmbeddedProcessors_dpi-2Dsample-
>2Dapp&d=DwIBAg&c=nKjWec2b6R0mOyPaz7
>>
>+xtfQ&r=WllrYaumVkxaWjgKto6E_rtDQshhIhik2jkvzFyRhW8&m=bFx_7eltw7S
>6zzVu
>>
>+1LNEdtsbwwynJfAKTja649QUwGNU_y4uWqGoEZ4f7JluYLjX&s=wOMzADbq
>9f4xxz1Oug
>> +-slj_xy4ZcbrnWfQJWeO0_ugA&e=
>> diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst
>> b/Documentation/userspace-api/ioctl/ioctl-number.rst
>> index a141e8e65c5d..def539770439 100644
>> --- a/Documentation/userspace-api/ioctl/ioctl-number.rst
>> +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
>> @@ -362,6 +362,7 @@ Code Seq# Include File
>Comments
>> 0xB6 all linux/fpga-dfl.h
>> 0xB7 all uapi/linux/remoteproc_cdev.h <mailto:linux-
>remoteproc@xxxxxxxxxxxxxxx>
>> 0xB7 all uapi/linux/nsfs.h <mailto:Andrei Vagin
><avagin@xxxxxxxxxx>>
>> +0xB8 01-02 uapi/misc/mrvl_cn10k_dpi.h Marvell CN10K DPI
>driver
>> 0xC0 00-0F linux/usb/iowarrior.h
>> 0xCA 00-0F uapi/misc/cxl.h
>> 0xCA 10-2F uapi/misc/ocxl.h
>> diff --git a/MAINTAINERS b/MAINTAINERS index
>> aae88b7a6c32..2c17d651954a 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -13477,6 +13477,11 @@ S: Supported
>> F: Documentation/devicetree/bindings/mmc/marvell,xenon-sdhci.yaml
>> F: drivers/mmc/host/sdhci-xenon*
>>
>> +MARVELL OCTEON CN10K DPI DRIVER
>> +M: Vamsi Attunuru <vattunuru@xxxxxxxxxxx>
>> +S: Supported
>> +F: drivers/misc/mrvl_cn10k_dpi.c
>> +
>> MATROX FRAMEBUFFER DRIVER
>> L: linux-fbdev@xxxxxxxxxxxxxxx
>> S: Orphan
>> diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index
>> faf983680040..965641017a62 100644
>> --- a/drivers/misc/Kconfig
>> +++ b/drivers/misc/Kconfig
>> @@ -585,6 +585,20 @@ config NSM
>> To compile this driver as a module, choose M here.
>> The module will be called nsm.
>>
>> +config MARVELL_CN10K_DPI
>> + tristate "Octeon CN10K DPI driver"
>> + depends on ARM64 && PCI
>
>Why does ARM64 matter here? I don't see any dependency required of it.
>
Thanks, Greg, for your time. This DPI device is an on-chip PCIe device and only present on
Marvell's CN10K platforms(which are 64-bit ARM SoC processors), so added those dependency.

Regards
Vamsi


>I just tested this, and it seems to build find for me. As I had to manually fix it
>up to get it to apply to my tree, I'll leave it only depending on PCI for now,
>thanks.
>
>greg k-h