RE: [EXTERNAL] Re: [PATCH v10 1/1] misc: mrvl-cn10k-dpi: add Octeon CN10K DPI administrative driver

[Vamsi Krishna Attunuru]

>-----Original Message-----
>From: Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx>
>Sent: Wednesday, July 10, 2024 6:57 PM
>To: Vamsi Krishna Attunuru <vattunuru@xxxxxxxxxxx>
>Cc: arnd@xxxxxxxx; Jerin Jacob <jerinj@xxxxxxxxxxx>; Srujana Challa
><schalla@xxxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx
>Subject: Re: [EXTERNAL] Re: [PATCH v10 1/1] misc: mrvl-cn10k-dpi: add
>Octeon CN10K DPI administrative driver
>
>On Wed, Jul 10, 2024 at 01: 19: 56PM +0000, Vamsi Krishna Attunuru wrote: > >
>> >-----Original Message----- > >From: Greg KH
><gregkh@ linuxfoundation. org> > >Sent: Wednesday, July 10, 2024 6: 28 PM >
>>To: Vamsi 
>On Wed, Jul 10, 2024 at 01:19:56PM +0000, Vamsi Krishna Attunuru wrote:
>>
>>
>> >-----Original Message-----
>> >From: Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx>
>> >Sent: Wednesday, July 10, 2024 6:28 PM
>> >To: Vamsi Krishna Attunuru <vattunuru@xxxxxxxxxxx>
>> >Cc: arnd@xxxxxxxx; Jerin Jacob <jerinj@xxxxxxxxxxx>; Srujana Challa
>> ><schalla@xxxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx
>> >Subject: [EXTERNAL] Re: [PATCH v10 1/1] misc: mrvl-cn10k-dpi: add
>> >Octeon CN10K DPI administrative driver
>> >
>> >On Sat, Jul 06, 2024 at 08: 30: 09AM -0700, Vamsi Attunuru wrote: >
>> >Adds a misc driver for Marvell CN10K DPI(DMA Engine) device's
>> >physical > function which initializes DPI DMA hardware's global
>> >configuration and > enables hardware mailbox On Sat, Jul 06, 2024 at
>> >08:30:09AM -0700, Vamsi Attunuru wrote:
>> >> Adds a misc driver for Marvell CN10K DPI(DMA Engine) device's
>> >> physical function which initializes DPI DMA hardware's global
>> >> configuration and enables hardware mailbox channels between
>> >> physical function (PF) and it's virtual functions (VF). VF device
>> >> drivers (User space drivers) use this hw mailbox to communicate any
>> >> required device configuration on it's respective VF device.
>> >> Accordingly, this DPI PF driver provisions the VF device resources.
>> >>
>> >> At the hardware level, the DPI physical function (PF) acts as a
>> >> management interface to setup the VF device resources, VF devices
>> >> are only provisioned to handle or control the actual DMA Engine's
>> >> data transfer
>> >capabilities.
>> >>
>> >> Signed-off-by: Vamsi Attunuru <vattunuru@xxxxxxxxxxx>
>> >> Reviewed-by: Srujana Challa <schalla@xxxxxxxxxxx>
>> >> ---
>> >> Changes V9 -> V10
>> >> - Added checks to ensure reserved fields are set to 0
>> >>
>> >> Changes V8 -> V9:
>> >> - Addressed minor comments
>> >>
>> >> Changes V7 -> V8:
>> >> - Used bit shift operations to access mbox msg fields
>> >> - Removed bitfields in mailbox msg structure
>> >>
>> >> Changes V6 -> V7:
>> >> - Updated documentation with required references
>> >> - Addressed V6 review comments
>> >>
>> >> Changes V5 -> V6:
>> >> - Updated documentation
>> >> - Fixed data types in uapi file
>> >>
>> >> Changes V4 -> V5:
>> >> - Fixed license and data types in uapi file
>> >>
>> >> Changes V3 -> V4:
>> >> - Moved ioctl definations to .h file
>> >> - Fixed structure alignements which are passed in ioctl
>> >>
>> >> Changes V2 -> V3:
>> >> - Added ioctl operation to the fops
>> >> - Used managed version of kzalloc & request_irq
>> >> - Addressed miscellaneous comments
>> >>
>> >> Changes V1 -> V2:
>> >> - Fixed return values and busy-wait loops
>> >> - Merged .h file into .c file
>> >> - Fixed directory structure
>> >> - Removed module params
>> >> - Registered the device as misc device
>> >>
>> >>  Documentation/misc-devices/index.rst          |   1 +
>> >>  Documentation/misc-devices/mrvl_cn10k_dpi.rst |  52 ++
>> >>  .../userspace-api/ioctl/ioctl-number.rst      |   1 +
>> >>  MAINTAINERS                                   |   5 +
>> >>  drivers/misc/Kconfig                          |  14 +
>> >>  drivers/misc/Makefile                         |   1 +
>> >>  drivers/misc/mrvl_cn10k_dpi.c                 | 676 ++++++++++++++++++
>> >>  include/uapi/misc/mrvl_cn10k_dpi.h            |  39 +
>> >>  8 files changed, 789 insertions(+)
>> >>
>> >> diff --git a/Documentation/misc-devices/index.rst
>> >> b/Documentation/misc-devices/index.rst
>> >> index 2d0ce9138588..8c5b226d8313 100644
>> >> --- a/Documentation/misc-devices/index.rst
>> >> +++ b/Documentation/misc-devices/index.rst
>> >> @@ -21,6 +21,7 @@ fit into other categories.
>> >>     isl29003
>> >>     lis3lv02d
>> >>     max6875
>> >> +   mrvl_cn10k_dpi
>> >>     oxsemi-tornado
>> >>     pci-endpoint-test
>> >>     spear-pcie-gadget
>> >> diff --git a/Documentation/misc-devices/mrvl_cn10k_dpi.rst
>> >> b/Documentation/misc-devices/mrvl_cn10k_dpi.rst
>> >> new file mode 100644
>> >> index 000000000000..a75e372723d8
>> >> --- /dev/null
>> >> +++ b/Documentation/misc-devices/mrvl_cn10k_dpi.rst
>> >> @@ -0,0 +1,52 @@
>> >> +.. SPDX-License-Identifier: GPL-2.0
>> >> +
>> >> +===============================================
>> >> +Marvell CN10K DMA packet interface (DPI) driver
>> >> +===============================================
>> >> +
>> >> +Overview
>> >> +========
>> >> +
>> >> +DPI is a DMA packet interface hardware block in Marvell's CN10K silicon.
>> >> +DPI hardware comprises a physical function (PF), its virtual
>> >> +functions, mailbox logic, and a set of DMA engines & DMA command
>> >queues.
>> >> +
>> >> +DPI PF function is an administrative function which services the
>> >> +mailbox requests from its VF functions and provisions DMA engine
>> >> +resources to it's VF functions.
>> >> +
>> >> +mrvl_cn10k_dpi.ko misc driver loads on DPI PF device and services
>> >> +the mailbox commands submitted by the VF devices and accordingly
>> >> +initializes the DMA engines and VF device's DMA command queues.
>> >> +Also, driver creates /dev/mrvl-cn10k-dpi node to set DMA engine
>> >> +and PEM (PCIe interface) port attributes like fifo length, molr, mps &
>mrrs.
>> >> +
>> >> +DPI PF driver is just an administrative driver to setup its VF
>> >> +device's queues and provisions the hardware resources, it cannot
>> >> +initiate any DMA operations. Only VF devices are provisioned with
>> >> +DMA
>> >capabilities.
>> >> +
>> >> +Driver location
>> >> +===============
>> >> +
>> >> +drivers/misc/mrvl_cn10k_dpi.c
>> >> +
>> >> +Driver IOCTLs
>> >> +=============
>> >> +
>> >> +:c:macro::`DPI_MPS_MRRS_CFG`
>> >> +ioctl that sets max payload size & max read request size
>> >> +parameters of a pem port to which DMA engines are wired.
>> >> +
>> >> +
>> >> +:c:macro::`DPI_ENGINE_CFG`
>> >> +ioctl that sets DMA engine's fifo sizes & max outstanding load
>> >> +request thresholds.
>> >> +
>> >> +User space code example
>> >> +=======================
>> >> +
>> >> +DPI VF devices are probed and accessed from user space
>> >> +applications using vfio-pci driver. Below is a sample dpi dma
>> >> +application to demonstrate on how applications use mailbox and
>> >> +ioctl services from DPI
>> >PF kernel driver.
>> >> +
>> >> +https://urldefense.proofpoint.com/v2/url?u=https-
>> <https://urldefense.proofpoint.com/v2/url?u=https-  >>
>> >3A__github.com_Marve
>> >> +llEmbeddedProcessors_dpi-2Dsample-
>> >2Dapp&d=DwIBAg&c=nKjWec2b6R0mOyPaz7
>> >>
>>
>>+xtfQ&r=WllrYaumVkxaWjgKto6E_rtDQshhIhik2jkvzFyRhW8&m=bFx_7eltw7
>S
>> >6zzVu
>> >>
>>
>>+1LNEdtsbwwynJfAKTja649QUwGNU_y4uWqGoEZ4f7JluYLjX&s=wOMzADb
>q
>> >9f4xxz1Oug
>> >> +-slj_xy4ZcbrnWfQJWeO0_ugA&e=
>> >> diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst
>> >> b/Documentation/userspace-api/ioctl/ioctl-number.rst
>> >> index a141e8e65c5d..def539770439 100644
>> >> --- a/Documentation/userspace-api/ioctl/ioctl-number.rst
>> >> +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
>> >> @@ -362,6 +362,7 @@ Code  Seq#    Include File
>> >Comments
>> >>  0xB6  all    linux/fpga-dfl.h
>> >>  0xB7  all    uapi/linux/remoteproc_cdev.h                            <mailto:linux-
>> >remoteproc@xxxxxxxxxxxxxxx>
>> >>  0xB7  all    uapi/linux/nsfs.h                                       <mailto:Andrei Vagin
>> ><avagin@xxxxxxxxxx>>
>> >> +0xB8  01-02  uapi/misc/mrvl_cn10k_dpi.h                              Marvell CN10K
>DPI
>> >driver
>> >>  0xC0  00-0F  linux/usb/iowarrior.h  0xCA  00-0F  uapi/misc/cxl.h
>> >> 0xCA  10-2F  uapi/misc/ocxl.h diff --git a/MAINTAINERS
>> >> b/MAINTAINERS index aae88b7a6c32..2c17d651954a 100644
>> >> --- a/MAINTAINERS
>> >> +++ b/MAINTAINERS
>> >> @@ -13477,6 +13477,11 @@ S:	Supported
>> >>  F:	Documentation/devicetree/bindings/mmc/marvell,xenon-sdhci.yaml
>> >>  F:	drivers/mmc/host/sdhci-xenon*
>> >>
>> >> +MARVELL OCTEON CN10K DPI DRIVER
>> >> +M:	Vamsi Attunuru <vattunuru@xxxxxxxxxxx>
>> >> +S:	Supported
>> >> +F:	drivers/misc/mrvl_cn10k_dpi.c
>> >> +
>> >>  MATROX FRAMEBUFFER DRIVER
>> >>  L:	linux-fbdev@xxxxxxxxxxxxxxx
>> >>  S:	Orphan
>> >> diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index
>> >> faf983680040..965641017a62 100644
>> >> --- a/drivers/misc/Kconfig
>> >> +++ b/drivers/misc/Kconfig
>> >> @@ -585,6 +585,20 @@ config NSM
>> >>  	  To compile this driver as a module, choose M here.
>> >>  	  The module will be called nsm.
>> >>
>> >> +config MARVELL_CN10K_DPI
>> >> +	tristate "Octeon CN10K DPI driver"
>> >> +	depends on ARM64 && PCI
>> >
>> >Why does ARM64 matter here?  I don't see any dependency required of it.
>> >
>> Thanks, Greg, for your time. This DPI device is an on-chip PCIe device
>> and only present on Marvell's CN10K platforms(which are 64-bit ARM SoC
>processors), so added those dependency.
>
>Then perhaps keep the ARM64 and add a COMPILE_TEST option as well so
>that we can build this as part of normal testing?
>
>So that would be:
>	depends on PCI && (ARM64 || COMPILE_TEST) right?
>
Yes, it makes sense to add. Can I send this fix as next version now so that it will show
up in next release, please suggest.

Regards
Vamsi

>thanks,
>
>greg k-h