Re: [PATCH v2] staging: fsl-mc: add DPAA2 overview readme
From: Tillmann Heidsieck
Date: Tue Aug 11 2015 - 07:25:48 EST
Hi Stuart,
I am by no means a native speaker, but I have proof-read my fair share of articles and theses, so here are a bunch of suggestions which might or might not help improve you document.
On 07.08.2015 03:09, Stuart Yoder wrote:
> add README file providing an overview of the DPAA2 architecture
> and how it is integrated in Linux
>
> Signed-off-by: Stuart Yoder <stuart.yoder@xxxxxxxxxxxxx>
> ---
> -v2: added changelog text
>
> drivers/staging/fsl-mc/README.txt | 364 ++++++++++++++++++++++++++++++++++++++
> drivers/staging/fsl-mc/TODO | 4 -
> 2 files changed, 364 insertions(+), 4 deletions(-)
> create mode 100644 drivers/staging/fsl-mc/README.txt
>
> diff --git a/drivers/staging/fsl-mc/README.txt b/drivers/staging/fsl-mc/README.txt
> new file mode 100644
> index 0000000..8214102
> --- /dev/null
> +++ b/drivers/staging/fsl-mc/README.txt
> @@ -0,0 +1,364 @@
> +Copyright (C) 2015 Freescale Semiconductor Inc.
> +
> +DPAA2 (Data Path Acceleration Architecture Gen2)
> +------------------------------------------------
> +
> +This document provides an overview of the Freescale DPAA2 architecture
> +and how it is integrated into the Linux kernel.
> +
> +Contents summary
> + -DPAA2 overview
> + -Overview of DPAA2 objects
> + -DPAA2 Linux driver architecture overview
> + -bus driver
> + -dprc driver
- DPRC driver
> + -allocator
> + -dpio driver
- DPIO driver
> + -Ethernet
> + -mac
mac -> MAC
> +
> +DPAA2 Overview
> +--------------
> +
> +DPAA2 is a hardware architecture designed for high-speeed network
> +packet processing. DPAA2 consists of sophisticated mechanisms for
> +processing Ethernet packets, queue management, buffer management,
> +autonomous L2 switching, virtual Ethernet bridging, and accelerator
> +(e.g. crypto) sharing.
> +
> +A DPAA2 hardware component called the Management Complex (or MC) manages the
> +DPAA2 hardware resources. The MC provides an object-based abstraction for
> +software drivers to use the DPAA2 hardware.
> +
> +The MC uses DPAA2 hardware resources such as queues, buffer pools, and
> +network ports to create functional objects/devices such as network
> +interfaces, an L2 switch, or accelerator instances.
> +
> +The MC provides memory-mapped I/O command interfaces (MC portals)
> +which DPAA2 software drivers use to operate on DPAA2 objects:
> +
> + +--------------------------------------+
> + | OS |
> + | DPAA2 drivers |
> + | | |
> + +-----------------------------|--------+
> + |
> + | (create,discover,connect
> + | config,use,destroy)
> + |
> + DPAA2 |
> + +------------------------| mc portal |-+
> + | | |
> + | +- - - - - - - - - - - - -V- - -+ |
> + | | | |
> + | | Management Complex (MC) | |
> + | | | |
> + | +- - - - - - - - - - - - - - - -+ |
> + | |
> + | Hardware Hardware |
> + | Resources Objects |
> + | --------- ------- |
> + | -queues -DPRC |
> + | -buffer pools -DPMCP |
> + | -Eth MACs/ports -DPIO |
> + | -network interface -DPNI |
> + | profiles -DPMAC |
> + | -queue portals -DPBP |
> + | -MC portals ... |
> + | ... |
> + | |
> + +--------------------------------------+
> +
> +The MC mediates operations such as create, discover,
> +connect, configuration, and destroy. Fast-path operations
> +on data, such as packet transmit/receive, are not mediated by
> +the MC and are done directly using memory mapped regions in
> +DPIO objects.
> +
> +Overview of DPAA2 Objects
> +-------------------------
> +The section provides a brief overview of some key objects
> +in the DPAA2 hardware. A simple scenario is described illustrating
> +the objects involved in creating a network interfaces.
> +
> +-DPRC (Datapath Resource Container)
> +
> + A DPRC is an container object that holds all the other
... is a container ...
> + types of DPAA2 objects. In the example diagram below there
> + are 8 objects of 5 types (DPMCP, DPIO, DPBP, DPNI, and DPMAC)
> + in the container.
> +
> + +---------------------------------------------------------+
> + | DPRC |
> + | |
> + | +-------+ +-------+ +-------+ +-------+ +-------+ |
> + | | DPMCP | | DPIO | | DPBP | | DPNI | | DPMAC | |
> + | +-------+ +-------+ +-------+ +---+---+ +---+---+ |
> + | | DPMCP | | DPIO | |
> + | +-------+ +-------+ |
> + | | DPMCP | |
> + | +-------+ |
> + | |
> + +---------------------------------------------------------+
> +
> + From the point of view of an OS, a DPRC is bus-like. Like
maybe replace "is bus-like" with "behaves similar to a bus" or "can be viewed as a bus"
> + a plug-and-play bus, such as PCI, DPRC commands can be used to
maybe replace "Like a pnp bus..." with "Similar to a plug-and-play bus, such as PCI, DPRC ..."
> + enumerate the contents of the DPRC, discover the hardware
> + objects present (including mappable regions and interrupts).
> +
> + dprc.1 (bus)
> + |
> + +--+--------+-------+-------+-------+
> + | | | | |
> + dpmcp.1 dpio.1 dpbp.1 dpni.1 dpmac.1
> + dpmcp.2 dpio.2
> + dpmcp.3
> +
> + Hardware objects can be created and destroyed dynamically, providing
> + the ability to hot plug/unplug objects in and out of the DPRC.
> +
> + A DPRC has a mappable mmio region (an MC portal) that can be used
- mmio -> MMIO (not sure whether mappable MMIO is redundant or not)
- a MC portal
> + to send MC commands. It has an interrupt for status events (like
> + hotplug).
> +
> + All objects in a container share the same hardware "isolation context".
> + This means that with respect to an IOMMU the isolation granularity
> + is at the DPRC (container) level, not at the individual object
> + level.
> +
> + DPRCs can be defined statically and populated with objects
> + via a config file passed to the MC when firmware starts
> + it. There is also a Linux user space tool called "restool"
> + that can be used to create/destroy containers and objects
> + dynamically.
> +
> +-DPAA2 Objects for an Ethernet Network Interface
> +
> + A typical Ethernet NIC is monolithic-- the NIC device contains TX/RX
> + queuing mechanisms, configuration mechanisms, buffer management,
> + physical ports, and interrupts. DPAA2 uses a more granular approach
> + utilizing multiple hardware objects. Each object has specialized
> + functions, and are used together by software to provide Ethernet network
Each object provides specialized functions. Groups of these objects are used by the software to provide Ethernet network interface functionality.
> + interface functionality. This approach provides efficient use of finite
> + hardware resources, flexibility, and performance advantages.
> +
> + The diagram below shows the objects needed for a simple
> + network interface configuration on a system with 2 CPUs.
> +
> + +---+---+ +---+---+
> + CPU0 CPU1
> + +---+---+ +---+---+
> + | |
> + +---+---+ +---+---+
> + DPIO DPIO
> + +---+---+ +---+---+
> + \ /
> + \ /
> + \ /
> + +---+---+
> + DPNI --- DPBP,DPMCP
> + +---+---+
> + |
> + |
> + +---+---+
> + DPMAC
> + +---+---+
> + |
> + port/PHY
> +
> + Below the objects are described. For each object a brief description
The objects depicted in this figure are described below.
> + is provided along with a summary of the kinds of operations the object
> + supports and a summary of key resources of the object (mmio regions
> + and irqs).
mmio -> MMIO
irqs -> IRQs
> +
> + -DPMAC (Datapath Ethernet MAC): represents an Ethernet MAC, a
> + hardware device that connects to an Ethernet PHY and allows
> + physical transmission and reception of Ethernet frames.
> + -mmio regions: none
mmio -> MMIO
> + -irqs: dpni link change
irqs -> IRQs
dpni -> DPNI
> + -commands: set link up/down, link config, get stats,
> + irq config, enable, reset
irq -> IRQ
> +
> + -DPNI (Datapath Network Interface): contains TX/RX queues,
> + network interface configuration, and rx buffer pool configuration
rx -> RX
> + mechanisms.
> + -mmio regions: none
mmio -> MMIO
> + -irqs: link state
irqs -> IRQs
> + -commands: port config, offload config, queue config,
> + parse/classify config, irq config, enable, reset
irq -> IRQ
> +
> + -DPIO (Datapath I/O): provides interfaces to enqueue and dequeue
> + packets and do hardware buffer pool management operations. For
> + optimum performance there is typically DPIO per CPU. This allows
For optimum performance there is typically one DPIO assigned to each CPU
> + each CPU to perform simultaneous enqueue/dequeue operations.
This allows different CPUs to simultaneously perform enqueue/dequeue operations.
> + -mmio regions: queue operations, buffer mgmt
mmio -> MMIO
mgmt -> management
> + -irqs: data availability, congestion notification, buffer
> + pool depletion
irqs -> IRQs
> + -commands: irq config, enable, reset
irq -> IRQ
> +
> + -DPBP (Datapath Buffer Pool): represents a hardware buffer
> + pool.
> + -mmio regions: none
> + -irqs: none
> + -commands: enable, reset
mmio -> MMIO
irqs -> IRQs
> +
> + -DPMCP (Datapath MC Portal): provides an MC command portal.
> + Used by drivers to send commands to the MC to manage
> + objects.
> + -mmio regions: MC command portal
> + -irqs: command completion
> + -commands: irq config, enable, reset
mmio -> MMIO
irqs -> IRQs
irq -> IRQ
> +
> + Object Connections
> + ------------------
> + Some objects have explicit relationships that must
> + be configured:
> +
> + -DPNI <--> DPMAC
> + -DPNI <--> DPNI
> + -DPNI <--> L2-switch-port
> + A DPNI must be connected to something such as a DPMAC,
> + another DPNI, or L2 switch port. The DPNI connection
> + is made via a DPRC command.
> +
> + +-------+ +-------+
> + | DPNI | | DPMAC |
> + +---+---+ +---+---+
> + | |
> + +==========+
> +
> + -DPNI <--> DPBP
> + A network interface requires a 'buffer pool' (DPBP
> + object) which provides a list of pointers to memory
> + where received Ethernet data is to be copied. The
> + Ethernet driver configures the DPBPs associated with
> + the network interface.
> +
> + Interrupts
> + ----------
> + All interrupts generated by DPAA2 objects are message
> + interrupts. At the hardware level message interrupts
> + generated by devices will normally have 3 components--
> + 1) a non-spoofable 'device-id' expressed on the hardware
> + bus, 2) an address, 3) a data value.
> +
> + In the case of DPAA2 devices/objects, all objects in the
> + same container/DPRC share the same 'device-id'.
> + For ARM-based SoC this is the same as the stream ID.
> +
> +
> +DPAA2 Linux Driver Overview
> +---------------------------
> +
> +This section provides an overview of the Linux kernel drivers for
> +DPAA2-- 1) the bus driver and associated "DPAA2 infrastructure"
> +drivers and 2) functional object drivers (such as Ethernet).
> +
> +As described previously, a DPRC is a container that holds the other
> +types of DPAA2 objects. It is functionally similar to a plug-and-play
> +bus controller.
> +
> +Each object in the DPRC is a Linux "device" and is bound to a driver.
> +The diagram below shows the Linux drivers involved in a networking
> +scenario and the objects bound to each driver. A brief description
> +of each driver follows.
> +
> + +------------+
> + | OS Network |
> + | Stack |
> + +------------+ +------------+
> + | Allocator |. . . . . . . | Ethernet |
> + |(dpmcp,dpbp)| | (dpni) |
> + +-.----------+ +---+---+----+
> + . . ^ |
> + . . <data avail, | |<enqueue,
> + . . tx confirm> | | dequeue>
> + +-------------+ . | |
> + | DPRC driver | . +---+---V----+ +---------+
> + | (dprc) | . . . . . .| DPIO driver| | MAC |
> + +----------+--+ | (dpio) | | (dpmac) |
> + | +------+-----+ +-----+---+
> + |<dev add/remove> | |
> + | | |
> + +----+--------------+ | +--+---+
> + | mc-bus driver | | | PHY |
> + | | | |driver|
> + | /fsl-mc@80c000000 | | +--+---+
> + +-------------------+ | |
> + | |
> + ================================ HARDWARE =========|=================|======
> + DPIO |
> + | |
> + DPNI---DPBP |
> + | |
> + DPMAC |
> + | |
> + PHY ---------------+
> + ===================================================|========================
> +
> +A brief description of each driver is provided below.
> +
> + mc-bus driver
> + -------------
mc-bus -> MC-Bus or MC-bus
> + The mc-bus driver is a platform driver and is probed from an
> + "/fsl-mc@xxxx" node in the device tree passed in by boot firmware.
> + It is responsible for bootstrapping the DPAA2 kernel infrastructure.
> + Key functions include:
> + -registering a new bus type named "fsl-mc" with the kernel,
> + and implementing bus call-backs (e.g. match/uevent/dev_groups)
> + -implemeting APIs for DPAA2 driver registration and for device
> + add/remove
> + -creates an MSI irq domain
irq -> IRQ
> + -do a device add of the 'root' DPRC device, which is needed
> + to bootstrap things
> +
> + DPRC driver
> + -----------
> + The dprc-driver is bound DPRC objects and does runtime management
> + of a bus instance. It performs the initial bus scan of the DPRC
> + and handles interrupts for container events such as hot plug.
> +
> + Allocator
> + ----------
> + Certain objects such as DPMCP and DPBP are generic and fungible,
> + and are intended to be used by other drivers. For example,
> + the DPAA2 Ethernet driver needs:
> + -DPMCPs to send MC commands, to configure network interfaces
> + -DPBPs for network buffer pools
> +
> + The allocator driver registers for these allocatable object types
> + and those objects are bound to the allocator when the bus is probed.
> + The allocator maintains a pool of objects that are available for
> + allocation by other DPAA2 drivers.
> +
> + DPIO driver
> + -----------
> + The DPIO driver is bound to DPIO objects and provides services that allow
> + other drivers such as the Ethernet driver to receive and transmit data.
> + Key services include:
> + -data availability notifications
> + -hardware queuing operations (enqueue and dequeue of data)
> + -hardware buffer pool management
> +
> + There is typically one DPIO object per physical CPU for optimum
> + performance, allowing each CPU to simultaneously enqueue
> + and dequeue data.
allowing different CPUs to simultaneously ...
> +
> + The DPIO driver operates on behalf of all DPAA2 drivers
> + active in the kernel-- Ethernet, crypto, compression,
> + etc.
> +
> + Ethernet
> + --------
> + The Ethernet driver is bound to a DPNI and implements the kernel
> + interfaces needed to connect the DPAA2 network interface to
> + the network stack.
> +
> + Each DPNI corresponds to a Linux network interface.
> +
> + MAC driver
> + ----------
> + An Ethernet PHY is an off-chip, board specific component and is managed
> + by the appropriate PHY driver via an mdio bus. The MAC driver
> + plays a role of being a proxy between the PHY driver and the
> + MC. It does this proxy via the MC commands to a DPMAC object.
> diff --git a/drivers/staging/fsl-mc/TODO b/drivers/staging/fsl-mc/TODO
> index c29516b..3894368 100644
> --- a/drivers/staging/fsl-mc/TODO
> +++ b/drivers/staging/fsl-mc/TODO
> @@ -1,7 +1,3 @@
> -* Add README file (with ASCII art) describing relationships between
> - DPAA2 objects and how combine them to make a NIC, an LS2 switch, etc.
> - Also, define all acronyms used.
> -
> * Decide if multiple root fsl-mc buses will be supported per Linux instance,
> and if so add support for this.
>
Cheers
Tillmann
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/