Re: [RFC PATCH 1/6] docs: networking: add the document for DFL Ether Group driver

[Jakub Kicinski]

On Tue, 27 Oct 2020 01:38:04 +0800 Xu Yilun wrote:
> > > The line/host side Ether Group is not the terminal of the network data stream.
> > > Eth1 will not paticipate in the network data exchange to host.
> > > 
> > > The main purposes for eth1 are:
> > > 1. For users to monitor the network statistics on Line Side, and by comparing the
> > > statistics between eth0 & eth1, users could get some knowledge of how the User
> > > logic is taking function.
> > > 
> > > 2. Get the link state of the front panel. The XL710 is now connected to
> > > Host Side of the FPGA and the its link state would be always on. So to
> > > check the link state of the front panel, we need to query eth1.  
> > 
> > This is very non-intuitive. We try to avoid this in the kernel and the
> > API to userspace. Ethernet switches are always modelled as
> > accelerators for what the Linux network stack can already do. You
> > configure an Ethernet switch port in just the same way configure any
> > other netdev. You add an IP address to the switch port, you get the
> > Ethernet statistics from the switch port, routing protocols use the
> > switch port.
> > 
> > You design needs to be the same. All configuration needs to happen via
> > eth1.
> > 
> > Please look at the DSA architecture. What you have here is very
> > similar to a two port DSA switch. In DSA terminology, we would call
> > eth0 the master interface.  It needs to be up, but otherwise the user
> > does not configure it. eth1 is the slave interface. It is the user
> > facing interface of the switch. All configuration happens on this
> > interface. Linux can also send/receive packets on this netdev. The
> > slave TX function forwards the frame to the master interface netdev,
> > via a DSA tagger. Frames which eth0 receive are passed through the
> > tagger and then passed to the slave interface.
> > 
> > All the infrastructure you need is already in place. Please use
> > it. I'm not saying you need to write a DSA driver, but you should make
> > use of the same ideas and low level hooks in the network stack which
> > DSA uses.  
> 
> I did some investigation about the DSA, and actually I wrote a
> experimental DSA driver. It works and almost meets my need, I can make
> configuration, run pktgen on slave inf.
> 
> A main concern for dsa is the wiring from slave inf to master inf depends
> on the user logic. If FPGA users want to make their own user logic, they
> may need a new driver. But our original design for the FPGA is, kernel
> drivers support the fundamental parts - FPGA FIU (where Ether Group is in)
> & other peripherals on board, and userspace direct I/O access for User
> logic. Then FPGA user don't have to write & compile a driver for their
> user logic change.
> It seems not that case for netdev. The user logic is a part of the whole
> functionality of the netdev, we cannot split part of the hardware
> component to userspace and the rest in kernel. I really need to
> reconsider this.

This is obviously on purpose. Your design as it stands will not fly
upstream, sorry.

From netdev perspective the user should not care how many hardware
blocks are in the pipeline, and on which piece of silicon. You have 
a 2 port (modulo port splitting) card, there should be 2 netdevs, and
the link config and forwarding should be configured through those.

Please let folks at Intel know that we don't like the "SDK in user
space with reuse [/abuse] of parts of netdev infra" architecture.
This is a second of those we see in a short time. Kernel is not a
library for your SDK to use.