RE: [PATCH 3/3] net: macb: add support for high speed interface

[Milind Parab]

>I'm still not getting a good enough view of what you are doing and
>how my understanding of your hardware fits with what you're doing
>with the software.
>
>My understanding is it's something like:
>
>	----+
>        SOC |             PCS
>	MAC --(USXGMII)-- PHY ----- PHY or SFP
>	    |
>	----+
>
>And you are just modelling the MAC part in phylink, where as phylink
>has so far been used on systems which have this model - where phylink
>knows about both the MAC and the PCS PHY:
>
>	---------------+
>	         PCS   |
>	MAC ---- PHY ----- PHY or SFP
>	     SOC       |
>      	---------------+
>

Here, I am describing the GEM component followed by the test setup. 
Please see, if the below explanation is depicting the correct picture

The Cadence MAC, referred to as GEM is a hardware module which implements 10/100/1000Mbps Ethernet MAC 
with the following interface types: 
MII, RMII, GMII, RGMII which can operate in either half or full duplex mode; 
and also as a separate instance a full duplex 10G MAC with an XGMII interface.
GEM comprises the following constituent components:
- MAC controlling transmit, receive, address checking and loopback
- Configuration registers (REG_TOP) providing control and status registers, statistics registers and synchronization logic
- Two Physical Coding Sublayer (PCS) components; one comprising 8B/10B encode/decode, PCS transmit, PCS
receive, and PCS auto-negotiation and another implementing USXGMII functionality.

As our ethernet controller (GEM) have MAC as well as USXGMII PCS, we program both appropriately based on the values passed from Phylink. 
And for fixed-link,  Phylink correctly read out these values from device tree node and relay it to mac_config.
Also note that we are not setting "sfp" node in device tree.

We are modelling MAC + USXGMII PCS. 
The test configuration is as below

                                    +-------------------------+
                                    |              GEM        |
Host PC1   < ------------------->   | MAC ---- USXGMII PCS    |----- SerDes ------ SFP+ <------ Direct attach cable ------->  Chelsio 10G Card   <--->   Host PC2
                                    |        PCI based NIC    | 
                                    +-------------------------+


The setup has a 10G fixed link between PCS and SFP+.
Our test setup is emulated on Xilinx Virtex VCU118 FPGA base board placed in a PCIe slot of a PC running Ubuntu. 
The FPGA base board contained an image implementing a NIC using Cadence PCIe and Ethernet IP cores. 
We had a separate PC also running Ubuntu containing  a Chelsio 10G NIC. 
We connected these with an SFP+ direct attach passive twinx-ax copper cable. Testing of the link was done with ping and iperf3
Also note that there is no PHY in the SFP+ cage. 
As I understand it everything regarding SFP+ in our setup is passive so there is nothing for PHYLINK to talk to. 
We do not have a module in the SFP+ cage, just a direct connection to a copper cable

>This is why I recently renamed mac_link_state() to mac_pcs_get_state()
>to make it clearer that it reads from the PCS not from the current
>settings of the MAC.  So far, all such setups do not implement the PCS
>PHY as an 802.3 register set; they implement it as part of the MAC
>register set.
>
>In the former case, if phylink is used to manage the connection between
>the MAC and the PCS PHY, phylink has nothing to do with the SFP at all.
>
>In the latter case, phylink is used to manage the connection between the
>PCS PHY and external device, controlling the MAC as appropriate.
>
>My problem is I believe your hardware is the former case, but you are
>trying to implement the latter case by ignoring in-band mode.  As SFPs
>rely on in-band mode, that isn't going to work.
>
>The options for the former case are:
>
>1) implement phylink covering both the MAC and the external PCS PHY
>2) implement phylink just for the MAC to PCS PHY connection but not
>   SFPs, and implement SFP support separately in the PCS PHY driver.
>
>Maybe phylink needs to split mac_pcs_get_state() so it can be supplied
>by a separate driver, or by the MAC driver as appropriate - but that
>brings with it other problems; phylink with a directly attached SFP
>considers the state of the link between the PCS PHY and the external
>device - not only speed but also interface mode for that part of the
>link.  What you'd see in the mac_config() callback are interface modes
>for that part of the link, not between the MAC and the PCS PHY.
>
>To change that would require reworking almost every driver that has
>already converted over to somehow remodel the built-in PCS and
>COMPHY as a separate PCS PHY for phylink. I'm not entirely clear
>whether that would work though.
>
>--
>RMK's Patch system: https://urldefense.proofpoint.com/v2/url?u=https-
>3A__www.armlinux.org.uk_developer_patches_&d=DwIBAg&c=aUq983L2pue
>2FqKFoP6PGHMJQyoJ7kl3s3GZ-
>_haXqY&r=BDdk1JtITE_JJ0519WwqU7IKF80Cw1i55lZOGqv2su8&m=ei26OYsu0
>JYGaBZjJMg7WhXT8l_kdzu_QwlOu3RTUhY&s=YHxF2EUKwhleTZ-
>fO9lorELZWnn9kArzxliO1KM0uMc&e=
>
>FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps
>up
>
>According to speedtest.net: 11.9Mbps down 500kbps up