Re: [PATCH net-next V2 4/5] net: lan743x: Implement phylink pcs

[Andrew Lunn]

On Fri, Sep 13, 2024 at 02:23:03PM +0000, Ronnie.Kunin@xxxxxxxxxxxxx wrote:
> > > It's my mistake. We don't need 2 MDIO buses.
> > > If SFP present, XPC's MDIO bus can use, If not sfp, LAN743x MDIO bus
> > > can use.
> > 
> > I still think this is wrong. Don't focus on 'sfp present'.
> > 
> > Other MAC drivers don't even know there is an SFP cage connected vs a PHY. They just tell phylink the list
> > of link modes they support, and phylink tells it which one to use when the media has link.
> > 
> > You have a set of hardware building blocks, A MAC, a PCS, an MDIO bus.
> > Use the given abstractions in Linux to export them to the core, and then let Linux combine them as
> > needed.
> > 
> > Back to my question about EEPROM vs fuses. I assume it is an EEPROM, ...
> 
> How RGMII vs "SGMII-BASEX"  is controlled ?
> The hardware default is RGMII. That can be overwritten by OTP (similar functionality to EFuse, inside the PCI11010), which also can be further overwritten by EEPROM (outside the PCI11010). That will setup the initial value the device will have by the time the software first sees it. But it is a live bit in a register, so it can be changed at runtime if it was desired.

In a DT system phy-mode will tell you this. You don't have DT, at
least not at the moment, so your EEPROM makes sense for this, the
RGMII vs "SGMII-BASEX" bit.

> 
> > ... and the PCS always exists. So
> > always instantiate an MDIO bus and instantiate the PCS. The MDIO bus always exists, so instantiate an
> > MDIO bus.
> 
> No, you can't do that with this device because:
> - There are shared pins in the chip between RGMII and SGMII/BASEX

Which is typical in SoC. What you generally have is lots of IP blocks,
I2C, SPI, GPIO, PWM, NAND controllers etc. The total number of
inputs/outputs of these blocks is more than the legs on the chip. You
then have a pinmux, which connects the internal blocks to the pins.

All the devices exist, but only a subset is connected to the outside
world.

For RGMII vs SGMII/BASEX, it probably does not make sense to
instantiate the PCS in RGMII mode. However, in SGMII/BASEX it should
always exist, because it is connected to the outside world.

> - Furthermore, I need to check with the HW architect, but I suspect
>   that the block that was not selected is shutdown to save power as
>   well.

I would also expect that when the PCS device is probed, it is left in
a lower power state. For an external PHY, you don't need the PCS
running until the PHY has link, autoneg has completed, and phylink
will tell you to configure the PCS to SGMII or 2500BaseX. For an SFP,
you need to read out the contents of the SFP EEPROM, look for LOS to
indicate there is link, and then phylink will determine SGMII,
1000BaseX or 2500BaseX and tell you how to configure it. It is only at
that point do you need to take the PCS out of low power mode.

Independent of RGMII vs SGMII/BASEX, the MDIO bus always exists. Both
modes need it. And Linux just considers it an MDIO, not necessarily an
MDIO bus for this MAC. So i would expect to always see a fully
functioning MDIO bus.

One of the weird and wonderful use cases: There are lots of ComExpress
boards with Intel 10G Ethernet interfaces. There are developers who
create base boards for them with Ethernet switches. They connect the
10G interface to one port of the switch. But to manage the switch they
need MDIO. The Intel 10G drivers bury the MDIO in firmware, Linux
cannot access is. So they are forced to use three GPIOs and bitbang
MDIO. It is slow. Now imaging i put your device on the baseboard. I
use its MAC connected to a 1G/2.5G PHY, on the MDIO bus, which i uses
as the management interface for the box. Additional i connect the MDIO
bus to the switch, to manage the switch. Linux has no problems with
this, MDIO is just a bus with devices on it. But phylink will want
access to the PCS to switch it between SGMII and 2500BaseX depending
on what the PHY negotiates. Plus we need C45 to talk to the switch.

The proposed hijacking for C45 from the MDIO bus to talk to the PCS
when there is an SFP breaks this, and as far as i can see, for no real
reason other than being too lazy to put the PCS on its own Linux MDIO
bus.

      Andrew