Re: [PATCH net-next v6 1/2] net: dsa: realtek: rtl8365mb: add SGMII support for RTL8367S

[Stanislaw]

Hi Luiz, Mieczyslaw,

> Stanislaw, do you have a reset pin or a reset controller configured?

Yes - a reset GPIO, no reset controller:

  reset-gpios = <&tlmm 33 GPIO_ACTIVE_LOW>;

on the switch node. So on this board the probe path does run both
resets: the GPIO hard reset from rtl83xx_probe() plus the soft reset
during setup. That is worth stressing, because the re-probe that cures
the cold-soak state performs *exactly the same sequence* - same GPIO
reset, same soft reset, same full init - as the probe that lands in
the bad state. The only difference is that it is the second run. So
it does not look like a missing reset; more like something about the
very first init pass on cold silicon.

On "loading too early": the driver is a module here and probes ~20 s
after power-on (rootfs mount), so the rails have long been up by then.
That said, 20 s of a cold board is a different thermal/electrical
state than the same board a minute after a reboot, and short
power-cycles (~1 min) never reproduce the issue - only a multi-hour
soak does.

Your reset-bit map is exactly the experiment this bench needs. Next
cold-bad morning I will try the individual reset bits (SDS_RST, SW_RST,
CHIP_RST, DW8051_RST...) on the degraded state, before falling back to
the full re-probe, to find the minimal action that cures it. I'll
report what I find.

> Please repeat the test only with the fix "the ~98 ms PLL-settle delay
> plus the 0x060C-0x060F Local Jam Table analog calibration".

Done - last night's cold soak ran an image with only those two changes
on top of the v6 series (busy-wait reverted). Result: the bad state
came up as usual, no improvement - consistent with Johan's analysis of
where those writes actually land.

This time I had ethtool on the box before curing it, so here is the
per-port MIB of the bad state (~35 min after the cold boot, switch
reporting 2.5G/Full link up on the SerDes, wan 1G/Full to the wire):

  CPU -> switch direction (broken):
    eth0 (SoC gmac1):        995 frames / 282,945 octets tx, no errors
    switch port 6 (SerDes):  dot3StatsFCSErrors: 506
                             etherStatsDropEvents: 506
    wan wire egress:         zero - no ifOut counters at all, although
                             the kernel handed ~990 frames for tx

  switch -> CPU direction (clean):
    p06_ifOutOctets: 172,231 == eth0 rx octet count: 172,231
    byte-exact, zero errors - that's why ingress from upstream works

So the cold-soak bad state is now precisely characterized: over half of
the frames arriving from the SoC FCS-fail at the switch SerDes ingress
and are dropped (and nothing reaches the wan wire), while the switch's
transmit side is byte-exact clean. Since a full re-probe of the switch
alone cures it - the SoC side is never touched - the bad half is the
switch's SerDes receiver, left in that state by the first cold init.

Next cold morning I'll do the reset-bit isolation on the degraded
state, per Luiz's map, before falling back to the full re-probe.

Best regards,
Stanislaw