Re: R8169: Network lockups in 4.18.{8,9,10} (and 4.19 dev)

From: Chris Clayton
Date: Tue Oct 09 2018 - 10:40:22 EST

Next message: David Woodhouse: "Re: [PATCH 0/2] xen: fix two issues in Xen pv qspinlock handling"
Previous message: Tony Lindgren: "Re: [RFC PATCH 02/11] dt-bindings: phy: add cpsw port interface mode selection phy bindings"
In reply to: Maciej S. Szmigiero: "Re: R8169: Network lockups in 4.18.{8,9,10} (and 4.19 dev)"
Next in thread: Heiner Kallweit: "Re: R8169: Network lockups in 4.18.{8,9,10} (and 4.19 dev)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Thanks to Maciej and Heiner for their replies.

On 09/10/2018 13:32, Maciej S. Szmigiero wrote:
> On 07.10.2018 21:36, Chris Clayton wrote:
>> Hi again,
>>
>> I didn't think there was anything in 4.19-rc7 to fix this regression, but tried it anyway. I can confirm that the
>> regression is still present and my network still fails when, after a resume from suspend (to ram or disk), I open my
>> browser or my mail client. In both those cases the failure is almost immediate - e.g. my home page doesn't get displayed
>> in the browser. Pinging one of my ISPs name servers doesn't fail quite so quickly but the reported time increases from
>> 14-15ms to more than 1000ms.
>
> You can try comparing chip registers (ethtool -d eth0) in the working
> state (before a suspend) and in the broken state (after a resume).
> Maybe there will be some obvious in the difference.
>
> The same goes for the PCI configuration (lspci -d :8168 -vv).
>
Maciej suggested comparing the output from lspci -vv for the ethernet device. They are identical.

Both Maciej and Heiner suggested comparing the output from "ethtool -d" pre and post suspend. Again, they are identical.
Heiner specifically suggested looking at the RxConfig. The value of that is 0x0002870e both pre and post suspend.

I've attached files I redirected the outputs to.

Please don't hesitate to ask for any other information needed to solve this problem. In the meantime, I've now got
scripts that stop the network during suspend and restart it during resume. (Those scripts were removed whilst I gathered
the diagnostics shown in the attachments.)

Chris

>> Chris
>
> Maciej
>
ethtool -d eth0
===============
RealTek RTL8411 registers:
--------------------------------------------------------
0x00: MAC Address 80:fa:5b:08:d0:3d
0x08: Multicast Address Filter 0x00000000 0x00000080
0x10: Dump Tally Counter Command 0x0c2ec000 0x00000004
0x20: Tx Normal Priority Ring Addr 0x07a0a000 0x00000004
0x28: Tx High Priority Ring Addr 0x00000000 0x00000000
0x30: Flash memory read/write 0x00000000
0x34: Early Rx Byte Count 0
0x36: Early Rx Status 0x00
0x37: Command 0x0c
Rx on, Tx on
0x3C: Interrupt Mask 0x803f
SERR LinkChg RxNoBuf TxErr TxOK RxErr RxOK
0x3E: Interrupt Status 0x0000

0x40: Tx Configuration 0x4b800f80
0x44: Rx Configuration 0x0002870e
0x48: Timer count 0x00000000
0x4C: Missed packet counter 0x000000
0x50: EEPROM Command 0x10
0x51: Config 0 0x00
0x52: Config 1 0xcf
0x53: Config 2 0x3c
0x54: Config 3 0x60
0x55: Config 4 0x10
0x56: Config 5 0x02
0x58: Timer interrupt 0x00000000
0x5C: Multiple Interrupt Select 0x0000
0x60: PHY access 0x80040de1
0x64: TBI control and status 0x27ffff01
0x68: TBI Autonegotiation advertisement (ANAR) 0xf70c
0x6A: TBI Link partner ability (LPAR) 0x0002
0x6C: PHY status 0xeb
0x84: PM wakeup frame 0 0x00000000 0x00000000
0x8C: PM wakeup frame 1 0x00000000 0x00000000
0x94: PM wakeup frame 2 (low) 0x00000000 0x00000000
0x9C: PM wakeup frame 2 (high) 0x00000000 0x00000000
0xA4: PM wakeup frame 3 (low) 0x00000000 0x00000000
0xAC: PM wakeup frame 3 (high) 0x00000000 0x00000000
0xB4: PM wakeup frame 4 (low) 0xffffffff 0xffffffff
0xBC: PM wakeup frame 4 (high) 0x00000000 0x00000000
0xC4: Wakeup frame 0 CRC 0x0000
0xC6: Wakeup frame 1 CRC 0x0000
0xC8: Wakeup frame 2 CRC 0x0000
0xCA: Wakeup frame 3 CRC 0x0000
0xCC: Wakeup frame 4 CRC 0x0000
0xDA: RX packet maximum size 0x4000
0xE0: C+ Command 0x20e1
VLAN de-tagging
RX checksumming
0xE2: Interrupt Mitigation 0x5151
TxTimer: 5
TxPackets: 1
RxTimer: 5
RxPackets: 1
0xE4: Rx Ring Addr 0x07935000 0x00000004
0xEC: Early Tx threshold 0x27
0xF0: Func Event 0x0040003f
0xF4: Func Event Mask 0x00000000
0xF8: Func Preset State 0x00031eff
0xFC: Func Force Event 0x00000000

lspci -d :8168 -vv
==================
pcilib: sysfs_read_vpd: read failed: Input/output error
05:00.2 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0a)
Subsystem: CLEVO/KAPOK Computer RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 19
Region 0: I/O ports at e000 [size=256]
Region 2: Memory at f0004000 (64-bit, prefetchable) [size=4K]
Region 4: Memory at f0000000 (64-bit, prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [70] Express (v2) Endpoint, MSI 01
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
Vector table: BAR=4 offset=00000000
PBA: BAR=4 offset=00000800
Capabilities: [d0] Vital Product Data
Not readable
Kernel driver in use: r8169
Kernel modules: r8169
ethtool -d eth0
===============
RealTek RTL8411 registers:
--------------------------------------------------------
0x00: MAC Address 80:fa:5b:08:d0:3d
0x08: Multicast Address Filter 0x00000000 0x00000080
0x10: Dump Tally Counter Command 0x0c2ec000 0x00000004
0x20: Tx Normal Priority Ring Addr 0x07a0a000 0x00000004
0x28: Tx High Priority Ring Addr 0x00000000 0x00000000
0x30: Flash memory read/write 0x00000000
0x34: Early Rx Byte Count 0
0x36: Early Rx Status 0x00
0x37: Command 0x0c
Rx on, Tx on
0x3C: Interrupt Mask 0x803f
SERR LinkChg RxNoBuf TxErr TxOK RxErr RxOK
0x3E: Interrupt Status 0x0000

0x40: Tx Configuration 0x4b800f80
0x44: Rx Configuration 0x0002870e
0x48: Timer count 0x00000000
0x4C: Missed packet counter 0x000000
0x50: EEPROM Command 0x10
0x51: Config 0 0x00
0x52: Config 1 0xcf
0x53: Config 2 0x3c
0x54: Config 3 0x60
0x55: Config 4 0x10
0x56: Config 5 0x02
0x58: Timer interrupt 0x00000000
0x5C: Multiple Interrupt Select 0x0000
0x60: PHY access 0x80040de1
0x64: TBI control and status 0x27ffff01
0x68: TBI Autonegotiation advertisement (ANAR) 0xf70c
0x6A: TBI Link partner ability (LPAR) 0x0002
0x6C: PHY status 0xeb
0x84: PM wakeup frame 0 0x00000000 0x00000000
0x8C: PM wakeup frame 1 0x00000000 0x00000000
0x94: PM wakeup frame 2 (low) 0x00000000 0x00000000
0x9C: PM wakeup frame 2 (high) 0x00000000 0x00000000
0xA4: PM wakeup frame 3 (low) 0x00000000 0x00000000
0xAC: PM wakeup frame 3 (high) 0x00000000 0x00000000
0xB4: PM wakeup frame 4 (low) 0xffffffff 0xffffffff
0xBC: PM wakeup frame 4 (high) 0x00000000 0x00000000
0xC4: Wakeup frame 0 CRC 0x0000
0xC6: Wakeup frame 1 CRC 0x0000
0xC8: Wakeup frame 2 CRC 0x0000
0xCA: Wakeup frame 3 CRC 0x0000
0xCC: Wakeup frame 4 CRC 0x0000
0xDA: RX packet maximum size 0x4000
0xE0: C+ Command 0x20e1
VLAN de-tagging
RX checksumming
0xE2: Interrupt Mitigation 0x5151
TxTimer: 5
TxPackets: 1
RxTimer: 5
RxPackets: 1
0xE4: Rx Ring Addr 0x07935000 0x00000004
0xEC: Early Tx threshold 0x27
0xF0: Func Event 0x0040003f
0xF4: Func Event Mask 0x00000000
0xF8: Func Preset State 0x00031eff
0xFC: Func Force Event 0x00000000

lspci -d :8168 -vv
==================
pcilib: sysfs_read_vpd: read failed: Input/output error
05:00.2 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0a)
Subsystem: CLEVO/KAPOK Computer RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 19
Region 0: I/O ports at e000 [size=256]
Region 2: Memory at f0004000 (64-bit, prefetchable) [size=4K]
Region 4: Memory at f0000000 (64-bit, prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [70] Express (v2) Endpoint, MSI 01
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 10.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ AuxPwr+ TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s unlimited, L1 <64us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
Vector table: BAR=4 offset=00000000
PBA: BAR=4 offset=00000800
Capabilities: [d0] Vital Product Data
Not readable
Kernel driver in use: r8169
Kernel modules: r8169

Next message: David Woodhouse: "Re: [PATCH 0/2] xen: fix two issues in Xen pv qspinlock handling"
Previous message: Tony Lindgren: "Re: [RFC PATCH 02/11] dt-bindings: phy: add cpsw port interface mode selection phy bindings"
In reply to: Maciej S. Szmigiero: "Re: R8169: Network lockups in 4.18.{8,9,10} (and 4.19 dev)"
Next in thread: Heiner Kallweit: "Re: R8169: Network lockups in 4.18.{8,9,10} (and 4.19 dev)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]