Response error to fragmented ICMP echo request

From: Song, Xiongwei
Date: Tue Mar 07 2023 - 07:37:35 EST

Next message: Miguel Ojeda: "Re: [PATCH] scripts: rust-analyzer: Skip crate module directories"
Previous message: Pavel Machek: "Motorola Droid 4 -- Stopping charger when battery is full"
Next in thread: Vladimir Oltean: "Re: Response error to fragmented ICMP echo request"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Hi Experts,

We are using a NXP LS1028A board and facing an issue on icmp:

1. Scenario
Our test sends 2 fragments from VM to our board and expects echo reply but
with no response is present. send a fragment first, 2nd fragment is sent
with delay and test expects "ICMP ip reassembly time exceeded" but there
is also no response.

Detailed description of the testcase:

* Action Send ICMP Echo request packet VM -> device. The packet contains: IP Source Address
* field set to address of VM, IP Destination Address field set to address of the device
* Identification field set to 1, Fragmentation offset field set to 0, MF flags
* set
* Action Wait 100ms
*
* Action Send ICMP Echo request packet VM -> device. The packet contains: IP Source Address
* field set to address of VM, IP Destination Address field set to address of the device
* Identification field set to 1, Fragmentation offset field set to 1, MF flags
* cleared
*
* Result The ICMP echo response is received

2. We are using scapy to do test, the following is scapy packet configuration

f1=Ether(src="26:84:d5:7f:7d:be", dst="7C:72:6E:D4:44:C1")/Dot1Q(prio=0, vlan=984)/IP(src="10.225.32.20", dst="10.225.32.21")/'\x08\x00\xF7\xFF\x00\x00\x00\x00\x00'
f1.flags='MF'
f1.frag=0
f1.id=1
f1.proto='icmp'
// result of f1.show()
###[ Ethernet ]###
dst = 7C:72:6E:D4:44:C1
src = 26:84:d5:7f:7d:be
type = 0x8100
###[ 802.1Q ]###
prio = 0
id = 1
vlan = 984
type = 0x800
###[ IP ]###
version = 4
ihl = None
tos = 0x0
len = None
id = 1
flags = MF
frag = 0
ttl = 64
proto = icmp
chksum = None
src = 10.225.32.20
dst = 10.225.32.21
\options \
###[ Raw ]###
load = '\x08\x00\xf7\xff\x00\x00\x00\x00\x00'

f2=Ether(src="26:84:d5:7f:7d:be", dst="7C:72:6E:D4:44:C1")/Dot1Q(prio=0, vlan=984)/IP(src="10.225.32.20", dst="10.225.32.21")/'\x00\x00\x00\x00\x00\x00\x00\x00'
f2.frag=1
f2.id=1
f2.proto='icmp'
// result of f2.show()
###[ Ethernet ]###
dst = 7C:72:6E:D4:44:C1
src = 26:84:d5:7f:7d:be
type = 0x8100
###[ 802.1Q ]###
prio = 0
id = 1
vlan = 984
type = 0x800
###[ IP ]###
version = 4
ihl = None
tos = 0x0
len = None
id = 1
flags =
frag = 1
ttl = 64
proto = icmp
chksum = None
src = 10.225.32.20
dst = 10.225.32.21
\options \
###[ Raw ]###
load = '\x00\x00\x00\x00\x00\x00\x00\x00'

3. Logs for this issue:

1). From ethtool
We use ethtool to check the hw status, found packet was dropped which pointed out
By drop_yellow_prio_0 field :

......snip......
failing SW:
rx_octets +64
rx_unicast +1
rx_frames_below_65_octets +1
rx_yellow_prio_0 +1
*drop_yellow_prio_0 +1
......snip......

2). From netstat

The outputs of netstat for failing and working cases:
// failing logs

///before test
root@xxx:~# netstat -s
Ip:
Forwarding: 1
3 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
3 requests sent out
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
3 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 252
OutOctets: 252
InNoECTPkts: 3

///after test
root@xxx:~# netstat -s
Ip:
Forwarding: 1
3 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
3 requests sent out
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
3 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 252
OutOctets: 252
InNoECTPkts: 3

// working case

///before test

root@xxx:~#netstat -s
Ip:
Forwarding: 1
3 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
3 requests sent out
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
3 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 252
OutOctets: 252
InNoECTPkts: 3

///after test

root@xxx:~#netstat -s
Ip:
Forwarding: 1
4 total packets received
0 forwarded
0 incoming packets discarded
3 incoming packets delivered
4 requests sent out
******difference******
1 fragments dropped after timeout
1 reassemblies required
1 packet reassemblies failed
Icmp:
3 ICMP messages received
0 input ICMP message failed
ICMP input histogram:
echo requests: 3
4 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
time exceeded: 1
echo replies: 3
IcmpMsg:
InType8: 3
OutType0: 3
OutType11: 1
******difference******
Tcp:
0 active connection openings
0 passive connection openings
0 failed connection attempts
0 connection resets received
0 connections established
0 segments received
0 segments sent out
0 segments retransmitted
0 bad segments received
0 resets sent
Udp:
0 packets received
0 packets to unknown port received
0 packet receive errors
0 packets sent
0 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
0 packet headers predicted
IpExt:
InOctets: 294
OutOctets: 308
InNoECTPkts: 4

3). From pcap file(the pcap was collected on the senderside (VM))

Frame 1: 64 bytes on wire (512 bits), 64 bytes captured (512 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:42.803640000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179802.803640000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 64 bytes (512 bits)
Capture Length: 64 bytes (512 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:arp]
Ethernet II, Src: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f), Dst: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Destination: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 6, DEI: 0, ID: 981
110. .... .... .... = Priority: Internetwork Control (6)
...0 .... .... .... = DEI: Ineligible
.... 0011 1101 0101 = ID: 981
Type: ARP (0x0806)
Padding: 0000000000000000000000000000
Trailer: 00000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Sender IP address: 10.225.32.5
Target MAC address: 00:00:00_00:00:00 (00:00:00:00:00:00)
Target IP address: 10.225.32.4

Frame 2: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:42.803664000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179802.803664000 seconds
[Time delta from previous captured frame: 0.000024000 seconds]
[Time delta from previous displayed frame: 0.000024000 seconds]
[Time since reference or first frame: 0.000024000 seconds]
Frame Number: 2
Frame Length: 46 bytes (368 bits)
Capture Length: 46 bytes (368 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:arp]
Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Destination: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 981
000. .... .... .... = Priority: Best Effort (default) (0)
...0 .... .... .... = DEI: Ineligible
.... 0011 1101 0101 = ID: 981
Type: ARP (0x0806)
Address Resolution Protocol (reply)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: reply (2)
Sender MAC address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Sender IP address: 10.225.32.4
Target MAC address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Target IP address: 10.225.32.5

Frame 3: 47 bytes on wire (376 bits), 47 bytes captured (376 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:49.915062000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179809.915062000 seconds
[Time delta from previous captured frame: 7.111398000 seconds]
[Time delta from previous displayed frame: 7.111398000 seconds]
[Time since reference or first frame: 7.111422000 seconds]
Frame Number: 3
Frame Length: 47 bytes (376 bits)
Capture Length: 47 bytes (376 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:ip:data]
Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Destination: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
000. .... .... .... = Priority: Best Effort (default) (0)
...1 .... .... .... = DEI: Eligible
.... 0011 1101 0101 = ID: 981
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.225.32.4, Dst: 10.225.32.5
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 29
Identification: 0x0001 (1)
Flags: 0x2000, More fragments
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..1. .... .... .... = More fragments: Set
Fragment offset: 0
Time to live: 64
Protocol: ICMP (1)
Header checksum: 0x0515 [validation disabled]
[Header checksum status: Unverified]
Source: 10.225.32.4
Destination: 10.225.32.5
Data (9 bytes)

0000 08 00 f7 ff 00 00 00 00 00 .........
Data: 0800f7ff0000000000
[Length: 9]

Frame 4: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Dec 16, 2022 16:36:50.063589000 CST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1671179810.063589000 seconds
[Time delta from previous captured frame: 0.148527000 seconds]
[Time delta from previous displayed frame: 0.148527000 seconds]
[Time since reference or first frame: 7.259949000 seconds]
Frame Number: 4
Frame Length: 46 bytes (368 bits)
Capture Length: 46 bytes (368 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:vlan:ethertype:ip:icmp:data]
Ethernet II, Src: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c), Dst: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Destination: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
Address: 7c:72:6e:d4:44:5f (7c:72:6e:d4:44:5f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
Address: aa:3a:b3:e7:67:5c (aa:3a:b3:e7:67:5c)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: 802.1Q Virtual LAN (0x8100)
802.1Q Virtual LAN, PRI: 0, DEI: 1, ID: 981
000. .... .... .... = Priority: Best Effort (default) (0)
...1 .... .... .... = DEI: Eligible
.... 0011 1101 0101 = ID: 981
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.225.32.4, Dst: 10.225.32.5
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 28
Identification: 0x0001 (1)
Flags: 0x0001
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 8
Time to live: 64
Protocol: ICMP (1)
Header checksum: 0x2515 [validation disabled]
[Header checksum status: Unverified]
Source: 10.225.32.4
Destination: 10.225.32.5
[2 IPv4 Fragments (16 bytes): #3(9), #4(8)]
[Frame: 3, payload: 0-8 (9 bytes)]
[Frame: 4, payload: 8-15 (8 bytes)]
[Fragment overlap: True]
[Fragment count: 2]
[Reassembled IPv4 length: 16]
[Reassembled IPv4 data: 0800f7ff000000000000000000000000]
Internet Control Message Protocol
Type: 8 (Echo (ping) request)
Code: 0
Checksum: 0xf7ff [correct]
[Checksum Status: Good]
Identifier (BE): 0 (0x0000)
Identifier (LE): 0 (0x0000)
Sequence number (BE): 0 (0x0000)
Sequence number (LE): 0 (0x0000)
Data (8 bytes)

0000 00 00 00 00 00 00 00 00 ........
Data: 0000000000000000
[Length: 8]

4). What we've found so far

According binary search, we found out the following commit causes this issue:
a4ae997adcbd("net: mscc: ocelot: initialize watermarks to sane defaults").
Without this commit the test case was passed.

Could you please take a look? Please let me know if you need more debug info.

Regards,
Xiongwei

Next message: Miguel Ojeda: "Re: [PATCH] scripts: rust-analyzer: Skip crate module directories"
Previous message: Pavel Machek: "Motorola Droid 4 -- Stopping charger when battery is full"
Next in thread: Vladimir Oltean: "Re: Response error to fragmented ICMP echo request"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]