[Networking] ERSPAN decapsulation drops DHCP unicast packets

[Volodymyr Litovka]

Hi colleagues,

I'm trying to catch and process (in 3rd party analytics app) DHCP 
packets from ERSPAN session, but cannot do this due to absence of DHCP 
unicast packets after decapsulation.

The model is pretty simple: there is PHY interface (enp2s0) which 
receive ERSPAN traffic and erspan-type interface to get decapsulated 
packets (inspan, created using command "ip link add inspan type erspan 
seq key 10 local 10.171.165.65 erspan_ver 1", where 10.171.165.65 is 
ERSPAN target). Then I'm going to rewrite headers in the proper ways 
(nftable's netdev family) and forward packets to the pool of workers.

Having this, I'm expecting everything, which is encapsulated inside 
ERSPAN, on 'inspan' interface. And there is _almost_ everything except 
DHCP unicast packets - tcpdump shows about 1kps on this interface of 
decapsulated packets, but no DHCP unicast (see below traces).

To avoid any interactions, I removed and disabled everything that can 
catch DHCP in userspace - systemd-networkd, netplan, dhcp-client. There 
is no DHCP server and ifupdown - for test purposes, I'm bringing 
networking manually. Apparmor disabled as well. Kernel (Linux 
5.19.0-42-generic #43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC) compiled 
without CONFIG_IP_PNP (according to /boot/config-5.19.0-42-generic). 
Nothing in userspace listens on UDP/68 and UDP/67:

# netstat -tunlpa
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         
State       PID/Program name
tcp        0      0 0.0.0.0:22 0.0.0.0:*               LISTEN      
544/sshd: /usr/sbin
tcp6       0      0 :::22 :::*                    LISTEN      544/sshd: 
/usr/sbin

I have no ideas, why this is happening. Decapsulation itself works, but 
particular kind of packets get lost.

I will appreciate if anyone can help me understand where is the bug - in 
my configuration or somewhere inside the kernel?

Evidence of traffic presence/absence is below.

Thank you.

Encapsulated ERSPAN session (udp and port 67/68) contains lot of 
different kinds of DHCP packets:

# tcpdump -s0 -w- -i enp2s0 'proto gre and ether[73:1]=17 and 
(ether[84:2]=67 or ether[84:2]=68)' | tshark -r- -l
 [ ... ]
    7   0.001942  0.0.0.0 → 255.255.255.255 DHCP 392 DHCP Discover - 
Transaction ID 0x25c096fc
    8   0.003432  z.z.z.z → a.a.a.a         DHCP 418 DHCP ACK      - 
Transaction ID 0x5515126a
    9   0.005170  m.m.m.m → z.z.z.z         DHCP 435 DHCP Discover - 
Transaction ID 0xa7b7
   10   0.005171  m.m.m.m → z.z.z.z         DHCP 435 DHCP Discover - 
Transaction ID 0xa7b7
   11   0.015399  n.n.n.n → z.z.z.z         DHCP 690 DHCP Request  - 
Transaction ID 0x54955233
   12   0.025537  z.z.z.z → n.n.n.n         DHCP 420 DHCP ACK      - 
Transaction ID 0x54955233
   13   0.030313  z.z.z.z → m.m.m.m         DHCP 413 DHCP Offer    - 
Transaction ID 0xa7b7

but decapsulated traffic (which I'm seeing on inspan interface) contains 
just the following:

# tcpdump -i inspan 'port 67 or port 68'
listening on inspan, link-type EN10MB (Ethernet), snapshot length 262144 
bytes
17:23:36.540721 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, 
Request from 00:1a:64:33:8d:fa (oui Unknown), length 300
17:23:39.760036 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, 
Request from 00:1a:64:33:8d:fa (oui Unknown), length 300
17:23:44.135711 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, 
Request from 00:1a:64:33:8d:fa (oui Unknown), length 300
17:23:52.008504 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, 
Request from 00:1a:64:33:8d:fa (oui Unknown), length 300

Thanks again for the help.


--
Volodymyr Litovka
  "Vision without Execution is Hallucination." -- Thomas Edison