ESP failure when PTB is received as non ESP packet and payload as ESP

[Bhat, Jayalakshmi Manjunath]

Hi All,

There are 2 difference scenarios our device receives PTB.

In the first scenario PTB is received as ESP packet. First ESP gets decrypted and has payload of ICMP packet as PTB. PTB is 1280.
PMTU is set on the device via icmp6_err function in kernel.  

This is followed by device receiving fragmented PING requests.  As part of ESP and XFRM processing esp6_get_mtu gets called.
And esp6_get_mtu calculates MTU as MTU-ESP HEADER-IPHEADER etc. Here MTU remains as 1500 and calculated MTU is also fine.
Thus the execution works good.  


In the second scenario PTB is received as non ESP packet. However payload in this packet is an ESP. PTB is 1280.  PMTU is set via 
esp6_err function in kernel. This function also invokes xfrm_policy_destroy. 

This is followed by device receiving fragmented PING requests. Device creates new xfrm_state and sets the MTU as 1280.
esp6_get_mtu calculates MTU as 1280- ESP HEADER-IPHEADER etc which result MTU < IP6_MIN_MTU and test fails.

Has anyone encountered scenario 2. Any inputs are appreciated.

Thanks and Regards,
Jayalakshmi