Re: [PATCH net] netfilter: nf_nat_masquerade: recalculate TCP TS offset when port is randomized

[Florian Westphal]

xietangxin <xietangxin@xxxxxxxxxxxxxx> wrote:
> Problem observed in Kubernetes environments where MASQUERADE target with
> --random-fully is configured by default. after commit
> 165573e41f2f ("tcp: secure_seq: add back ports to TS offset") TCP short
> connection QPS dropped from ~20000 to ~10000. This added source and
> destination ports into TS offset calculation.
> 
> However, with MASQUERADE --random-fully, when multiple internal connections
> (e.g sport 10000,20000) are mapped to the same external port (e.g 30000),
> their TS offsets are calculated as ts_offset(10000) and ts_offset(20000).
> If the server reuses the TIME_WAIT slot from the first connection, there is
> a chance that ts_offset(20000) < ts_offset(10000), breaking TSval
> monotonicity for the same 4-tuple and causing RST packets:
>   Client -> Server 24870 -> 80 [SYN] TSval=2294041168
>   Server -> Client 80 -> 24870 [ACK] TSecr=2846236456
>   Client -> Server 24870 -> 80 [RST] Seq=855605690
> 
> After nf_nat_setup_info() successfully assigns a new randomized
> source port, recalculate the TS offset using the new port and
> update the SYN packet's TSval accordingly.

I don't think this is related to masquerade but to snat (port address
rewrite) in general.

I think you could place your new helper in nf_nat_core.c and call it
from nf_nat_l4proto_unique_tuple() once we've found a usable tuple:

 668 another_round:
 669         for (i = 0; i < attempts; i++, off++) {
 670                 *keyptr = htons(min + off % range_size);
 671                 if (!nf_nat_used_tuple_harder(tuple, ct, attempts - i))

	 		     ... here.
 672                         return;
 673         }