Re: [PATCH net] tcp: introduce a compack timer handler in sack compression

[Jason Xing]

On Tue, May 30, 2023 at 6:01 PM Eric Dumazet <edumazet@xxxxxxxxxx> wrote:
>
> On Tue, May 30, 2023 at 9:45 AM Jason Xing <kerneljasonxing@xxxxxxxxx> wrote:
> >
> > On Tue, May 30, 2023 at 3:12 PM Eric Dumazet <edumazet@xxxxxxxxxx> wrote:
> > >
> > > On Mon, May 29, 2023 at 1:38 PM fuyuanli <fuyuanli@xxxxxxxxxxxxxx> wrote:
> > > >
> > > > We've got some issues when sending a compressed ack is deferred to
> > > > release phrase due to the socket owned by another user:
> > > > 1. a compressed ack would not be sent because of lack of ICSK_ACK_TIMER
> > > > flag.
> > >
> > > Are you sure ? Just add it then, your patch will be a one-liner
> > > instead of a complex one adding
> > > more code in a fast path.
> >
> > Honestly, at the very beginning, we just added one line[1] to fix
> > this. After I digged more into this part, I started to doubt if we
> > should reuse the delayed ack logic.
> >
> > Because in the sack compression logic there is no need to do more
> > things as delayed ack does in the tcp_delack_timer_handler() function.
> >
> > Besides, here are some things extra to be done if we defer to send an
> > ack in sack compression:
> > 1) decrease tp->compressed_ack. The same as "tp->compressed_ack--;" in
> > tcp_compressed_ack_kick().
> > 2) initialize icsk->icsk_ack.timeout. Actually we don't need to do
> > this because we don't modify the expiration time in the sack
> > compression hrtimer.
>
> Yes, we do not need this, see my following comment.
>
> > 3) don't need to count the LINUX_MIB_DELAYEDACKS counter.
> > 4) I wonder even if those checks about the ack schedule or ping pong
> > mode in tcp_delack_timer_handler() for sack compression? I'm not sure
> > about it.
> >
> > So one line cannot solve it perfectly. That's the reason why we
> > introduce a new logic which can be clearer.
> >
> > I'm wondering if adding one check in the fast path is really that
> > unacceptable (it may hurt performance?) because a new logic would be
> > clearer for the whole sack compression.
>
> We definitely can solve the minor issue by not polluting the fast path.
>
> We also want simple/localised fixes, not something invasive (and risky).

Now I got it :)

>
> >
> > [1]
> > diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
> > index cc072d2cfcd8..d9e76d761cc6 100644
> > --- a/net/ipv4/tcp_input.c
> > +++ b/net/ipv4/tcp_input.c
> > @@ -5568,6 +5568,7 @@ static void __tcp_ack_snd_check(struct sock *sk,
> > int ofo_possible)
> >
> > READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_comp_sack_delay_ns),
> >                       rtt * (NSEC_PER_USEC >> 3)/20);
> >         sock_hold(sk);
> > +       inet_csk(sk)->icsk_ack.pending |= ICSK_ACK_TIMER;
>
> Why not simply use existing storage/variables (tp->compressed_ack),
> instead of trying
> to reuse something else or add another bit, then complain that this
> does not work well ?
>
> Again, just fix tcp_delack_timer_handler(), it already can fetch existing state.
>
> As a bonus, no need to send one patch for net, and another in net-next,
> trying to 'fix' issues that should have been fixed cleanly in a single patch.
>
> Something like:
>
> diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c
> index b839c2f91292f7346f33d6dcbf597594473a5aca..16bc4cedceb8a5e88f61f9abc2c0a8cc9322676a
> 100644
> --- a/net/ipv4/tcp_timer.c
> +++ b/net/ipv4/tcp_timer.c
> @@ -290,10 +290,20 @@ static int tcp_write_timeout(struct sock *sk)
>  void tcp_delack_timer_handler(struct sock *sk)
>  {
>         struct inet_connection_sock *icsk = inet_csk(sk);
> +       struct tcp_sock *tp = tcp_sk(sk);
>
> -       if (((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN)) ||
> -           !(icsk->icsk_ack.pending & ICSK_ACK_TIMER))
> +       if ((1 << sk->sk_state) & (TCPF_CLOSE | TCPF_LISTEN))
> +               return;
> +
> +       if (!(icsk->icsk_ack.pending & ICSK_ACK_TIMER) &&
> +           !tp->compressed_ack)
> +               return;
> +
> +       if (tp->compressed_ack) {
> +               tcp_mstamp_refresh(tp);
> +               tcp_sack_compress_send_ack(sk);

I wonder if we could use this combination as below instead since the
above function counts the snmp counter and clears the @compressed_ack,
which is against what we normally do in tcp_compressed_ack_kick() if
the socket is not owned?

"
if (hrtimer_try_to_cancel(&tp->compressed_ack_timer) == 1)
    __sock_put(sk);
tcp_send_ack(sk);
"

Above is extracted from tcp_sack_compress_send_ack()

>                 return;
> +       }
>
>         if (time_after(icsk->icsk_ack.timeout, jiffies)) {
>                 sk_reset_timer(sk, &icsk->icsk_delack_timer,
> icsk->icsk_ack.timeout);
> @@ -312,7 +322,7 @@ void tcp_delack_timer_handler(struct sock *sk)
>                         inet_csk_exit_pingpong_mode(sk);
>                         icsk->icsk_ack.ato      = TCP_ATO_MIN;
>                 }
> -               tcp_mstamp_refresh(tcp_sk(sk));
> +               tcp_mstamp_refresh(tp);
>                 tcp_send_ack(sk);
>                 __NET_INC_STATS(sock_net(sk), LINUX_MIB_DELAYEDACKS);
>         }

Thank you so much, Eric.

I will let fuyuanli submit a v2 patch including that one without
introducing a new logic/flag.

Thanks again,
Jason