Re: Question on tw_timer TIMER_PINNED

[Eric Dumazet]

On Wed, Sep 6, 2023 at 1:58 PM Juri Lelli <juri.lelli@xxxxxxxxxx> wrote:
>
> Hi Eric,
>
> I'm bothering you with a question about timewait_sock tw_timer, as I
> believe you are one of the last persons touching it sometime ago. Please
> feel free to redirect if I failed to git blame it correctly.
>
> At my end, latency spikes (entering the kernel) have been reported when
> running latency sensitive applications in the field (essentially a
> polling userspace application that doesn't want any interruption at
> all). I think I've been able to track down one of such interruptions to
> the servicing of tw_timer_handler. This system isolates application CPUs
> dynamically, so what I think it happens is that at some point tw_timer
> is armed on a CPU, and it is PINNED to that CPU, meanwhile (before the
> 60s timeout) such CPU is 'isolated' and the latency sensitive app
> started on it. After 60s the timer fires and interrupts the app
> generating a spike.
>
> I'm not very familiar with this part of the kernel and from staring
> at code for a while I had mixed feeling about the need to keep tw_timer
> as TIMER_PINNED. Could you please shed some light on it? Is it a strict
> functional requirement or maybe a nice to have performance (locality I'd
> guess) improvement? Could we in principle make it !PINNED (so that it
> can be moved/queued away and prevent interruptions)?
>

It is a functional requirement in current implementation.

cfac7f836a71 ("tcp/dccp: block bh before arming time_wait timer")
changelog has some details about it.

Can this be changed to non pinned ? Probably, but with some care.

You could simply disable tw completely, it is a best effort mechanism.


> Thanks a lot in advance!
> Juri
>