Re: Question on tw_timer TIMER_PINNED

[Juri Lelli]

On 06/09/23 14:10, Eric Dumazet wrote:
> On Wed, Sep 6, 2023 at 1:58 PM Juri Lelli <juri.lelli@xxxxxxxxxx> wrote:
> >
> > Hi Eric,
> >
> > I'm bothering you with a question about timewait_sock tw_timer, as I
> > believe you are one of the last persons touching it sometime ago. Please
> > feel free to redirect if I failed to git blame it correctly.
> >
> > At my end, latency spikes (entering the kernel) have been reported when
> > running latency sensitive applications in the field (essentially a
> > polling userspace application that doesn't want any interruption at
> > all). I think I've been able to track down one of such interruptions to
> > the servicing of tw_timer_handler. This system isolates application CPUs
> > dynamically, so what I think it happens is that at some point tw_timer
> > is armed on a CPU, and it is PINNED to that CPU, meanwhile (before the
> > 60s timeout) such CPU is 'isolated' and the latency sensitive app
> > started on it. After 60s the timer fires and interrupts the app
> > generating a spike.
> >
> > I'm not very familiar with this part of the kernel and from staring
> > at code for a while I had mixed feeling about the need to keep tw_timer
> > as TIMER_PINNED. Could you please shed some light on it? Is it a strict
> > functional requirement or maybe a nice to have performance (locality I'd
> > guess) improvement? Could we in principle make it !PINNED (so that it
> > can be moved/queued away and prevent interruptions)?
> >
> 
> It is a functional requirement in current implementation.
> 
> cfac7f836a71 ("tcp/dccp: block bh before arming time_wait timer")
> changelog has some details about it.

Thanks for the reference.

> Can this be changed to non pinned ? Probably, but with some care.

I see. I will need to ponder about this.

> You could simply disable tw completely, it is a best effort mechanism.

But, first I think we are going to experiment with this route.

Thanks a lot for the super quick reply!

Best,
Juri