Re: Remove __napi_schedule_irqoff?

[Grygorii Strashko]

On 18/10/2020 11:20, Heiner Kallweit wrote:
> On 18.10.2020 10:02, Eric Dumazet wrote:
> > On Sun, Oct 18, 2020 at 1:29 AM Jakub Kicinski <kuba@xxxxxxxxxx> wrote:
> > > On Sat, 17 Oct 2020 15:45:57 +0200 Heiner Kallweit wrote:
> > > > When __napi_schedule_irqoff was added with bc9ad166e38a
> > > > ("net: introduce napi_schedule_irqoff()") the commit message stated:
> > > > "Many NIC drivers can use it from their hard IRQ handler instead of
> > > > generic variant."
> > >
> > > Eric, do you think it still matters? Does it matter on x86?
> > >
> > > > It turned out that this most of the time isn't safe in certain
> > > > configurations:
> > > > - if CONFIG_PREEMPT_RT is set
> > > > - if command line parameter threadirqs is set
> > > >
> > > > Having said that drivers are being switched back to __napi_schedule(),
> > > > see e.g. patch in [0] and related discussion. I thought about a
> > > > __napi_schedule version checking dynamically whether interrupts are
> > > > disabled. But checking e.g. variable force_irqthreads also comes at
> > > > a cost, so that we may not see a benefit compared to calling
> > > > local_irq_save/local_irq_restore.
> > > >
> > > > If more or less all users have to switch back, then the question is
> > > > whether we should remove __napi_schedule_irqoff.
> > > > Instead of touching all users we could make  __napi_schedule_irqoff
> > > > an alias for __napi_schedule for now.
> > > >
> > > > [0] https://lkml.org/lkml/2020/10/8/706
> > >
> > > We're effectively calling raise_softirq_irqoff() from IRQ handlers,
> > > with force_irqthreads == true that's no longer legal.
> > >
> > > Thomas - is the expectation that IRQ handlers never assume they have
> > > IRQs disabled going forward? We don't have any performance numbers
> > > but if I'm reading Agner's tables right POPF is 18 cycles on Broadwell.
> > > Is PUSHF/POPF too cheap to bother?
> > >
> > > Otherwise a non-solution could be to make IRQ_FORCED_THREADING
> > > configurable.
> >
> > I have to say I do not understand why we want to defer to a thread the
> > hard IRQ that we use in NAPI model.
> Seems like the current forced threading comes with the big hammer and
> thread-ifies all hard irq's. To avoid this all NAPI network drivers
> would have to request the interrupt with IRQF_NO_THREAD.

I did some work in this area for TI drivers long time ago, just FYI
https://patchwork.kernel.org/project/linux-omap/patch/20160811161540.9613-1-grygorii.strashko@xxxxxx/
but, not re-checked it with more recent RT Kernels.

> > Whole point of NAPI was to keep hard irq handler very short.
> >
> > We should focus on transferring the NAPI work (potentially disrupting
> > ) to a thread context, instead of the very minor hard irq trigger.

--
Best regards,
grygorii