Re: [PATCH 3/6] dataplane nohz: run softirqs synchronously on user entry

[Chris Metcalf]

On 05/09/2015 03:04 AM, Mike Galbraith wrote:
> On Fri, 2015-05-08 at 13:58 -0400, Chris Metcalf wrote:
> > For tasks which have elected dataplane functionality, we run
> > any pending softirqs for the core before returning to userspace,
> > rather than ever scheduling ksoftirqd to run.  The problem we
> > fix is that by allowing another task to run on the core, we
> > guarantee more interrupts in the future to the dataplane task,
> > which is exactly what dataplane mode is required to prevent.
> If ksoftirqd were rt class

I realize I actually don't know if this is true or not.  Is
ksoftirqd rt class?  If not, it does seem pretty plausible that
it should be...

> softirqs would be gone when the soloist gets
> the CPU back and heads to userspace.  Being a soloist, it has no use for
> a priority, so why can't it just let ksoftirqd run if it raises the
> occasional softirq?  Meeting a contended lock while processing it will
> wreck the soloist regardless of who does that processing.

The thing you want to avoid is having two processes both
runnable at once, since then the "quiesce" mode can't make
forward progress and basically spins in cpu_idle() until ksoftirqd
can come in.  Alas, my recollection of the precise failure mode
is somewhat dimmed; my commit notes from a year ago (for
a variant of the patch I'm upstreaming now):

        - Trying to return to userspace with pending softirqs is not
          currently allowed.  Prior to this patch, when this happened
          we would just wait in cpu_idle.  Instead, what we now do is
          directly run any pending softirqs, then go back and retry the
          path where we return to userspace.
        
        - Raising softirqs (in this case for hrtimer support) could
          cause the ksoftirqd daemon to be woken on a core.  This is
          bad because on a dataplane core, a QUIESCE process will
          then block until the ksoftirqd runs, and the system sometimes
          seems to flag that soft irqs are available but not schedule
          the timer to arrange for a context switch to ksoftirqd.
          To handle this, we avoid bailing out in __do_softirq() when
          we've been working for a while, if we're on a dataplane core,
          and just keep working until done.  Similarly, on a dataplane
          core running a userspace task, we don't wake ksoftirqd when
          we are raising a softirq, even if we're not in an interrupt
          context where it will run promptly, since a non-interrupt
          context will also run promptly.

I'm happy to drop this patch entirely from the series for now, and
if ksoftirqd shows up as a problem going forward, we can address it
as necessary at that time.   What do you think?

--
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/