Re: [PATCH 0/6] support "dataplane" mode for nohz_full

[Chris Metcalf]

Thanks for the clarification, and sorry for the slow reply; I had a busy
week of meetings last week, and then the long weekend in the U.S.

On 05/15/2015 02:44 PM, Mike Galbraith wrote:
> Just because the nohz_full feature itself is currently static is no
> reason to put users thereof in a straight jacket by mandating that any
> set they define irrevocably disappears from the generic resource pool .
> Those CPUS are useful until the moment someone cripples them, which
> making nohz_full imply isolcpus does if isolcpus then also becomes
> immutable, which Rik's patch does.  Making nohz_full imply isolcpus
> sounds perfectly fine until someone comes along and makes isolcpus
> immutable (Rik's patch), at which point the user loses a choice due to
> two people making it imply things that _alone_ sound perfectly fine.
>
> See what I'm saying now?

That does make sense; my argument was that 99% of the time when
someone specifies nohz_full they also need isolcpus.  You're right
that someone playing with nohz_full would be unpleasantly surprised.
And of course having more flexibility always feels like a plus.
On balance I suspect it's still better to make command line arguments
handle the common cases most succinctly.

Hopefully we'll get a to a point where all of this is dynamic and how
we play with the boot arguments no longer matters.  If not, perhaps
we revisit this and make a cpu_isolation=1-15 type command line
argument that enables isolcpus and nohz_full both.

> > > Thomas has nuked the hrtimer softirq.
> > Yes, this I didn't know.  So I will drop my "no ksoftirqd" patch and
> > we will see if ksoftirqs emerge as an issue for my "cpu isolation"
> > stuff in the future; it may be that that was the only issue.
> >
> > > Inlining softirqs may save a context switch, but adds cycles that we may
> > > consume at higher frequency than the thing we're avoiding.
> > Yes but consuming cycles is not nearly as much of a concern
> > as avoiding interrupts or scheduling, certainly for the case of
> > userspace drivers that I described above.
> If you're raising softirqs in an SMP kernel, you're also doing something
> that puts you at very serious risk of meeting the jitter monster, locks,
> and worse, sleeping locks, no?

The softirqs were being raised by third parties for hrtimer, not by
the application code itself, if I remember correctly.  In any case
this appears not to be an issue for nohz_full any more now.

--
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/