Re: [PATCH] nohz: prevent tilegx network driver interrupts

[Chris Metcalf]

On 07/10/2015 02:24 PM, Frederic Weisbecker wrote:
> On Fri, Jul 10, 2015 at 01:33:44PM -0400, Chris Metcalf wrote:
> > In nohz_full mode, by default distribute networking shim
> > interrupts across the housekeeping cores, not all the cores.
> I can't really tell, I have no idea what this driver does. It seems
> to be about networking CPUs but I have no idea what we are affining
> here. Whether it is task, interrupt, ... And what those affine things do,
> if it is safe to do that reduce affinity etc..
>
> I looked at the driver but I can't make my way there. I think you
> need a more detailed changelog :-)

Fair enough! :-)  See updated changelog to follow.

> > Signed-off-by: Chris Metcalf <cmetcalf@xxxxxxxxxx>
> > ---
> > The alternate approaches to this might be:
> >
> > 1. "#define housekeeping_mask cpu_online_mask" in the non-nohz_full
> >     arm in <linux/tick.h>, then just unconditionally use
> >     "housekeeping_mask".
> Indeed we are doing more and more references on housekeeping_mask, so
> we should probably think about an off-case.
>
> Now the nohz-full off-case should rather be cpu_possible_mask than
> cpu_online_mask. housekeeping_mask doesn't take into account onlining
> at all.

That suggests that in this case, we might want to default to
something like "housekeeping_mask & cpu_online_mask",
since you really don't want to send irqs to offline cores to
process your packets :-)

The tilegx chips typically don't do cpu offlining anyway, since
we've never really found a usecase, so whatever you boot with
you always have available.  We do have support for a bare-metal
mode which you can run on some of the cores, so you may start
with fewer than cpu_possible actually running, but it will always
be that same set of cores.

So this does suggest that my original patch is wrong for that
same reason.

> > 2. Provide an accessor that returns the cpumask to use for housekeeping
> >     chores and implement it in the obvious ways for both nohz_full
> >     and non-nohz_full.
> >
> > The latter seems like arguably the most satisfying approach, but
> > the patch below is, if nothing else, suitable to push for 4.3
> > without any further API development work.
> I don't know. 1) looks easier.

On reflection, the problem with (1) is that if you are in NO_HZ_FULL
mode but !tick_nohz_full_enabled(), you want to fall back to just
using cpu_possible_mask anyway.  So I think a simple accessor that
returns an appropriate cpumask pointer is probably the best bet
(along the lines of the existing is_housekeeping_cpu() accessor).

--
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/