Re: [QUERY]: Is using CPU hotplug right for isolating CPUs?

[Viresh Kumar]

On 23 January 2014 20:28, Frederic Weisbecker <fweisbec@xxxxxxxxx> wrote:
> On Tue, Jan 21, 2014 at 04:03:53PM +0530, Viresh Kumar wrote:

>> So, the main problem in my case was caused by this:
>>
>>            <...>-2147  [001] d..2   302.573881: hrtimer_start:
>> hrtimer=c172aa50 function=tick_sched_timer expires=602075000000
>> softexpires=602075000000
>>
>> I have mentioned this earlier when I sent you attachments. I think
>> this is somehow
>> tied with the NO_HZ_FULL stuff? As the timer is queued for 300 seconds after
>> current time.
>>
>> How to get this out?
>
> So it's scheduled away 300 seconds later. It might be a pending timer_list. Enabling the
> timer tracepoints may give you some clues.

Trace was done with that enabled. /proc/timer_list confirms that a hrtimer
is queued for 300 seconds later for tick_sched_timer. And so I assumed
this is part of the current NO_HZ_FULL implementation.

Just to confirm, when we decide that a CPU is running a single task and so
can enter tickless mode, do we queue this tick_sched_timer for 300 seconds
ahead of time? If not, then who is doing this :)

>> Which CPUs are housekeeping CPUs? How do we declare them?
>
> It's not yet implemented, but it's an idea (partly from Thomas) of something we can do to
> define some general policy on various periodic/async work affinity to enforce isolation.
>
> The basic idea is to define the CPU handling the timekeeping duty to be the housekeeping
> CPU. Given that CPU must keep a periodic tick, lets move all the unbound timers and
> workqueues there. And also try to move some CPU affine work as well. For example
> we could handle the scheduler tick of the full dynticks CPUs into that housekeeping
> CPU, at a low freqency. This way we could remove that 1 second scheduler tick max deferment
> per CPU. It may be an overkill though to run all the scheduler ticks on a single CPU so there
> may be other ways to cope with that.
>
> And I would like to keep that housekeeping notion flexible enough to be extendable on more
> than one CPU, as I heard that some people plan to reserve one CPU per node on big
> NUMA machines for such a purpose. So that could be a cpumask, augmented with an infrastructure.
>
> Of course, if some people help contributing in this area, some things may eventually move foward
> on the support of CPU isolation. I can't do that all alone, at least not quickly, given all the
> things already pending in my queue (fix buggy nohz iowait accounting, support RCU full sysidle detection,
> apply AMD range breakpoints patches, further cleanup posix cpu timers, etc...).

I see. As I am currently working on the isolation stuff which is very
much required
for my usecase, I will try to do that as the second step of my work.
The first one
stays something like a cpuset.quiesce option that PeterZ suggested.

Any pointers of earlier discussion on this topic would be helpful to
start working on
this..
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/