Re: [GIT PULL] Introduce housekeeping subsystem v4

[Frederic Weisbecker]

2017-09-29 15:34 GMT+02:00 Frederic Weisbecker <fweisbec@xxxxxxxxx>:
> On 28/09/2017 11:54, Ingo Molnar wrote:
>>
>>
>> * Frederic Weisbecker <fweisbec@xxxxxxxxx> wrote:
>>
>>> Ingo,
>>>
>>> Please pull the core/isolation-v4 branch that can be found at:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
>>>         core/isolation-v4
>>>
>>> HEAD: cf4c55aad44251369c8507c3823f9f9c51d4dc77
>>>
>>> Summary of changes:
>>>
>>> * Move the housekeeping code that was tied to NO_HZ to its own subsystem.
>>>    Currently NO_HZ governs the other isolation features which is not
>>> right
>>>    as dynticks is just an isolation feature like the others. We want to
>>>    centralize the CPU isolation decisions to a subsystem of its own
>>> instead.
>>>
>>> * Integrate isolcpus code to housekeeping and treat it as a CPU isolation
>>>    feature.
>>>
>>> * Reuse the "isolcpus=" kernel parameter to control the CPU isolation.
>>>    For now only tick and domains can be isolated after this patchset:
>>>
>>>         isolcpus=1-7         # isolate domains on CPU range 1 to 7
>>>                              # "domain" flag is implicit by default to
>>>                              # keep the current behaviour
>>>                                      isolcpus=domain,1-7  # do the same
>>>
>>>         isolcpus=nohz,1-7    # apply nohz_full to CPU range 1 to 7
>>>
>>>         isolcpus=nohz,domain,1-7  # apply nohz_full and isolate domains
>>> of
>>>                                   # CPU range 1 to 7
>>>
>>> Thanks,
>>>         Frederic
>>> ---
>>>
>>> Frederic Weisbecker (12):
>>>        housekeeping: Move housekeeping related code to its own file
>>>        watchdog: Use housekeeping_cpumask() instead of ad-hoc version
>>>        housekeeping: Provide a dynamic off-case to housekeeping_any_cpu()
>>>        housekeeping: Make housekeeping cpumask private
>>>        housekeeping: Use its own static key
>>>        housekeeping: Rename is_housekeeping_cpu to housekeeping_cpu
>>>        housekeeping: Move it under its own config, independant from NO_HZ
>>>        housekeeping: Introduce housekeeping flags
>>>        housekeeping: Handle nohz_full= parameter
>>>        housekeeping: Move isolcpus to housekeeping
>>>        housekeeping: Add basic isolcpus flags
>>>        housekeeping: Document isolcpus flags
>>>
>>>
>>>   Documentation/admin-guide/kernel-parameters.txt |  33 +++---
>>>   drivers/base/cpu.c                              |  11 +-
>>>   drivers/net/ethernet/tile/tilegx.c              |   6 +-
>>>   include/linux/housekeeping.h                    |  51 ++++++++
>>>   include/linux/sched.h                           |   2 -
>>>   include/linux/tick.h                            |  39 +------
>>>   init/Kconfig                                    |   7 ++
>>>   init/main.c                                     |   2 +
>>>   kernel/Makefile                                 |   1 +
>>>   kernel/cgroup/cpuset.c                          |  15 +--
>>>   kernel/housekeeping.c                           | 149
>>> ++++++++++++++++++++++++
>>>   kernel/rcu/tree_plugin.h                        |   3 +-
>>>   kernel/rcu/update.c                             |   3 +-
>>>   kernel/sched/core.c                             |  25 +---
>>>   kernel/sched/fair.c                             |   3 +-
>>>   kernel/sched/topology.c                         |  24 +---
>>>   kernel/time/tick-sched.c                        |  31 +----
>>>   kernel/watchdog.c                               |  13 +--
>>>   18 files changed, 276 insertions(+), 142 deletions(-)
>>
>>
>> Yeah, so while I agree that all this functionality needs to be factored
>> out and organized, I have a problem with this specific high level
>> organization.
>>
>> The main problem I think is that it's all called "housekeeping", which is
>> pretty
>> fuzzy and opaque. What does 'housekeeping' exactly mean? A dozen of
>> details if you
>> look at the code - and this name does not make it much easier to think
>> about this
>> whole problem category.
>
>
> Indeed I feel that housekeeping is probably not the best concept to express
> all these things. I'm all for something clearer.
>
>>
>> So how about introducing _two_ new high level concepts:
>>
>>    1) 'global time handling'
>>    2) 'double async CPU callbacks'
>>
>> The notion of 'global time' handling is obvious to everyone I think: it
>> involves
>> the system-global guarantee that certain kernel jobs will be executed
>> periodically. At least one CPU in the system needs to handle 'global
>> time'.
>>
>> The notion of 'double async CPU callbacks' is less obvious: it involves
>> the action
>> of invoking a callback on a CPU, that might be executed on _another_ CPU.
>>
>> I.e. there are 3 CPUs involved:
>>
>>         - the invoking CPU
>>         - the target CPU
>>         - the CPU(s!) that will handle the callback (the housekeeping CPU
>> mask)
>>
>> For example the kmem-cache on_each_cpu() calls in mm/slab.c would fall
>> into this
>> category.
>
>
> Hmm, I'm not clear on this one. Do you mean works that can be executed
> concurrently?
>
>>
>> I don't know to what extent it makes sense to formalize and unify these
>> facilities: it's certain that the (former) housekeeping CPU mask should be
>> shared
>> by these two facilities: the CPU executing global time callbacks
>> periodically
>> should be one of the CPUs that execute double-async CPU callbacks.
>>
>> But by separating all this functionality into these two categories, it's
>> already
>> much easier to me to argue about which bit does what and why.
>
>
> Note that some housekeeping concepts may not fall into any of these
> categories. For example domain isolation.
>
> Thanks.

Hi Ingo,

Any detail to add about this housekeeping division?

Thanks!