Re: [RFC PATCH v3 1/7] ktask: add documentation
From: Daniel Jordan
Date: Fri Dec 08 2017 - 08:46:54 EST
On 12/08/2017 07:43 AM, Michal Hocko wrote:
On Wed 06-12-17 15:32:48, Daniel Jordan wrote:
On 12/06/2017 09:35 AM, Michal Hocko wrote:
[...]
There is also no mention about other
characteristics (e.g. power management), resource isloataion etc. So > let me ask again. How do you control that the parallelized operation
doesn't run outside of the limit imposed to the calling context?
The current code doesn't do this, and the answer is the same for the rest of
your questions.
I really believe this should be addressed before this can be considered
for merging. While what you have might be sufficient for early boot
initialization stuff I am not sure the amount of code is really
justified by that usecase alone. Any runtime enabled parallelized work
really have to care about the rest of the system. The last thing you
really want to see is to make a highly utilized system overloaded just
because of some optimization. And I do not see how can you achive that
with a limit on the number of paralelization threads.
That's fair, I'll see what I can do in the next version.
For resource isolation, I'll experiment with moving ktask threads into and
out of the cgroup of the calling thread.
Do any resources not covered by cgroup come to mind? I'm trying to think if
I've left anything out.
This is mostly about cpu so dealing with the cpu cgroup controller
should do the work.
Ok, thanks. Luckily cgroup v2's cpu controller was recently merged.
[...]
Anyway, I think scalability bottlenecks should be weighed with the rest of
this. It seems wrong that the kernel should always assume that one thread
is enough to free all of a process's memory or evict all the pages of a file
system no matter how much work there is to do.
Well, this will be always a double edge sword. Sure if you have spare
cycles (whatever that means) than using them is really nice. But the
last thing you really want is to turn an optimization into an
utilization nightmare where few processes dominant the whole machine
even though they could be easily contained normally inside a single
execution context. >
Your work targets larger machines and I understand that you are mainly
focused on a single large workload running on that machine but there are
many others running with many smaller workloads which would like to be
independent. Not everything is a large DB running on a large HW.
Well of course, yes, but the struct page initialization stuff benefits
any large-memory machine (9x faster on a 2-socket machine!) and the
(forthcoming) page freeing parallelization will similarly benefit a
variety of workloads.
Anyway, I'll put more controls in and see where I get. Thanks for the
feedback.
Daniel