Re: [PATCH] mm: add config option to select the initial overcommit mode

[Michal Hocko]

On Fri 13-05-16 14:39:01, Sebastian Frias wrote:
> Hi Michal,
> 
> On 05/13/2016 02:00 PM, Michal Hocko wrote:
> > On Fri 13-05-16 11:52:30, Sebastian Frias wrote:
[...]
> >> Indeed, I was hoping we could throw some light into that.
> >> My patch had another note:
> > 
> > I cannot really tell because this was way before my time but I guess the
> > reason was that userspace is usually very address space hungry while the
> > actual memory consumption is not that bad. See my other email.
> 
> Yes, I saw that, thanks for the example.
> It's just that it feels like the default value is there to deal with
> (what it should be?) very specific cases, right?

The default should cover the most use cases. If you can prove that the
vast majority of embeded systems are different and would _benefit_ from
a different default I wouldn't be opposed to change the default there.

> >> It'd be nice to know more about why was overcommit introduced.
> >> Furthermore, it looks like allowing overcommit and the introduction
> >> of the OOM-killer has given rise to lots of other options to try to
> >> tame the OOM-killer.
> >> Without context, that may seem like a form of "feature creep" around it.
> >> Moreover, it makes Linux behave differently from let's say Solaris.
> >>
> >>    https://www.win.tue.nl/~aeb/linux/lk/lk-9.html#ss9.6
> > 
> > Well, those are some really strong statements which do not really
> > reflect the reality of the linux userspace. I am not going to argue with
> > those points because it doesn't make much sense. Yes in an ideal world
> > everybody consumes only so much he needs. Well the real life is a bit
> > different...
> 
> :-)
> I see, so basically it is a sort of workaround.

No it is not a workaround. It is just serving the purpose of the
operating system. The allow using the HW as much as possible to the
existing userspace. You cannot expect userspace will change just because
we do not like the overcommiting the memory with all the fallouts.

> Anyway, in the embedded world the memory and system requirements are
> usually controlled.

OK, but even when it is controlled does it suffer in any way just
because of the default setting? Do you see OOM killer invocation
when the overcommit would prevent from that?

> Would you agree to the option if it was dependent on
> CONFIG_EMBEDDED? Or if it was a hidden option?
> (I understand though that it wouldn't affect the size of config space)

It could be done in the code and make the default depending on the
existing config. But first try to think about what would be an advantage
of such a change.
 
> >> Hopefully this discussion could clear some of this up and maybe result
> >> in more documentation around this subject.
> > 
> > What kind of documentation would help?
> 
> Well, mostly the history of this setting, why it was introduced, etc.
> more or less what we are discussing here.  Because honestly, killing
> random processes does not seems like a straightforward idea, ie: it is
> not obvious.  Like I was saying, without context, such behaviour looks
> a bit crazy.

But we are not killing a random process. The semantic is quite clear. We
are trying to kill the biggest memory hog and if it has some children
try to sacrifice them to save as much work as possible.

> >> From what I remember, one of the LTP maintainers said that it is
> >> highly unlikely people test (or run LTP for that matter) with
> >> different settings for overcommit.
> > 
> > Yes this is sad and the result of a excessive configuration space.
> > That's why I was pushing back to adding yet another one without having
> > really good reasons...
> 
> Well, a more urgent problem would be that in that case
> overcommit=never is not really well tested.

This is a problem of the userspace and am really skeptical that a change
in default would make any existing bugs going away. It is more likely we
will see reports that ENOMEM has been returned even though there is
pletny of memory available.

[...]

> > Killing random tasks is definitely a misbehavior and it happened a lot
> > in the past when heuristics were based on multiple metrics (including
> > the run time etc.). Things have changed considerably since then and
> > seeing random tasks being selected shouldn't happen all that often and
> > if it happens it should be reported, understood and fixed.
> > 
> 
> Well, it's hard to report, since it is essentially the result of a
> dynamic system.

Each oom killer invocation will provide a detailed report which will
help MM developers to debug what went wrong and why.

-- 
Michal Hocko
SUSE Labs