Re: [PATCH for 3.2.34] memcg: do not trigger OOM fromadd_to_page_cache_locked

[Michal Hocko]

On Mon 10-12-12 11:18:17, azurIt wrote:
> >Hmm, this is _really_ surprising. The latest patch didn't add any new
> >logging actually. It just enahanced messages which were already printed
> >out previously + changed few functions to be not inlined so they show up
> >in the traces. So the only explanation is that the workload has changed
> >or the patches got misapplied.
> 
> 
> This time i installed 3.2.35, maybe some changes between .34 and .35
> did this? Should i try .34?

I would try to limit changes to minimum. So the original kernel you were
using + the first patch to prevent OOM from the write path + 2 debugging
patches.
 
> >> Dec 10 02:03:29 server01 kernel: [  220.366486] grsec: From 141.105.120.152: bruteforce prevention initiated for the next 30 minutes or until service restarted, stalling each fork 30 seconds.  Please investigate the crash report for /usr/lib/apache2/mpm-itk/apache2[apache2:3586] uid/euid:1258/1258 gid/egid:100/100, parent /usr/lib/apache2/mpm-itk/apache2[apache2:2142] uid/euid:0/0 gid/egid:0/0
> >
> >This explains why you have seen your machine hung. I am not familiar
> >with grsec but stalling each fork 30s sounds really bad.
> 
> 
> Btw, i never ever saw such a message from grsecurity yet. Will write to grsec mailing list about explanation.
> 
> 
> >Anyway this will not help me much. Do you happen to still have any of
> >those logged traces from the last run?
> 
> 
> Unfortunately not, it didn't log anything and tons of messages were
> printed only to console (i was logged via IP-KVM). It looked that
> printing is infinite, i rebooted it after few minutes.

But was it at least related to the debugging from the patch or it was
rather a totally unrelated thing?

> >Apart from that. If my current understanding is correct then this is
> >related to transparent huge pages (and leaking charge to the page fault
> >handler). Do you see the same problem if you disable THP before you
> >start your workload? (echo never > /sys/kernel/mm/transparent_hugepage/enabled)
> 
> # cat /sys/kernel/mm/transparent_hugepage/enabled
> cat: /sys/kernel/mm/transparent_hugepage/enabled: No such file or directory

Weee. Then it cannot be related to THP at all. Which makes this even
bigger mystery.
We really need to find out who is leaking that charge.

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/