Re: [question] how to figure out OOM reason? should dump slab/vmallocinfo when OOM?

[David Rientjes]

On Mon, 20 Jan 2014, Jianguo Wu wrote:

> When OOM happen, will dump buddy free areas info, hugetlb pages info,
> memory state of all eligible tasks, per-cpu memory info.
> But do not dump slab/vmalloc info, sometime, it's not enough to figure out the
> reason OOM happened.
> 
> So, my questions are:
> 1. Should dump slab/vmalloc info when OOM happen? Though we can get these from proc file,
> but usually we do not monitor the logs and check proc file immediately when OOM happened.
> 

The problem is that slabinfo becomes excessively verbose and dumping it 
all to the kernel log often times causes important messages to be lost.  
This is why we control things like the tasklist dump with a VM sysctl.  It 
would be possible to dump, say, the top ten slab caches with the highest 
memory usage, but it will only be helpful for slab leaks.  Typically there 
are better debugging tools available than analyzing the kernel log; if you 
see unusually high slab memory in the meminfo dump, you can enable it.

> 2. /proc/$pid/smaps and pagecache info also helpful when OOM, should also be dumped?
> 

Also very verbose and would cause important messages to be lost, we try to 
avoid spamming the kernel log with all of this information as much as 
possible.

> 3. Without these info, usually how to figure out OOM reason?
> 

Analyze the memory usage in the meminfo and determine what is unusually 
high; if it's mostly anonymous memory, you can usually correlate it back 
to a high rss for a process in the tasklist that you didn't suspect to be 
using that much memory, for example.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/