Re: OOM killer invoked on 2.6.24.4

[Miquel van Smoorenburg]

On Mon, 2008-06-16 at 16:55 -0300, Marcelo Tosatti wrote:
> On Sat, Jun 14, 2008 at 05:31:18PM +0200, Bart Van Assche wrote:
> > On Sat, Jun 14, 2008 at 12:13 PM, Marco Barbero <marco.barbero@xxxxxxxxx> wrote:
> > > samba/slapd/heartbeat/drbd/mon
> > ...
> > > Jun 14 01:00:17 pippo kernel: Out of memory: kill process 6873 (slapd)
> > > score 5003 or a child
> > > Jun 14 01:00:17 pippo kernel: Killed process 6873 (slapd)
> > >
> > > Anyone can help me in understanding what went wrong? And if I need to
> > > upgrade to last kernel version?
> > 
> > One or more processes on your system used too much memory. Dumping the
> > output of the following command periodically (e.g. every 10 minutes)
> > to a file will tell you which process is using too much memory:
> > 
> > { ps aux | head -n 1; ps aux | sort -n +4 | tail -n 10; }
> 
> There were lots of swap free:
> 
> Jun 14 01:00:17 pippo kernel: Free swap  = 3903752kB
> Jun 14 01:00:17 pippo kernel: Total swap = 3903752kB
> Jun 14 01:00:17 pippo kernel: Free swap:       3903752kB
> 
> Something is wrong.

It was a normal GFP_KERNEL allocation:

Jun 14 01:00:17 pippo kernel: sendmail-mta invoked oom-killer:
gfp_mask=0x800d0, order=0, oomkilladj=0

And the system /is/ low on memory:

Jun 14 01:00:17 pippo kernel: DMA free:3548kB min:68kB low:84kB
high:100kB active:0kB inactive:4kB present:16256kB pages_scanned:25
all_unreclaimable? yes
Jun 14 01:00:17 pippo kernel: lowmem_reserve[]: 0 873 4810 4810
Jun 14 01:00:17 pippo kernel: Normal free:3736kB min:3744kB low:4680kB
high:5616kB active:4616kB inactive:4672kB present:894080kB
pages_scanned:14277 all_unreclaimable? yes
Jun 14 01:00:17 pippo kernel: lowmem_reserve[]: 0 0 31496 31496
Jun 14 01:00:17 pippo kernel: HighMem free:2975684kB min:512kB
low:4736kB high:8960kB active:248580kB inactive:21644kB
present:4031488kB pages_scanned:0 all_unreclaimable? no

Still high memory available, and swap, but ZONE_DMA and ZONE_NORMAL were
all used and "all_unreclaimable? yes" . So __alloc_pages(GFP_KERNEL)
fails.

It might be tcp buffers- see
http://marc.info/?l=linux-netdev&m=121362441431941&w=2

In which case, use this workaround:

echo "98304 131072 196608" > /proc/sys/net/ipv4/tcp_mem

Mike.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/