Re: oom-killings, but I'm not out of memory!

[Roy Keene]

Howdy,

	Roy Keene
	Planning Systems Inc.

On Sun, 3 Jul 2005, Marcelo Tosatti wrote:

> Hi Roy,
>
> On Sun, Jul 03, 2005 at 09:44:37PM -0500, Roy Keene wrote:
> > I think I'm having the same issue.
> >
> > I've 2 systems with 4GB of RAM and 2GB of swap that kill processes when
> > they get a lot of disk I/O.  I've attached the full dmesg output which
> > includes portions where it killed stuff despite having massive amounts of
> > free memory.
>
> What kernel version is that?

Linux cog2 2.6.9-11.ELsmp #1 SMP Fri May 20 18:26:27 EDT 2005 i686 i686 i386 GNU/Linux


> > oom-killer: gfp_mask=0xd0
> > Mem-info:
> > DMA per-cpu:
> > cpu 0 hot: low 2, high 6, batch 1
> > cpu 0 cold: low 0, high 2, batch 1
> > cpu 1 hot: low 2, high 6, batch 1
> > cpu 1 cold: low 0, high 2, batch 1
> > cpu 2 hot: low 2, high 6, batch 1
> > cpu 2 cold: low 0, high 2, batch 1
> > cpu 3 hot: low 2, high 6, batch 1
> > cpu 3 cold: low 0, high 2, batch 1
> > Normal per-cpu:
> > cpu 0 hot: low 32, high 96, batch 16
> > cpu 0 cold: low 0, high 32, batch 16
> > cpu 1 hot: low 32, high 96, batch 16
> > cpu 1 cold: low 0, high 32, batch 16
> > cpu 2 hot: low 32, high 96, batch 16
> > cpu 2 cold: low 0, high 32, batch 16
> > cpu 3 hot: low 32, high 96, batch 16
> > cpu 3 cold: low 0, high 32, batch 16
> > HighMem per-cpu:
> > cpu 0 hot: low 32, high 96, batch 16
> > cpu 0 cold: low 0, high 32, batch 16
> > cpu 1 hot: low 32, high 96, batch 16
> > cpu 1 cold: low 0, high 32, batch 16
> > cpu 2 hot: low 32, high 96, batch 16
> > cpu 2 cold: low 0, high 32, batch 16
> > cpu 3 hot: low 32, high 96, batch 16
> > cpu 3 cold: low 0, high 32, batch 16
> >
> > Free pages:       14304kB (1664kB HighMem)
> > Active:7971 inactive:994335 dirty:327523 writeback:25721 unstable:0 free:3576 slab:29113 mapped:7996 pagetables:341
>
> There are about 100M of writeout data onflight - I suppose thats too much.
>
> Guess: can you switch to another IO scheduler than CFQ or reduce its queue size?
>
> IIRC you can do that by reducing /sys/block/device/queue/nr_requests.

For all the devices I use this is set to 8192

I set "sda" to 512 now.

[root@cog2 ~]# cat /sys/block/{sda,nbd0,nbd1}/queue/nr_requests
8192
8192
8192


> > DMA free:12640kB min:16kB low:32kB high:48kB active:0kB inactive:0kB present:16384kB pages_scanned:877 all_unreclaimable? yes
> > protections[]: 0 0 0
> >
> > Normal free:0kB min:928kB low:1856kB high:2784kB active:0kB inactive:739100kB present:901120kB pages_scanned:1556742 all_unreclaimable? yes
> > protections[]: 0 0 0
>
> You've got no reservations for the normal zone either.
>
> How does /proc/sys/vm/lowmem_reserve_ratio looks like?

It looks like it doesn't exist..
[root@cog2 ~]# ls /proc/sys/vm
block_dump              dirty_ratio                laptop_mode 
max_map_count    nr_pdflush_threads  page-cluster
dirty_background_ratio  dirty_writeback_centisecs  legacy_va_layout 
min_free_kbytes  overcommit_memory   swappiness
dirty_expire_centisecs  hugetlb_shm_group          lower_zone_protection 
nr_hugepages     overcommit_ratio    vfs_cache_pressure
[root@cog2 ~]#
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/