Many unexplainable OOMs after upgrading to 4.7.x kernel
From: David Madore
Date: Thu Aug 25 2016 - 11:51:03 EST
TL;DR: Why is Firefox getting OOM-killed while I have 24GB free swap?
Dear list,
A few days ago I upgraded the kernel on my desktop PC from 4.5.5 to
4.7.2 and, since then, I've witnessed a huge number of cases where
various processes (typically Firefox) got OOM-killed by the kernel.
Before this kernel upgrade, I had never seen a single OOM event in
normal use; now I've had dozens in couple of days. Nothing has
changed in my config apart from the kernel. Clearly, something has
changed for the worse!
In fact, this morning, the problem was so bad that I was simply unable
to start Firefox (any attempt to do so would result in it getting
killed immediately), even though I had about 24GB of free swap
available (and the system was idle). I had to kill a few unrelated
processes to be able to launch Firefox; and even then, at some point,
running "sync" in a different window caused the Firefox to be
OOM-killed.
(Note: the problem IS NOT the choice of the process being killed:
Firefox is a reasonable target. The problem is that the OOM-killer is
being invoked at all, whereas there is plenty of free swap, and,
before this kernel upgrade, all seemed to work perfectly.)
Unfortunately, all of this is very unreproducible: now it seems to
have disappeared completely, so I can't really test anything any more.
All I can offer is some sample output (below) from the kernel's logs.
Any suggestions as to what I should do if and when the problem returns
is welcome, either to debug or to work around these OOMs.
The computer in question is an x86_64 (Intel Core 2 Quad Q6600) box
with 8GB RAM and 24GB swap (4*6GB swap across four different disks): I
can of course offer any additional details as to its hardware, kernel
or userland config.
Sample log output from OOM-killer:
### cut after ###
Aug 25 11:46:12 vega kernel: [115461.357412] firefox invoked oom-killer: gfp_mask=0x27000c0(GFP_KERNEL_ACCOUNT|__GFP_NOTRACK), order=2, oom_score_adj=0
Aug 25 11:46:12 vega kernel: [115461.357421] CPU: 2 PID: 11342 Comm: firefox Not tainted 4.7.2-vega #1
Aug 25 11:46:12 vega kernel: [115461.357424] Hardware name: System manufacturer System Product Name/P5W64 WS Pro, BIOS 0903 07/31/2007
Aug 25 11:46:12 vega kernel: [115461.357427] 0000000000000000 ffff880068897bc8 ffffffff812e4a7f 0000000000000000
Aug 25 11:46:12 vega kernel: [115461.357433] 0000000000000000 ffff880068897c28 ffffffff8117cd40 ffffffff818a9840
Aug 25 11:46:12 vega kernel: [115461.357438] 00000002810e98a5 ffff88023fd196a8 0000000000000002 0000000000000206
Aug 25 11:46:12 vega kernel: [115461.357442] Call Trace:
Aug 25 11:46:12 vega kernel: [115461.357451] [<ffffffff812e4a7f>] dump_stack+0x4d/0x6e
Aug 25 11:46:12 vega kernel: [115461.357456] [<ffffffff8117cd40>] dump_header.isra.16+0x51/0x191
Aug 25 11:46:12 vega kernel: [115461.357462] [<ffffffff8112e42b>] oom_kill_process+0x33b/0x430
Aug 25 11:46:12 vega kernel: [115461.357465] [<ffffffff8112e7c8>] out_of_memory+0x258/0x2a0
Aug 25 11:46:12 vega kernel: [115461.357470] [<ffffffff811326d9>] __alloc_pages_nodemask+0xaf9/0xc90
Aug 25 11:46:12 vega kernel: [115461.357474] [<ffffffff81132b66>] alloc_kmem_pages_node+0x16/0x20
Aug 25 11:46:12 vega kernel: [115461.357479] [<ffffffff81050c2e>] copy_process.part.42+0xfe/0x17a0
Aug 25 11:46:12 vega kernel: [115461.357483] [<ffffffff81138e70>] ? lru_cache_add_active_or_unevictable+0x30/0xa0
Aug 25 11:46:12 vega kernel: [115461.357487] [<ffffffff81158b7a>] ? handle_mm_fault+0x172a/0x1910
Aug 25 11:46:12 vega kernel: [115461.357491] [<ffffffff8105249c>] _do_fork+0xdc/0x350
Aug 25 11:46:12 vega kernel: [115461.357496] [<ffffffff81045b1d>] ? __do_page_fault+0x19d/0x4a0
Aug 25 11:46:12 vega kernel: [115461.357499] [<ffffffff81052794>] SyS_clone+0x14/0x20
Aug 25 11:46:12 vega kernel: [115461.357503] [<ffffffff810018eb>] do_syscall_64+0x4b/0xa0
Aug 25 11:46:12 vega kernel: [115461.357507] [<ffffffff8158713c>] entry_SYSCALL64_slow_path+0x25/0x25
Aug 25 11:46:12 vega kernel: [115461.357510] Mem-Info:
Aug 25 11:46:12 vega kernel: [115461.357516] active_anon:44076 inactive_anon:99299 isolated_anon:0
Aug 25 11:46:12 vega kernel: [115461.357516] active_file:392644 inactive_file:215736 isolated_file:0
Aug 25 11:46:12 vega kernel: [115461.357516] unevictable:560 dirty:85 writeback:0 unstable:0
Aug 25 11:46:12 vega kernel: [115461.357516] slab_reclaimable:1033111 slab_unreclaimable:31255
Aug 25 11:46:12 vega kernel: [115461.357516] mapped:34613 shmem:10947 pagetables:4477 bounce:0
Aug 25 11:46:12 vega kernel: [115461.357516] free:77874 free_pcp:0 free_cma:0
Aug 25 11:46:12 vega kernel: [115461.357529] DMA free:15028kB min:260kB low:324kB high:388kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15992kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Aug 25 11:46:12 vega kernel: [115461.357531] lowmem_reserve[]: 0 2960 7954 7954
Aug 25 11:46:12 vega kernel: [115461.357544] DMA32 free:187232kB min:50208kB low:62760kB high:75312kB active_anon:50528kB inactive_anon:94780kB active_file:512192kB inactive_file:389260kB unevictable:200kB isolated(anon):0kB isolated(file):0kB present:3128576kB managed:3053692kB mlocked:200kB dirty:56kB writeback:0kB mapped:38884kB shmem:14412kB slab_reclaimable:1574416kB slab_unreclaimable:42448kB kernel_stack:3264kB pagetables:6064kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Aug 25 11:46:12 vega kernel: [115461.357547] lowmem_reserve[]: 0 0 4993 4993
Aug 25 11:46:12 vega kernel: [115461.357558] Normal free:109236kB min:84696kB low:105868kB high:127040kB active_anon:125776kB inactive_anon:302416kB active_file:1058384kB inactive_file:473684kB unevictable:2040kB isolated(anon):0kB isolated(file):0kB present:5242880kB managed:5113700kB mlocked:2040kB dirty:284kB writeback:0kB mapped:99568kB shmem:29376kB slab_reclaimable:2558028kB slab_unreclaimable:82572kB kernel_stack:4960kB pagetables:11844kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:112 all_unreclaimable? no
Aug 25 11:46:12 vega kernel: [115461.357561] lowmem_reserve[]: 0 0 0 0
Aug 25 11:46:12 vega kernel: [115461.357565] DMA: 1*4kB (U) 0*8kB 1*16kB (U) 1*32kB (U) 0*64kB 1*128kB (U) 0*256kB 1*512kB (U) 0*1024kB 1*2048kB (U) 3*4096kB (M) = 15028kB
Aug 25 11:46:12 vega kernel: [115461.357585] DMA32: 43670*4kB (UME) 1596*8kB (UME) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 187448kB
Aug 25 11:46:12 vega kernel: [115461.357598] Normal: 27427*4kB (UME) 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 109708kB
Aug 25 11:46:12 vega kernel: [115461.357611] 620279 total pagecache pages
Aug 25 11:46:12 vega kernel: [115461.357614] 521 pages in swap cache
Aug 25 11:46:12 vega kernel: [115461.357617] Swap cache stats: add 8912, delete 8391, find 92286/92596
Aug 25 11:46:12 vega kernel: [115461.357619] Free swap = 24386280kB
Aug 25 11:46:12 vega kernel: [115461.357621] Total swap = 24418784kB
Aug 25 11:46:12 vega kernel: [115461.357623] 2096862 pages RAM
Aug 25 11:46:12 vega kernel: [115461.357625] 0 pages HighMem/MovableOnly
Aug 25 11:46:12 vega kernel: [115461.357626] 51038 pages reserved
Aug 25 11:46:12 vega kernel: [115461.357628] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Aug 25 11:46:12 vega kernel: [115461.357634] [ 2577] 0 2577 10235 524 22 3 265 -1000 udevd
Aug 25 11:46:12 vega kernel: [115461.357641] [15334] 0 15334 5956 478 16 3 46 0 sulogin
Aug 25 11:46:12 vega kernel: [115461.357644] [16567] 0 16567 6055 1 15 3 51 0 uuidd
Aug 25 11:46:12 vega kernel: [115461.357648] [16586] 0 16586 5891 590 16 3 16 0 cgmanager
Aug 25 11:46:12 vega kernel: [115461.357652] [16588] 0 16588 102634 446 33 3 95 0 rsyslogd
Aug 25 11:46:12 vega kernel: [115461.357656] [16653] 125 16653 102927 8159 63 3 1973 0 named
Aug 25 11:46:12 vega kernel: [115461.357659] [16732] 0 16732 4756 3 12 3 39 0 atd
Aug 25 11:46:12 vega kernel: [115461.357663] [16795] 0 16795 1848 277 8 3 13 0 gpm
Aug 25 11:46:12 vega kernel: [115461.357667] [16813] 127 16813 23186 434 46 3 1208 0 timidity
Aug 25 11:46:12 vega kernel: [115461.357670] [16818] 0 16818 10637 433 25 3 66 0 xdm
Aug 25 11:46:12 vega kernel: [115461.357674] [16850] 0 16850 59539 13195 112 3 757 0 Xorg
Aug 25 11:46:12 vega kernel: [115461.357677] [16852] 106 16852 10634 688 23 3 39 0 dbus-daemon
Aug 25 11:46:12 vega kernel: [115461.357681] [16878] 108 16878 1057 152 7 3 15 0 uml_switch
Aug 25 11:46:12 vega kernel: [115461.357685] [16960] 0 16960 23539 1068 51 3 129 0 xdm
Aug 25 11:46:12 vega kernel: [115461.357688] [16969] 0 16969 23284 632 48 3 260 0 apache2
Aug 25 11:46:12 vega kernel: [115461.357692] [16971] 33 16971 23218 453 47 3 307 0 apache2
Aug 25 11:46:12 vega kernel: [115461.357696] [16973] 33 16973 23231 415 47 3 270 0 apache2
Aug 25 11:46:12 vega kernel: [115461.357699] [16974] 33 16974 343482 1163 93 4 265 0 apache2
Aug 25 11:46:12 vega kernel: [115461.357703] [16976] 33 16976 261558 1432 88 4 263 0 apache2
Aug 25 11:46:12 vega kernel: [115461.357707] [17106] 0 17106 6417 434 16 4 78 0 bluetoothd
Aug 25 11:46:12 vega kernel: [115461.357711] [17131] 65534 17131 3159 389 10 3 34 0 gidentd
Aug 25 11:46:12 vega kernel: [115461.357714] [17138] 0 17138 3566 667 10 3 62 0 mdadm
Aug 25 11:46:12 vega kernel: [115461.357718] [17260] 0 17260 4819 400 13 3 29 0 irqbalance
Aug 25 11:46:12 vega kernel: [115461.357722] [17299] 0 17299 6476 545 17 3 46 0 cron
Aug 25 11:46:12 vega kernel: [115461.357725] [17348] 113 17348 8091 490 20 3 31 0 avahi-daemon
Aug 25 11:46:12 vega kernel: [115461.357729] [17349] 113 17349 8025 15 19 3 48 0 avahi-daemon
Aug 25 11:46:12 vega kernel: [115461.357733] [17552] 0 17552 13796 636 29 3 139 -1000 sshd
Aug 25 11:46:12 vega kernel: [115461.357736] [17672] 0 17672 5054 493 13 3 27 0 xinetd
Aug 25 11:46:12 vega kernel: [115461.357740] [17710] 0 17710 6187 501 17 3 162 0 smartd
Aug 25 11:46:12 vega kernel: [115461.357743] [17747] 140 17747 30233 2684 47 4 239 -900 postgres
Aug 25 11:46:12 vega kernel: [115461.357747] [17838] 140 17838 30233 2025 56 4 228 0 postgres
Aug 25 11:46:12 vega kernel: [115461.357751] [17839] 140 17839 30233 1006 46 4 234 0 postgres
Aug 25 11:46:12 vega kernel: [115461.357755] [17840] 140 17840 30233 1201 42 4 253 0 postgres
Aug 25 11:46:12 vega kernel: [115461.357758] [17841] 140 17841 30340 1365 44 4 262 0 postgres
Aug 25 11:46:12 vega kernel: [115461.357763] [17842] 140 17842 20108 810 39 4 235 0 postgres
Aug 25 11:46:12 vega kernel: [115461.357766] [18013] 0 18013 9042 635 21 3 116 0 master
Aug 25 11:46:12 vega kernel: [115461.357770] [18037] 103 18037 9599 671 22 3 116 0 qmgr
Aug 25 11:46:12 vega kernel: [115461.357773] [18053] 141 18053 8861 838 19 3 43 0 ntpd
Aug 25 11:46:12 vega kernel: [115461.357777] [18139] 0 18139 1076 562 7 3 0 -1000 watchdog
Aug 25 11:46:12 vega kernel: [115461.357781] [18198] 0 18198 4212 468 12 3 1 0 getty
Aug 25 11:46:12 vega kernel: [115461.357784] [18199] 0 18199 4212 488 12 3 1 0 getty
Aug 25 11:46:12 vega kernel: [115461.357788] [18200] 0 18200 4212 486 11 3 2 0 getty
Aug 25 11:46:12 vega kernel: [115461.357791] [18201] 0 18201 4212 448 12 3 2 0 getty
Aug 25 11:46:12 vega kernel: [115461.357795] [20639] 0 20639 4212 445 12 3 0 0 getty
Aug 25 11:46:12 vega kernel: [115461.357798] [20641] 0 20641 4212 449 12 3 0 0 getty
Aug 25 11:46:12 vega kernel: [115461.357802] [20658] 500 20658 3918 440 12 3 0 0 sh
Aug 25 11:46:12 vega kernel: [115461.357805] [20701] 500 20701 2675 84 8 3 0 0 ssh-agent
Aug 25 11:46:12 vega kernel: [115461.357809] [20706] 500 20706 6126 424 16 3 0 0 dbus-launch
Aug 25 11:46:12 vega kernel: [115461.357812] [20707] 500 20707 10589 541 23 3 0 0 dbus-daemon
Aug 25 11:46:12 vega kernel: [115461.357816] [20720] 500 20720 16894 1144 39 3 0 0 xscreensaver
Aug 25 11:46:12 vega kernel: [115461.357819] [20722] 500 20722 10128 942 25 3 0 0 xsm
Aug 25 11:46:12 vega kernel: [115461.357823] [20723] 500 20723 2675 488 10 3 0 0 ssh-agent
Aug 25 11:46:12 vega kernel: [115461.357827] [20725] 500 20725 17555 1264 37 3 0 0 xclock
Aug 25 11:46:12 vega kernel: [115461.357830] [20726] 500 20726 21062 2324 45 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357834] [20727] 500 20727 128194 4945 117 3 0 0 gnome-panel
Aug 25 11:46:12 vega kernel: [115461.357837] [20728] 500 20728 34770 2968 66 3 0 0 fvwm
Aug 25 11:46:12 vega kernel: [115461.357841] [20737] 500 20737 84439 1325 33 3 0 0 at-spi-bus-laun
Aug 25 11:46:12 vega kernel: [115461.357844] [20741] 500 20741 10531 568 24 3 0 0 dbus-daemon
Aug 25 11:46:12 vega kernel: [115461.357848] [20744] 500 20744 31306 977 31 3 0 0 at-spi2-registr
Aug 25 11:46:12 vega kernel: [115461.357852] [20749] 500 20749 12876 1065 28 3 0 0 gconfd-2
Aug 25 11:46:12 vega kernel: [115461.357855] [20752] 500 20752 53206 924 36 3 0 0 gvfsd
Aug 25 11:46:12 vega kernel: [115461.357859] [20756] 500 20756 44612 730 23 3 0 0 dconf-service
Aug 25 11:46:12 vega kernel: [115461.357862] [20761] 500 20761 5269 330 13 3 0 0 FvwmCommandS
Aug 25 11:46:12 vega kernel: [115461.357866] [20762] 500 20762 30629 1306 59 3 0 0 FvwmPager
Aug 25 11:46:12 vega kernel: [115461.357870] [20763] 500 20763 32987 1691 66 3 0 0 FvwmTaskBar
Aug 25 11:46:12 vega kernel: [115461.357873] [20765] 500 20765 9965 785 22 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357877] [20782] 500 20782 55428 882 40 4 0 0 gvfs-udisks2-vo
Aug 25 11:46:12 vega kernel: [115461.357880] [20836] 500 20836 54679 1514 39 4 0 0 gvfs-gphoto2-vo
Aug 25 11:46:12 vega kernel: [115461.357884] [20841] 500 20841 52973 980 37 4 0 0 gvfs-mtp-volume
Aug 25 11:46:12 vega kernel: [115461.357888] [20845] 500 20845 51382 846 34 3 0 0 gvfs-goa-volume
Aug 25 11:46:12 vega kernel: [115461.357892] [20849] 500 20849 75773 935 45 4 0 0 gvfs-afc-volume
Aug 25 11:46:12 vega kernel: [115461.357895] [20854] 500 20854 77000 1070 51 3 0 0 mission-control
Aug 25 11:46:12 vega kernel: [115461.357899] [20876] 500 20876 33408 870 33 3 0 0 gvfsd-metadata
Aug 25 11:46:12 vega kernel: [115461.357903] [20892] 500 20892 21063 2305 43 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357906] [20894] 500 20894 10494 892 24 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357910] [21586] 500 21586 21063 2333 45 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357913] [21588] 500 21588 10626 924 25 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357917] [21603] 500 21603 134536 14593 158 3 0 0 emacs
Aug 25 11:46:12 vega kernel: [115461.357920] [22199] 500 22199 11346 430 24 3 0 0 aspell
Aug 25 11:46:12 vega kernel: [115461.357924] [25744] 500 25744 91726 1023 43 4 0 0 gvfsd-trash
Aug 25 11:46:12 vega kernel: [115461.357927] [29740] 0 29740 6411 527 17 3 0 0 vsftpd
Aug 25 11:46:12 vega kernel: [115461.357931] [ 588] 500 588 21063 2304 44 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357935] [ 590] 500 590 10097 971 23 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357938] [ 606] 500 606 21063 2341 43 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357942] [ 610] 500 610 10097 921 22 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357945] [ 626] 500 626 21063 2312 45 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357948] [ 628] 500 628 10097 902 22 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357952] [ 643] 500 643 21063 2323 45 4 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357955] [ 645] 500 645 10097 936 23 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357959] [ 677] 500 677 21063 2313 45 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357962] [ 679] 500 679 10097 912 24 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357966] [ 1439] 500 1439 21954 3203 47 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357969] [ 1441] 500 1441 10626 933 24 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357973] [ 1456] 500 1456 21063 2331 44 4 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357976] [ 1458] 500 1458 9965 784 24 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357980] [ 1501] 500 1501 21063 2322 45 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357983] [ 1503] 500 1503 9965 814 24 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357987] [ 1519] 500 1519 21063 2323 44 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357990] [ 1521] 500 1521 9965 800 23 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.357994] [ 1539] 500 1539 21063 2339 43 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.357997] [ 1541] 500 1541 10626 943 25 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.358001] [ 1563] 500 1563 82916 2930 91 3 0 0 audacious
Aug 25 11:46:12 vega kernel: [115461.358029] [ 1569] 500 1569 21063 2332 42 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.358033] [ 1571] 500 1571 10626 946 24 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.358036] [ 1589] 500 1589 7970 661 20 3 0 0 alsamixer
Aug 25 11:46:12 vega kernel: [115461.358040] [ 2436] 500 2436 21063 2331 45 3 0 0 xterm
Aug 25 11:46:12 vega kernel: [115461.358043] [ 2438] 500 2438 10494 792 25 3 0 0 zsh
Aug 25 11:46:12 vega kernel: [115461.358047] [ 2880] 143 2880 561268 25824 155 6 0 0 java
Aug 25 11:46:12 vega kernel: [115461.358050] [ 2927] 140 2927 30488 2984 52 4 192 0 postgres
Aug 25 11:46:12 vega kernel: [115461.358054] [ 3035] 500 3035 2626 42 9 3 0 0 idn
Aug 25 11:46:12 vega kernel: [115461.358058] [ 8964] 500 8964 106938 7001 107 4 0 0 gweather-applet
Aug 25 11:46:12 vega kernel: [115461.358061] [ 9286] 0 9286 39732 1787 46 3 0 0 cupsd
Aug 25 11:46:12 vega kernel: [115461.358065] [10663] 500 10663 2060 168 9 3 0 0 sleep
Aug 25 11:46:12 vega kernel: [115461.358069] [11095] 103 11095 9558 971 22 3 0 0 pickup
Aug 25 11:46:12 vega kernel: [115461.358072] [11294] 500 11294 12138 1305 26 3 0 0 ssh
Aug 25 11:46:12 vega kernel: [115461.358076] [11295] 500 11295 12138 1320 28 3 0 0 ssh
Aug 25 11:46:12 vega kernel: [115461.358079] [11339] 109 11339 5258 517 15 3 0 0 in.telnetd
Aug 25 11:46:12 vega kernel: [115461.358083] [11340] 0 11340 18520 728 40 3 0 0 login
Aug 25 11:46:12 vega kernel: [115461.358086] [11342] 500 11342 207359 58119 291 4 0 0 firefox
Aug 25 11:46:12 vega kernel: [115461.358089] Out of memory: Kill process 11342 (firefox) score 7 or sacrifice child
Aug 25 11:46:12 vega kernel: [115461.358161] Killed process 11342 (firefox) total-vm:829436kB, anon-rss:142748kB, file-rss:83152kB, shmem-rss:6576kB
### cut before ###
--
David A. Madore
( http://www.madore.org/~david/ )