[QUESTION] oom killed the key system process triggered by a bad process alloc memory with MAP_LOCKED

From: Yongqiang Liu
Date: Mon Nov 01 2021 - 04:06:00 EST


Hi,

I was just wondering the process of  the patch "oom reaper: handle mlocked pages".

links:

https://lore.kernel.org/linux-mm/1454505240-23446-3-git-send-email-mhocko@xxxxxxxxxx/

https://lore.kernel.org/linux-mm/20160223132157.GD14178@xxxxxxxxxxxxxx/


In linux master, we found that when we start a process and alloc large memory with MAP_LOCKED, the oom will triggered and kill some system process such as sshd ,rsyslog etc.

...

[   45.110665] Out of memory: Killed process 2551 (oom) total-vm:1035324kB, anon-rss:1028196kB, file-rss:704kB, shmem-rss:0kB, UID:0 pgtables:2056kB oom_score_adj:1000
[   45.115303] Out of memory: Killed process 2554 (oom) total-vm:1025084kB, anon-rss:1015596kB, file-rss:388kB, shmem-rss:0kB, UID:0 pgtables:2032kB oom_score_adj:1000
[   45.115685] Out of memory: Killed process 2553 (oom) total-vm:953404kB, anon-rss:947748kB, file-rss:388kB, shmem-rss:0kB, UID:0 pgtables:1896kB oom_score_adj:1000
[   45.116031] Out of memory: Killed process 2552 (oom) total-vm:789564kB, anon-rss:783272kB, file-rss:388kB, shmem-rss:0kB, UID:0 pgtables:1576kB oom_score_adj:1000
[   45.117199] Out of memory: Killed process 2523 (sshd) total-vm:77052kB, anon-rss:804kB, file-rss:4672kB, shmem-rss:4kB, UID:0 pgtables:196kB oom_score_adj:0
[   45.120936] Out of memory: Killed process 2526 (bash) total-vm:17792kB, anon-rss:1180kB, file-rss:2868kB, shmem-rss:0kB, UID:0 pgtables:76kB oom_score_adj:0

...

the process demo is:

...

#define ALLOC_SIZE (10 * 1024 * 1024)

while (1) {

    addr = mmap(NULL, ALLOC_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_LOCKED, -1, 0);

    if (addr != MAP_FAILED)

        memset(addr, 1, ALLOC_SIZE);

}

...

And we found that when the oom_reaper is done but the memory is still high:

[   45.115685] Out of memory: Killed process 2553 (oom) total-vm:953404kB, anon-rss:947748kB, file-rss:388kB, shmem-rss:0kB, UID:0 pgtables:1896kB oom_score_adj:1000
[   45.115739] oom_reaper: reaped process 2553 (oom), now anon-rss:947708kB, file-rss:0kB, shmem-rss:0kB

This is because the bad proccess which recieved SIGKILL is unlocking the mem to exit which needs more time. And the next oom is triggered to kill the other system process.

Kind regards,

Yongqiang Liu