Re: backing up ext4 fs, system unresponsive, thrashing like crazyeven though swap is unused

From: Dimitrios Apostolou
Date: Sun Dec 02 2012 - 18:56:20 EST

Dear Ronald,

sorry for not replying at your first message but I didn't consider changing kernel as a resolution to my problem.

On Sun, 2 Dec 2012, Roland Eggner wrote:
Hello Dimitrios!

Which part of â

â0,5 â 1 G RAM is usually occupied just by kernel slab [1]; this
memory cannot be swapped, it cannot be released by other means than rebooting,
and there seems to be _no_ adjustment to memory pressure.â

â should I explain?

Check my initial message. If you see the /proc/meminfo attached you'll see that from all slab memory most is reclaimable, however it's not being reclaimed despite the memory pressure:

Slab: 64880 kB
SReclaimable: 60496 kB
SUnreclaim: 4384 kB

Also I mentioned that a nice'd -20 (Realtime!) process was unresponsive because of two nice'd + ionice'd processes. This is a bug no matter the memory pressure, IMHO. If not I'd rather hear it from someone more authoritative.

When tar or gzip writes to ext4 filesystem on your external disk, the kernel
keeps all inode data in slab memory _by design_ , not by memory leaks.

ext4_inode_cache is a *cache*. So the kernel does not keep /all/ inode data there, but only what was needed recently. And drops it afterwards, which didn't happen after the backup ended.

Several 100 M slab data cannot be stored within 128 M total RAM.
This cannot by solved by usage of /proc/sys/vm/drop_caches.

Please take a look at [2], it's what is called in that case. Via this call all slabs that have registered a shrinker, get actually shrinked. Other fs do, but I can't find whether ext4 actually registers a shrinker for its slabs.


This cannot by solved by oom-killer-actions.
This cannot be solved by zram tricks.
Unmounting the external disk would release part of slab memory, but then you
cannot backup.

Unmounting the back'ed up fs didn't really help in my case. Of course all my filesystems are ext[234], mounted by ext4 code, so they use the same slab for caches. But anyway, the slab didn't shrink significantly as I was expecting.

Reformatting the ext4 filesystem on your external disk with 128 byte inode size,
largest possible blocksize and with extents enabled might mitigate the memory
pressure slightly â probably not sufficient to get a working system with 3.6

One advantage of Linux compared to other OS is much more support for old
hardware, if a _proper_ kernel version is selected. Many years ago I used
a notebook with 40 M total RAM, with a 2.4 kernel, Blackbox window manager and
Opera web browser â it worked flawlessly â just rather slowly due to swapping.
Your notebook has much more RAM than 40 M, thus there is surely a Linux solution
for you. Try a kernel, it supports ext4 (âext4devâ), probably it
supports all devices of your notebook, and with a slim window manager e.g.
WindowMaker or OpenBox your notebook will probably âflyâ.

I appreciate your advice. I remember when we were running 2.4 much less memory was needed, but still I consider the kernel /fairly/ lean considering the time passed and the bloat in userspace applications. I choose to run latest kernels on old hardware, hopefully this will continue to be the case, and if things deteriorate much then let's hope we'll have enough time to help and fix them! :-)