Re: Why is Shmem included in Cached in /proc/meminfo?

[Mikko Rantalainen]

Matthew Wilcox (2021-08-30 19:17 Europe/Helsinki):
> On Mon, Aug 30, 2021 at 06:05:58PM +0200, Vlastimil Babka wrote:
>> On 8/30/21 16:41, Matthew Wilcox wrote:
>>> Another point of view is that everything in tmpfs is part of the page
>>> cache and can be written out to swap, so keeping it as part of Cached
>>> is not misleading.
>>
>> Yeah, but with that view, anonymous memory can be also written out to swap. But
>> it's non-intuitive that something called "Cached" will contain something that
>> (if not dirty) can't be just dropped.
> 
> That's equally true for normal filesystems & shmem though.  Consider shmem
> written to swap, then brought back in by a read.  Now it can be dropped
> without being swapped out.  Or even a file on shmem ftruncated to a
> large size, then only read.  The pages will be clean and full of zeroes.
> They can be dropped under memory pressure without being written out.
> 
>> I've had to point this Shmem oddity out a
>> number of times to someone, so I would say that it would be much better if it
>> was not part of Cached.
>> However, if we change it now, we might create even larger confusion. People
>> looking at the output for the first time (and IIRC also the 'free' command uses
>> it) on a new kernel wouldn't be misled anymore. But people working with both old
>> and new kernels will now have to take in account that it changed at some
>> point... not good.
> 
> Another good point.

I agree that backwards compatibility is a huge point to consider.

That said, I always assumed that Shmem was in *addition* to Cached and I
was happy to see lots of Cached on all my systems. However, when I
started to run into OOM situations with 10+ GB of Cached I guessed that
something was not working correctly with memory reclaim - it turned out
that the problem was that over 95% of my "Cached" was actually Shmem.

Do you know any existing software where it's important to have the
behavior of Cached and Shmem that the current kernel has?

Without backwards compatibility issues I'd prefer following fields:

- Dirty: total amount of RAM used to buffer data to be written on
permanent storage (dirty). Gets converted to Cached when write is
complete. (Actually I would call this "Buffers" but Dirty is okay, too.)
- Cached: total amount of RAM used to improve *performance* that can be
*immediately dropped* without any data-loss – note that this includes
all untouched RAM backed by swap.
- Shared: total amount of RAM shared between multiple process that
cannot be freed even if any single process gets killed. (If this is even
possible to know - note that this would *only* contain COW pages in
practice. We already have Committed_AS which is about as good for real
world heuristics.)
- RamDrive: total amount of RAM used for tmpfs storage and similar. Note
that this would tend towards zero when tmpfs storage is written to swap.
- Committed_RamDrive: virtual amount of space used for tmpfs (e.g. large
truncated files that, if written, will require to be backed for real).
- Swapped_RamDrive: the amount of swap backing tmpfs and similar.
- LegacyIPCS: all stuff reported by ipcs.

That is, I would declare "Cached" as true cache and split Shmem into
separate parts that are easier to understand. The Shmem currently seems
to be catch-all class for more and more stuff. Obviously the Shmem field
could be kept as precomputed sum of the separate parts for backwards
compatibility. Does Shmem nowadays include other big memory users but
ipcs and tmpfs?

My point is that it's currently too hard to understand (at least) tmpfs
and ipcs load on RAM and swap.

I guess the important thing to decide is if "Cached" should represent
actual cache size or is backwards compatibility more important? My guess
is that the backwards compability would affect statistics and heuristics
only but that's obviously just a guess.

Of course one possible solution is to keep "Cached" as is and introduce
"Cache" with the real cache semantics (that is, it includes sum of
(Cached - Shmem) and memory backed RAM). That way system administrators
would at least see two different fields with unique values and look for
the documentation.

-- 
Mikko