Re: 'disposable' dirty pages [was: Out Of Memory in v. 2.1]

Jamie Lokier (lkd@tantalophile.demon.co.uk)
Fri, 9 Oct 1998 16:03:11 +0100


On Thu, Oct 08, 1998 at 10:50:14AM -0400, Kenneth Albanowski wrote:
> Elegant. I proposed something related to the libc list a while ago
> (calling mmap(/dev/zero) on large-enough released chunks of memory), but
> this has the benefit of not making the VMA list grow for each release.

Note, mmap(/dev/zero) ought not to make the VMA list grow either -- the
memory is already in a zero-mapped VMA. I don't know if the VMA merging
code is clever enough to notice the file offset is irrelevant in this
case. It should be.

> However, it does mean that the memory is not automatically available to
> the system, at least not without a reclamation cycle. Consider a program
> which allocates 20meg on start, and then lives with 64k of RAM forever
> afterwards. That memory will never be released until the system actively
> goes looking, due to low-memory conditions.

The system already has this problem for many kinds of free memory, when
it's deciding what to swap out etc. Don't worry about it, the
reclamation seems to be reasonably efficient.

It could perhaps make better use of noticing pmds (not ptes) which are
old to narrow the search? I digress.

> I don't think it would make
> sense to count the DISCARDABLE pages as actual free memory, as it isn't
> available as such. Maybe another column in the memory report?

It might be reasonable count them in the "cached" column.

> If this were implemented (and, assuming a flag bit for DISCARDABLE bit is
> available, it seems to be worthwhile without any significant costs), then
> I'd also suggest a "harvester daemon" that goes searching for discardable
> pages, and reclaims then for the OS. Here (and with the low-memory
> reclamation, come to think of it) some concern needs to be taken for the
> VMA lists. Reclaiming a contiguous chunk of pages is more efficient then
> individual.

It's called kswapd, which already does that looking for pages to swap
out. Reclaiming DISCARDABLE pages is very similar. Or am I mistaken?

> > The process doesn't care whether any of those pages are discarded or
> > not.
>
> Which isn't quite true, unfortunately. To make this true, the act of
> making the page DISCARDABLE should also zero it.

That's silly. You don't need DISCARDABLE to zero pages -- just zero-map
them. If that makes a new VMA in malloc()ed memory, fix the kernel so
it doesn't because it shouldn't.

> Otherwise, you can tell whether a page was discarded by whether it
> contains data, which in turn means programs with bugs behaving
> differently depending on whether their pages were discarded.

Well they already have that problem, because free() does not,
traditionally, zero freed regions. Some debugging allocators do, and
there's nothing stopping you using one to debug the appliation.

> A minor problem, but these pages would not be transparent. (Free could
> of course zero the pages itself, and everyone using mmap directly
> merely needs to be aware of the issue.)

free() doesn't traditionally zero pages because that's slow.
Applications generally cope, though you're right to point out that bugs
get in this way. To make matters worse, user-space zeroing would dirty
all the pages in a block, which may will not have been touched
otherwise. For large regions, zero-mapping is better for this.

Efence is probably even better for debuggering.

-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/