Re: [PATCH 5/8] mm: move lazily freed pages to inactive list

[Daniel Micay]

>>> It probably makes sense to stop thinking about them as anonymous pages
>>> entirely at this point when it comes to aging. They're really not. The
>>> LRU lists are split to differentiate access patterns and cost of page
>>> stealing (and restoring). From that angle, MADV_FREE pages really have
>>> nothing in common with in-use anonymous pages, and so they shouldn't
>>> be on the same LRU list.
>>>
>>> That would also fix the very unfortunate and unexpected consequence of
>>> tying the lazy free optimization to the availability of swap space.
>>>
>>> I would prefer to see this addressed before the code goes upstream.
>>
>> I don't think it would be ideal for these potentially very hot pages to
>> be dropped before very cold pages were swapped out. It's the kind of
>> tuning that needs to be informed by lots of real world experience and
>> lots of testing. It wouldn't impact the API.
> 
> What about them is hot? They contain garbage, you have to write to
> them before you can use them. Granted, you might have to refetch
> cachelines if you don't do cacheline-aligned populating writes, but
> you can do a lot of them before it's more expensive than doing IO.

It's hot because applications churn through memory via the allocator.

Drop the pages and the application is now churning through page faults
and zeroing rather than simply reusing memory. It's not something that
may happen, it *will* happen. A page in the page cache *may* be reused,
but often won't be, especially when the I/O patterns don't line up well
with the way it works.

The whole point of the feature is not requiring the allocator to have
elaborate mechanisms for aging pages and throttling purging. That ends
up resulting in lots of memory held by userspace where the kernel can't
reclaim it under memory pressure. If it's dropped before page cache, it
isn't going to be able to replace any of that logic in allocators.

The page cache is speculative. Page caching by allocators is not really
speculative. Using MADV_FREE on the pages at all is speculative. The
memory is probably going to be reused fairly soon (unless the process
exits, and then it doesn't matter), but purging will end up reducing
memory usage for the portions that aren't.

It would be a different story for a full unpinning/pinning feature since
that would have other use cases (speculative caches), but this is really
only useful in allocators.

>> Whether MADV_FREE is useful as an API vs. something like a pair of
>> system calls for pinning and unpinning memory is what should be worried
>> about right now. The internal implementation just needs to be correct
>> and useful right now, not perfect. Simpler is probably better than it
>> being more well tuned for an initial implementation too.
> 
> Yes, it wouldn't impact the API, but the dependency on swap is very
> random from a user experience and severely limits the usefulness of
> this. It should probably be addressed before this gets released. As
> this involves getting the pages off the anon LRU, we need to figure
> out where they should go instead.

From a user perspective, it doesn't depend on swap. It's just slower
without swap because it does what MADV_DONTNEED does. The current
implementation can be dropped in where MADV_DONTNEED was previously used.

Attachment: signature.asc
Description: OpenPGP digital signature