Re: [PATCH RFC] fadvise: move active pages to inactive list withPOSIX_FADV_DONTNEED

[Pádraig Brady]

On 23/06/11 14:57, Andrea Righi wrote:
> On Thu, Jun 23, 2011 at 12:14:21PM +0100, Pádraig Brady wrote:
>> On 22/06/11 22:51, Andrea Righi wrote:
>>> There were some reported problems in the past about trashing page cache
>>> when a backup software (i.e., rsync) touches a huge amount of pages (see
>>> for example [1]).
>>>
>>> This problem has been almost fixed by the Minchan Kim's patch [2] and a
>>> proper use of fadvise() in the backup software. For example this patch
>>> set [3] has been proposed for inclusion in rsync.
>>>
>>> However, there can be still other similar trashing problems: when the
>>> backup software reads all the source files, some of them may be part of
>>> the actual working set of the system. When a
>>> posix_fadvise(POSIX_FADV_DONTNEED) is performed _all_ pages are evicted
>>> from pagecache, both the working set and the use-once pages touched only
>>> by the backup software.
>>>
>>> With the following solution when posix_fadvise(POSIX_FADV_DONTNEED) is
>>> called for an active page instead of removing it from the page cache it
>>> is added to the tail of the inactive list. Otherwise, if it's already in
>>> the inactive list the page is removed from the page cache.
>>>
>>> In this way if the backup was the only user of a page, that page will
>>> be immediately removed from the page cache by calling
>>> posix_fadvise(POSIX_FADV_DONTNEED). If the page was also touched by
>>> other processes it'll be moved to the inactive list, having another
>>> chance of being re-added to the working set, or simply reclaimed when
>>> memory is needed.
>>>
>>> Testcase:
>>>
>>>   - create a 1GB file called "zero"
>>>   - run md5sum zero to read all the pages in page cache (this is to
>>>     simulate the user activity on this file)
>>>   - run "rsync zero zero_copy" (rsync is patched with [3])
>>>   - re-run md5sum zero (user activity on the working set) and measure
>>>     the time to complete this command
>>>
>>> The test has been performed using 3.0.0-rc4 vanilla and with this patch
>>> applied (3.0.0-rc4-fadvise).
>>>
>>> Results:
>>>                   avg elapsed time      block:block_bio_queue
>>>  3.0.0-rc4                  4.127s                      8,214
>>>  3.0.0-rc4-fadvise          2.146s                          0
>>>
>>> In the first case the file is evicted from page cache completely and we
>>> must re-read it from the disk. In the second case the file is still in
>>> page cache (in the inactive list) and we don't need any other additional
>>> I/O operation.
>>>
>>> [1] http://marc.info/?l=rsync&m=128885034930933&w=2
>>> [2] https://lkml.org/lkml/2011/2/20/57
>>> [3] http://lists.samba.org/archive/rsync/2010-November/025827.html
>>>
>>> Signed-off-by: Andrea Righi <andrea@xxxxxxxxxxxxxxx>
>>
>> Hmm, What if you do want to evict it from the cache for testing purposes?
>> Perhaps this functionality should be associated with POSIX_FADV_NOREUSE?
>> dd has been recently modified to support invalidating the cache for a file,
>> and it uses POSIX_FADV_DONTNEED for that.
>> http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commitdiff;h=5f311553
> 
> I don't have any objection to associate POSIX_FADV_NOREUSE to this
> functionality. Actually maintaining a specific functionality to drop
> file cache pages can be useful, indeed.
> 
> However, I'm not sure if POSIX_FADV_NOREUSE or POSIX_FADV_DONTNEED
> either are suitable.
> 
> According to the standard:
> 
>  POSIX_FADV_NOREUSE = data will be accessed only once
>  POSIX_FADV_DONTNEED = data will not be accessed in the near future


> So, associating the "drop the page cache" semantic sounds like an
> implementation detail and applications shouldn't implicitly rely on this
> behaviour.

Well the "standard" really is what has been implemented up to now.
POSIX_FADV_NOREUSE currently does nothing so, associating this
new behavior with it seems less problematic for user space.
Also the names fit pretty well I think.

  POSIX_FADV_DONTNEED = drop if possible
  POSIX_FADV_NOREUSE = current app won't reuse so reduce cache eligibility

cheers,
Pádraig.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/