Re: [PATCH v10 00/16] Volatile Ranges v10

[KOSAKI Motohiro]

Hi Minchan,


On Thu, Jan 2, 2014 at 2:12 AM, Minchan Kim <minchan@xxxxxxxxxx> wrote:
> Hey all,
>
> Happy New Year!
>
> I know it's bad timing to send this unfamiliar large patchset for
> review but hope there are some guys with freshed-brain in new year
> all over the world. :)
> And most important thing is that before I dive into lots of testing,
> I'd like to make an agreement on design issues and others
>
> o Syscall interface
> o Not bind with vma split/merge logic to prevent mmap_sem cost and
> o Not bind with vma split/merge logic to avoid vm_area_struct memory
>   footprint.
> o Purging logic - when we trigger purging volatile pages to prevent
>   working set and stop to prevent too excessive purging of volatile
>   pages
> o How to test
>   Currently, we have a patched jemalloc allocator by Jason's help
>   although it's not perfect and more rooms to be enhanced but IMO,
>   it's enough to prove vrange-anonymous. The problem is that
>   lack of benchmark for testing vrange-file side. I hope that
>   Mozilla folks can help.
>
> So its been a while since the last release of the volatile ranges
> patches, again. I and John have been busy with other things.
> Still, we have been slowly chipping away at issues and differences
> trying to get a patchset that we both agree on.
>
> There's still a few issues, but we figured any further polishing of
> the patch series in private would be unproductive and it would be much
> better to send the patches out for review and comment and get some wider
> opinions.
>
> You could get full patchset by git
>
> git clone -b vrange-v10-rc5 --single-branch git://git.kernel.org/pub/scm/linux/kernel/git/minchan/linux.git

Brief comments.

- You should provide jemalloc patch too. Otherwise we cannot
understand what the your mesurement mean.
- Your number only claimed the effectiveness anon vrange, but not file vrange.
- Still, Nobody likes file vrange. At least nobody said explicitly on
the list. I don't ack file vrange part until
  I fully convinced Pros/Cons. You need to persuade other MM guys if
you really think anon vrange is not
  sufficient. (Maybe LSF is the best place)
- I wrote you need to put a mesurement current implementation vs
VMA-based implementation at several
  previous iteration. Because You claimed fast, but no number and you
haven't yet. I guess the reason is
  you don't have any access to large machine. If so, I'll offer it.
Plz collaborate with us.

Unfortunately, I'm very busy and I didn't have a chance to review your
latest patch yet. But I'll finish it until
mm summit. And, I'll show you guys how much this patch improve glibc malloc too.

I and glibc folks agreed we push vrange into glibc malloc.

https://sourceware.org/ml/libc-alpha/2013-12/msg00343.html

Even though, I still dislike some aspect of this patch. I'd like to
discuss and make better design decision
with you.

Thanks.


>
> In v10, there are some notable changes following as
>
> Whats new in v10:
> * Fix several bugs and build break
> * Add shmem_purge_page to correct purging shmem/tmpfs
> * Replace slab shrinker with direct hooked reclaim path
> * Optimize pte scanning by caching previous place
> * Reorder patch and tidy up Cc-list
> * Rebased on v3.12
> * Add vrange-anon test with jemalloc in Dhaval's test suite
>   - https://github.com/volatile-ranges-test/vranges-test
>   so, you could test any application with vrange-patched jemalloc by
>   LD_PRELOAD but please keep in mind that it's just a prototype to
>   prove vrange syscall concept so it has more rooms to optimize.
>   So, please do not compare it with another allocator.
>
> Whats new in v9:
> * Updated to v3.11
> * Added vrange purging logic to purge anonymous pages on
>   swapless systems
> * Added logic to allocate the vroot structure dynamically
>   to avoid added overhead to mm and address_space structures
> * Lots of minor tweaks, changes and cleanups
>
> Still TODO:
> * Sort out better solution for clearing volatility on new mmaps
>         - Minchan has a different approach here
> * Agreement of systemcall interface
> * Better discarding trigger policy to prevent working set evction
> * Review, Review, Review.. Comment.
> * A ton of test
>
> Feedback or thoughts here would be particularly helpful!
>
> Also, thanks to Dhaval for his maintaining and vastly improving
> the volatile ranges test suite, which can be found here:
> [1]     https://github.com/volatile-ranges-test/vranges-test
>
> These patches can also be pulled from git here:
>     git://git.linaro.org/people/jstultz/android-dev.git dev/vrange-v9
>
> We'd really welcome any feedback and comments on the patch series.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/