Re: [PATCH] thp: reduce usage of huge zero page's atomic counter

[Anshuman Khandual]

On 08/30/2016 04:20 AM, Andrew Morton wrote:
> On Mon, 29 Aug 2016 14:31:20 +0800 Aaron Lu <aaron.lu@xxxxxxxxx> wrote:
> 
>> > 
>> > The global zero page is used to satisfy an anonymous read fault. If
>> > THP(Transparent HugePage) is enabled then the global huge zero page is used.
>> > The global huge zero page uses an atomic counter for reference counting
>> > and is allocated/freed dynamically according to its counter value.
>> > 
>> > CPU time spent on that counter will greatly increase if there are
>> > a lot of processes doing anonymous read faults. This patch proposes a
>> > way to reduce the access to the global counter so that the CPU load
>> > can be reduced accordingly.
>> > 
>> > To do this, a new flag of the mm_struct is introduced: MMF_USED_HUGE_ZERO_PAGE.
>> > With this flag, the process only need to touch the global counter in
>> > two cases:
>> > 1 The first time it uses the global huge zero page;
>> > 2 The time when mm_user of its mm_struct reaches zero.
>> > 
>> > Note that right now, the huge zero page is eligible to be freed as soon
>> > as its last use goes away.  With this patch, the page will not be
>> > eligible to be freed until the exit of the last process from which it
>> > was ever used.
>> > 
>> > And with the use of mm_user, the kthread is not eligible to use huge
>> > zero page either. Since no kthread is using huge zero page today, there
>> > is no difference after applying this patch. But if that is not desired,
>> > I can change it to when mm_count reaches zero.

> I suppose we could simply never free the zero huge page - if some
> process has used it in the past, others will probably use it in the
> future.  One wonders how useful this optimization is...

Yeah, what prevents us from doing away with this lock altogether and
keep one zero filled huge page (after a process has used it once) for
ever to be mapped across all the read faults ? A 16MB / 2MB huge page
is too much of memory loss on a THP enabled system ? We can also save
on allocation time.