Re: [PATCH 4/4] thp: rewrite freeze_page()/unfreeze_page() with generic rmap walkers

[Kirill A. Shutemov]

On Wed, Feb 03, 2016 at 07:42:01AM -0800, Dave Hansen wrote:
> On 02/03/2016 07:14 AM, Kirill A. Shutemov wrote:
> > But the new variant is somewhat slower. Current helpers iterates over
> > VMAs the compound page is mapped to, and then over ptes within this VMA.
> > New helpers iterates over small page, then over VMA the small page
> > mapped to, and only then find relevant pte.
> 
> The code simplification here is really attractive.  Can you quantify
> what the slowdown is?  Is it noticeable, or would it be in the noise
> during all the other stuff that happens under memory pressure?

I don't know how to quantify it within whole memory pressure picture.
There're just too many variables to get some sense from split_huge_page()
contribution.

I've tried to measure split_huge_page() performance itself.

Testcase:

	#define _GNU_SOURCE
	#include <stdio.h>
	#include <stdlib.h>
	#include <unistd.h>
	#include <sys/mman.h>

	#define MB (1024UL * 1024)
	#define SIZE (4 * 1024 * 2 * MB)
	#define BASE ((void *)0x400000000000)

	#define FORKS 0

	int main()
	{
		char *p;
		unsigned long i;

		p = mmap(BASE, SIZE, PROT_READ | PROT_WRITE,
				MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE,
				-1, 0);
		if (p == MAP_FAILED)
			perror("mmap"), exit(1);

		for (i = 0; i < SIZE; i += 2 * MB) {
			munmap(p + i, 4096);
		}

		for (i = 0; i < FORKS; i++) {
			if (!fork())
				pause();
		}

		system("grep thp /proc/vmstat");
		system("time /bin/echo 3 > /proc/sys/vm/drop_caches");
		system("grep thp /proc/vmstat");
		return 0;
	}

Basically, we allocate 4k THP, make them partially unmapped, optionally
fork() the process multiple times and then trigger shrinker, measuring how
long would it take.

Optional fork() will make THP shared, meaning we need to freeze/unfreeze
ptes in multiple VMAs.

Numbers doesn't look pretty:

		FORKS == 0		FORKS == 100
Baseline:	1.93s ± 0.017s		32.08s ± 0.246s
Patched:	5.636s ± 0.021s		405.943s ± 6.126s
Slowdown:	2.92x			12.65x

With FORKS == 100, it looks especially bad. But having that many mapping
of the page is uncommon.

Any comments?

-- 
 Kirill A. Shutemov