Re: [PATCH] x86: enable RCU based table free when PARAVIRT

[Kirill A. Shutemov]

On Wed, Aug 23, 2017 at 08:27:18PM +0000, Linus Torvalds wrote:
> On Wed, Aug 23, 2017 at 12:59 PM, Kirill A. Shutemov
> <kirill@xxxxxxxxxxxxx> wrote:
> >
> > In this case we need performance numbers for !PARAVIRT kernel.
> 
> Yes.
> 
> > Numbers for tight loop of "mmap(MAP_POPULATE); munmap()" might be
> > interesting too for worst case scenario.
> 
> Actually, I don't think you want to populate all the pages. You just
> want to populate *one* page, in order to build up the page directory
> structure, not allocate all the final points.
> 
> And we only free the actual page tables when there is nothing around,
> so it should be at least a 2MB-aligned region etc.
> 
> So you should do a *big* allocation, and then touch a single page in
> the middle, and then minmap it - that should give you maximal page
> table activity. Otherwise the page tables will generally just stay
> around.
> 
> Realistically, it's mainly exit() that frees page tables. Yes, you may
> have a few page tables free'd by a normal munmap(), but it's usually
> very limited. Which is why I suggested that script-heavy thing with
> lots of small executables. That tends to be the main realistic load
> that really causes a ton of page directory activity.

Below is test cases that allocates a lot of page tables and measuare
fork/exit time. (I'm not entirely sure it's the best way to stress the
codepath.)

Unpatched:	average 4.8322s, stddev	0.114s
Patched:	average 4.8362s, stddev	0.111s

Both without PARAVIRT. Patch is modified to enable HAVE_RCU_TABLE_FREE for
!PARAVIRT too.

The test-case requires "echo 1 > /proc/sys/vm/overcommit_memory".

#include <assert.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/prctl.h>
#include <sys/types.h>
#include <sys/wait.h>

#define PUD_SIZE (1UL << 30)
#define PMD_SIZE (1UL << 21)

#define NR_PUD 4096

#define NSEC_PER_SEC	1000000000L

int main(void)
{
	char *addr = NULL;
	unsigned long i, j;
	struct timespec start, finish;
	long long nsec;

	prctl(PR_SET_THP_DISABLE);
	for (i = 0; i < NR_PUD ; i++) {
		addr = mmap(addr + PUD_SIZE, PUD_SIZE, PROT_WRITE|PROT_READ,
				MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
		if (addr == MAP_FAILED) {
			perror("mmap");
			break;
		}

		for (j = 0; j < PUD_SIZE; j += PMD_SIZE)
			assert(addr[j] == 0);
	}

	for (i = 0; i < 10; i++) {
		pid_t pid;
		
		clock_gettime(CLOCK_MONOTONIC, &start);
		pid = fork();
		if (pid == -1)
			perror("fork");
		if (!pid)
			exit(0);
		wait(NULL);
		clock_gettime(CLOCK_MONOTONIC, &finish);

		nsec = (finish.tv_sec - start.tv_sec) * NSEC_PER_SEC +
			(finish.tv_nsec - start.tv_nsec);
		printf("%lld\n", nsec);
	}

	return 0;
}
-- 
 Kirill A. Shutemov