Re: [PATCH] mm: do not drain pagevecs for mlock

From: Tao Ma
Date: Fri Dec 30 2011 - 03:49:11 EST

On 12/30/2011 04:11 PM, KOSAKI Motohiro wrote:
> 2011/12/30 Tao Ma <tm@xxxxxx>:
>> In our test of mlock, we have found some severe performance regression
>> in it. Some more investigations show that mlocked is blocked heavily
>> by lur_add_drain_all which calls schedule_on_each_cpu and flush the work
>> queue which is very slower if we have several cpus.
>> So we have tried 2 ways to solve it:
>> 1. Add a per cpu counter for all the pagevecs so that we don't schedule
>> and flush the lru_drain work if the cpu doesn't have any pagevecs(I
>> have finished the codes already).
>> 2. Remove the lru_add_drain_all.
>> The first one has some problems since in our product system, all the cpus
>> are busy, so I guess there is very little chance for a cpu to have 0 pagevecs
>> except that you run several consecutive mlocks.
>> From the commit log which added this function(8891d6da), it seems that we
>> don't have to call it. So the 2nd one seems to be both easy and workable and
>> comes this patch.
> Could you please show us your system environment and benchmark programs?
> Usually lru_drain_** is very fast than mlock() body because it makes
> plenty memset(page).
The system environment is: 16 core Xeon E5620. 24G memory.

I have attached the program. It is very simple and just uses mlock/munlock.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <time.h>
#include <sys/time.h>
#include <sys/mman.h>

#define MM_SZ1 24
#define MM_SZ2 56
#define MM_SZ3 4168

void mlock_test()
char ptr1[MM_SZ1];
char ptr2[MM_SZ2];
char ptr3[MM_SZ3];

if(0 != mlock(ptr1, MM_SZ1) )
perror("mlock MM_SZ1\n");
if(0 != mlock(ptr2, MM_SZ2) )
perror("mlock MM_SZ2\n");
if(0 != mlock(ptr3, MM_SZ3) )
perror("mlock MM_SZ3\n");

if(0 != munlock(ptr1, MM_SZ1) )
perror("munlock MM_SZ1\n");
if(0 != munlock(ptr2, MM_SZ2) )
perror("munlock MM_SZ2\n");
if(0 != munlock(ptr3, MM_SZ3) )
perror("munlock MM_SZ3\n");

int main(int argc, char *argv[])
int ret, opt;
int i,cnt;

while((opt = getopt(argc, argv, "c:")) != -1 )
case 'c':
cnt = atoi(optarg);
printf("Usage: %s [-c count] arg...\n", argv[0]);

for(i = 0; i < cnt; i++)

return 0;