Re: [PATCH v3 2/2] mm, thp: avoid unnecessary swapin in khugepaged

[Ebru Akagunduz]

On Thu, Mar 17, 2016 at 12:07:44PM +0100, Vlastimil Babka wrote:
> On 03/14/2016 10:40 PM, Ebru Akagunduz wrote:
> >Currently khugepaged makes swapin readahead to improve
> >THP collapse rate. This patch checks vm statistics
> >to avoid workload of swapin, if unnecessary. So that
> >when system under pressure, khugepaged won't consume
> >resources to swapin.
> >
> >The patch was tested with a test program that allocates
> >800MB of memory, writes to it, and then sleeps. The system
> >was forced to swap out all. Afterwards, the test program
> >touches the area by writing, it skips a page in each
> >20 pages of the area. When waiting to swapin readahead
> >left part of the test, the memory forced to be busy
> >doing page reclaim. There was enough free memory during
> >test, khugepaged did not swapin readahead due to business.
> >
> >Test results:
> >
> >                         After swapped out
> >-------------------------------------------------------------------
> >               | Anonymous | AnonHugePages | Swap      | Fraction  |
> >-------------------------------------------------------------------
> >With patch    | 206608 kB |  204800 kB    | 593392 kB |    %99    |
> >-------------------------------------------------------------------
> >Without patch | 351308 kB | 350208 kB     | 448692 kB |    %99    |
> >-------------------------------------------------------------------
> >
> >                         After swapped in (waiting 10 minutes)
> >-------------------------------------------------------------------
> >               | Anonymous | AnonHugePages | Swap      | Fraction  |
> >-------------------------------------------------------------------
> >With patch    | 551992 kB | 368640 kB     | 248008 kB |    %66    |
> >-------------------------------------------------------------------
> >Without patch | 586816 kB | 464896 kB     | 213184 kB |    %79    |
> >-------------------------------------------------------------------
> >
> >Signed-off-by: Ebru Akagunduz <ebru.akagunduz@xxxxxxxxx>
> 
> Looks like a step in a good direction. Still might be worthwile to
> also wait for the swapin to complete, and actually collapse
> immediately, no?
> 
I'll send a patch to solve mmap_sem issues after getting this
patch series accepted.
> >---
> >Changes in v2:
> >  - Add reference to specify which patch fixed (Ebru Akagunduz)
> 
> The reference is again missing in v3.
> 
> >  - Fix commit subject line (Ebru Akagunduz)
> >
> >Changes in v3:
> >  - Remove default values of allocstall (Kirill A. Shutemov)
> >
> >  mm/huge_memory.c | 13 +++++++++++--
> >  1 file changed, 11 insertions(+), 2 deletions(-)
> >
> >diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> >index 86e9666..67a398c 100644
> >--- a/mm/huge_memory.c
> >+++ b/mm/huge_memory.c
> >@@ -102,6 +102,7 @@ static DECLARE_WAIT_QUEUE_HEAD(khugepaged_wait);
> >   */
> >  static unsigned int khugepaged_max_ptes_none __read_mostly;
> >  static unsigned int khugepaged_max_ptes_swap __read_mostly;
> >+static unsigned long int allocstall;
> 
> "int" here is unnecessary
> 
> >
> >  static int khugepaged(void *none);
> >  static int khugepaged_slab_init(void);
> >@@ -2438,7 +2439,7 @@ static void collapse_huge_page(struct mm_struct *mm,
> >  	struct page *new_page;
> >  	spinlock_t *pmd_ptl, *pte_ptl;
> >  	int isolated = 0, result = 0;
> >-	unsigned long hstart, hend;
> >+	unsigned long hstart, hend, swap, curr_allocstall;
> >  	struct mem_cgroup *memcg;
> >  	unsigned long mmun_start;	/* For mmu_notifiers */
> >  	unsigned long mmun_end;		/* For mmu_notifiers */
> >@@ -2493,7 +2494,14 @@ static void collapse_huge_page(struct mm_struct *mm,
> >  		goto out;
> >  	}
> >
> >-	__collapse_huge_page_swapin(mm, vma, address, pmd);
> >+	swap = get_mm_counter(mm, MM_SWAPENTS);
> >+	curr_allocstall = sum_vm_event(ALLOCSTALL);
> >+	/*
> >+	 * When system under pressure, don't swapin readahead.
> >+	 * So that avoid unnecessary resource consuming.
> >+	 */
> >+	if (allocstall == curr_allocstall && swap != 0)
> >+		__collapse_huge_page_swapin(mm, vma, address, pmd);
> >
> >  	anon_vma_lock_write(vma->anon_vma);
> >
> >@@ -2790,6 +2798,7 @@ skip:
> >  			VM_BUG_ON(khugepaged_scan.address < hstart ||
> >  				  khugepaged_scan.address + HPAGE_PMD_SIZE >
> >  				  hend);
> >+			allocstall = sum_vm_event(ALLOCSTALL);
> 
> Why here? Rik said in v2:
> 
> >Khugepaged stores the allocstall value when it goes to sleep,
> >and checks it before calling (or not) __collapse_huge_page_swapin.
> 
> But that's not true, this is not "when it goes to sleep".
> So AFAICS it only observes the allocstalls between starting to scan
> a single pmd, and trying to collapse the pmd. So the window is quite
> tiny especially compared to I/O speeds, and this will IMHO catch
> only really frequent stalls. Placing it really at "when it goes to
> sleep" sounds better.
> 
> >  			ret = khugepaged_scan_pmd(mm, vma,
> >  						  khugepaged_scan.address,
> >  						  hpage);
> >
>