Re: [PATCH -v4 0/9] migrate_pages(): batch TLB flushing

[Jonathan Cameron]

On Wed, 8 Feb 2023 14:27:32 +0800
haoxin <xhao@xxxxxxxxxxxxxxxxx> wrote:

> 在 2023/2/8 下午2:21, haoxin 写道:
> >
> > On my arm64 server with 128 cores, 2 numa nodes.
> >
> > I used memhog as benchmark :
> >
> >     numactl -m -C 5 memhog -r100000 1G
> >  
>      Do a fix, numactl -m 0 -C 5 memhog -r100000 1G

Nice results - thanks for sharing.

Just to confirm, is this with this tlb batching on arm64 patch set?
https://lore.kernel.org/linux-arm-kernel/20221117082648.47526-1-yangyicong@xxxxxxxxxx/

I think that still hasn't been applied upstream.

Jonathan

> 
> > The test result as below:
> >
> >  With this patch:
> >
> >     #time migratepages 8490 0 1
> >
> >     real 0m1.161s
> >
> >     user 0m0.000s
> >
> >     sys 0m1.161s
> >
> > without this patch:
> >
> >     #time migratepages 8460 0 1
> >
> >     real 0m2.068s
> >
> >     user 0m0.001s
> >
> >     sys 0m2.068s
> >
> > So you can see the migration performance improvement about *+78%*
> >
> > This is the perf record info.
> >
> > w/o
> > +   51.07%     0.09%  migratepages  [kernel.kallsyms]  [k] 
> > migrate_folio_extra
> > +   42.43%     0.04%  migratepages  [kernel.kallsyms]  [k] folio_copy
> > +   42.34%    42.34%  migratepages  [kernel.kallsyms]  [k] __pi_copy_page
> > +   33.99%     0.09%  migratepages  [kernel.kallsyms]  [k] rmap_walk_anon
> > +   32.35%     0.04%  migratepages  [kernel.kallsyms]  [k] try_to_migrate
> > *+   27.78%    27.78%  migratepages  [kernel.kallsyms]  [k] 
> > ptep_clear_flush *
> > +    8.19%     6.64%  migratepages  [kernel.kallsyms]  [k] 
> > folio_migrate_flagsmigrati_tlb_flush
> >
> > w/ this patch
> > +   18.57%     0.13%  migratepages     [kernel.kallsyms]   [k] 
> > migrate_pages
> > +   18.23%     0.07%  migratepages     [kernel.kallsyms]   [k] 
> > migrate_pages_batch
> > +   16.29%     0.13%  migratepages     [kernel.kallsyms]   [k] 
> > migrate_folio_move
> > +   12.73%     0.10%  migratepages     [kernel.kallsyms]   [k] 
> > move_to_new_folio
> > +   12.52%     0.06%  migratepages     [kernel.kallsyms]   [k] 
> > migrate_folio_extra
> >
> > Therefore, this patch helps improve performance in page migration
> >
> >
> > So,  you can add Tested-by: Xin Hao <xhao@xxxxxxxxxxxxxxxxx>
> >
> >
> > 在 2023/2/6 下午2:33, Huang Ying 写道:  
> >> From: "Huang, Ying"<ying.huang@xxxxxxxxx>
> >>
> >> Now, migrate_pages() migrate folios one by one, like the fake code as
> >> follows,
> >>
> >>    for each folio
> >>      unmap
> >>      flush TLB
> >>      copy
> >>      restore map
> >>
> >> If multiple folios are passed to migrate_pages(), there are
> >> opportunities to batch the TLB flushing and copying.  That is, we can
> >> change the code to something as follows,
> >>
> >>    for each folio
> >>      unmap
> >>    for each folio
> >>      flush TLB
> >>    for each folio
> >>      copy
> >>    for each folio
> >>      restore map
> >>
> >> The total number of TLB flushing IPI can be reduced considerably.  And
> >> we may use some hardware accelerator such as DSA to accelerate the
> >> folio copying.
> >>
> >> So in this patch, we refactor the migrate_pages() implementation and
> >> implement the TLB flushing batching.  Base on this, hardware
> >> accelerated folio copying can be implemented.
> >>
> >> If too many folios are passed to migrate_pages(), in the naive batched
> >> implementation, we may unmap too many folios at the same time.  The
> >> possibility for a task to wait for the migrated folios to be mapped
> >> again increases.  So the latency may be hurt.  To deal with this
> >> issue, the max number of folios be unmapped in batch is restricted to
> >> no more than HPAGE_PMD_NR in the unit of page.  That is, the influence
> >> is at the same level of THP migration.
> >>
> >> We use the following test to measure the performance impact of the
> >> patchset,
> >>
> >> On a 2-socket Intel server,
> >>
> >>   - Run pmbench memory accessing benchmark
> >>
> >>   - Run `migratepages` to migrate pages of pmbench between node 0 and
> >>     node 1 back and forth.
> >>
> >> With the patch, the TLB flushing IPI reduces 99.1% during the test and
> >> the number of pages migrated successfully per second increases 291.7%.
> >>
> >> This patchset is based on v6.2-rc4.
> >>
> >> Changes:
> >>
> >> v4:
> >>
> >> - Fixed another bug about non-LRU folio migration.  Thanks Hyeonggon!
> >>
> >> v3:
> >>
> >> - Rebased on v6.2-rc4
> >>
> >> - Fixed a bug about non-LRU folio migration.  Thanks Mike!
> >>
> >> - Fixed some comments.  Thanks Baolin!
> >>
> >> - Collected reviewed-by.
> >>
> >> v2:
> >>
> >> - Rebased on v6.2-rc3
> >>
> >> - Fixed type force cast warning.  Thanks Kees!
> >>
> >> - Added more comments and cleaned up the code.  Thanks Andrew, Zi, Alistair, Dan!
> >>
> >> - Collected reviewed-by.
> >>
> >> from rfc to v1:
> >>
> >> - Rebased on v6.2-rc1
> >>
> >> - Fix the deadlock issue caused by locking multiple pages synchronously
> >>    per Alistair's comments.  Thanks!
> >>
> >> - Fix the autonumabench panic per Rao's comments and fix.  Thanks!
> >>
> >> - Other minor fixes per comments. Thanks!
> >>
> >> Best Regards,
> >> Huang, Ying