Re: [RFC] mm/vmstat: Add events for THP migration without split

[John Hubbard]

On 2020-05-11 21:22, Anshuman Khandual wrote:
> Add the following new trace events which will help in validating migration
> events involving PMD based THP pages.
>
> 1. THP_PMD_MIGRATION_ENTRY_SET
> 2. THP_PMD_MIGRATION_ENTRY_REMOVE
>
> There are no clear method to confirm whether a THP migration happened with
> out involving it's split. These trace events along with PGMIGRATE_SUCCESS
> and PGMIGRATE_FAILURE will provide additional insights. After this change,


Hi Anshuman,

It's very nice to see this work, and I think that reporting a bit more
about THP migration stats is going to make development and performance
debugging a lot more efficient (and pleasant).


> A single 2M THP (2K base page) when migrated
>
> 1. Without split
>
> ................
> pgmigrate_success 1
> pgmigrate_fail 0
> ................
> thp_pmd_migration_entry_set 1
> thp_pmd_migration_entry_remove 1
> ................

I do think we should decouple the trace event name(s) just a *little* more,
from the mechanisms used to migrate THPs. In other words, let's report
the number of THP migration successes, and name it accordingly--rather
than "set" and "remove", which are pretty low-level and furthermore
depend on today's exact code.

Maybe Zi Yan's recommended name is exactly right, in fact:

    THP_PMD_MIGRATION_SUCCESS


thanks,
--
John Hubbard
NVIDIA


> 2. With split
>
> ................
> pgmigrate_success 512
> pgmigrate_fail 0
> ................
> thp_pmd_migration_entry_set 0
> thp_pmd_migration_entry_remove 0
> ................
>
> pgmigrate_success as 1 instead of 512, provides a hint for possible THP
> migration event. But then it gets mixed with normal page migrations over
> time. These additional trace events provide required co-relation.
>
> Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> Cc: "Kirill A. Shutemov" <kirill.shutemov@xxxxxxxxxxxxxxx>
> Cc: Zi Yan <ziy@xxxxxxxxxx>
> Cc: linux-mm@xxxxxxxxx
> Cc: linux-kernel@xxxxxxxxxxxxxxx
> Signed-off-by: Anshuman Khandual <anshuman.khandual@xxxxxxx>
> ---
> This is an indirect way for counting PMD migrations without split. Is there
> a better method possible ? Just request for comments at the moment.
>
>   include/linux/vm_event_item.h | 4 ++++
>   mm/migrate.c                  | 1 +
>   mm/rmap.c                     | 1 +
>   mm/vmstat.c                   | 4 ++++
>   4 files changed, 10 insertions(+)
>
> diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
> index ffef0f279747..4b25102cf3ad 100644
> --- a/include/linux/vm_event_item.h
> +++ b/include/linux/vm_event_item.h
> @@ -91,6 +91,10 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
>   		THP_ZERO_PAGE_ALLOC_FAILED,
>   		THP_SWPOUT,
>   		THP_SWPOUT_FALLBACK,
> +#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION
> +		THP_PMD_MIGRATION_ENTRY_SET,
> +		THP_PMD_MIGRATION_ENTRY_REMOVE,
> +#endif
>   #endif
>   #ifdef CONFIG_MEMORY_BALLOON
>   		BALLOON_INFLATE,
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 7160c1556f79..8d50d55cbe97 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -228,6 +228,7 @@ static bool remove_migration_pte(struct page *page, struct vm_area_struct *vma,
>   		if (!pvmw.pte) {
>   			VM_BUG_ON_PAGE(PageHuge(page) || !PageTransCompound(page), page);
>   			remove_migration_pmd(&pvmw, new);
> +			count_vm_event(THP_PMD_MIGRATION_ENTRY_REMOVE);
>   			continue;
>   		}
>   #endif
> diff --git a/mm/rmap.c b/mm/rmap.c
> index f79a206b271a..3c1fe3f45cb5 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -1418,6 +1418,7 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
>   			VM_BUG_ON_PAGE(PageHuge(page) || !PageTransCompound(page), page);
>   
>   			set_pmd_migration_entry(&pvmw, page);
> +			count_vm_event(THP_PMD_MIGRATION_ENTRY_SET);
>   			continue;
>   		}
>   #endif
> diff --git a/mm/vmstat.c b/mm/vmstat.c
> index 96d21a792b57..a5254b7ee531 100644
> --- a/mm/vmstat.c
> +++ b/mm/vmstat.c
> @@ -1274,6 +1274,10 @@ const char * const vmstat_text[] = {
>   	"thp_zero_page_alloc_failed",
>   	"thp_swpout",
>   	"thp_swpout_fallback",
> +#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION
> +	"thp_pmd_migration_entry_set",
> +	"thp_pmd_migration_entry_remove",
> +#endif
>   #endif
>   #ifdef CONFIG_MEMORY_BALLOON
>   	"balloon_inflate",