Re: [PATCH v3 4/4] KVM: arm64: Clear active_vmids on vCPU schedule out
From: Will Deacon
Date: Mon Aug 09 2021 - 09:09:26 EST
On Fri, Aug 06, 2021 at 12:24:36PM +0000, Shameerali Kolothum Thodi wrote:
> These are some test numbers with and without this patch, run on two
> different test setups.
>
>
> a)Test Setup -1
> -----------------------
>
> Platform: HiSilicon D06 with 128 CPUs, VMID bits = 16
> Run 128 VMs concurrently each with 2 vCPUs. Each Guest will execute hackbench
> 5 times before exiting.
>
> Measurements taken avg. of 10 Runs.
>
> Image : 5.14-rc3
> ---------------------------
> Time(s) 44.43813888
> No. of exits 145,348,264
>
> Image: 5.14-rc3 + vmid-v3
> ----------------------------------------
> Time(s) 46.59789034
> No. of exits 133,587,307
>
> %diff against 5.14-rc3
> Time: 4.8% more
> Exits: 8% less
>
> Image: 5.14-rc3 + vmid-v3 + Without active_asid clear
> ---------------------------------------------------------------------------
> Time(s) 44.5031782
> No. of exits 144,443,188
>
> %diff against 5.14-rc3
> Time: 0.15% more
> Exits: 2.42% less
>
> b)Test Setup -2
> -----------------------
>
> Platform: HiSilicon D06 + Kernel with maxcpus set to 8 and VMID bits set to 4.
> Run 40 VMs concurrently each with 2 vCPUs. Each Guest will execute hackbench
> 5 times before exiting.
>
> Measurements taken avg. of 10 Runs.
>
> Image : 5.14-rc3-vmid4bit
> ------------------------------------
> Time(s) 46.19963266
> No. of exits 23,699,546
>
> Image: 5.14-rc3-vmid4bit + vmid-v3
> ---------------------------------------------------
> Time(s) 45.83307736
> No. of exits 23,260,203
>
> %diff against 5.14-rc3-vmid4bit
> Time: 0.8% less
> Exits: 1.85% less
>
> Image: 5.14-rc3-vmid4bit + vmid-v3 + Without active_asid clear
> -----------------------------------------------------------------------------------------
> Time(s) 44.5031782
> No. of exits 144,443,188
Really? The *exact* same numbers as the "Image: 5.14-rc3 + vmid-v3 + Without
active_asid clear" configuration? Guessing a copy-paste error here.
> %diff against 5.14-rc3-vmid4bit
> Time: 1.05% less
> Exits: 2.06% less
>
> As expected, the active_asid clear on schedule out is not helping.
> But without this patch, the numbers seems to be better than the
> vanilla kernel when we force the setup(cpus=8, vmd=4bits)
> to perform rollover.
I'm struggling a bit to understand these numbers. Are you saying that
clearing the active_asid helps in the 16-bit VMID case but not in the
4-bit case?
Why would the active_asid clear have any impact on the number of exits?
The problem I see with not having the active_asid clear is that we will
roll over more frequently as the number of reserved VMIDs increases.
Will