Re: [PATCH v2] arm64: kvm: Use has_vhe() instead of hyp_alternate_select()

From: Shanker Donthineni
Date: Mon Mar 06 2017 - 08:46:15 EST


Hi Marc,


On 03/06/2017 02:34 AM, Marc Zyngier wrote:
Hi Shanker,

On Mon, Mar 06 2017 at 2:33:18 am GMT, Shanker Donthineni <shankerd@xxxxxxxxxxxxxx> wrote:
Now all the cpu_hwcaps features have their own static keys. We don't
need a separate function hyp_alternate_select() to patch the vhe/nvhe
code. We can achieve the same functionality by using has_vhe(). It
improves the code readability, uses the jump label instructions, and
also compiler generates the better code with a fewer instructions.
How do you define "better"? Which compiler? Do you have any benchmarking data?
I'm using gcc version 5.2.0. With has_vhe() it shows the smaller code size as shown below. I tried to benchmark
the code changes using Cristiffer's microbench tool, but not seeing a noticeable difference on QDF2400 platform.

hyp_alternate_select() uses BR/BLR instructions to patch vhe/mvhe code, which is not good for branch prediction purpose.
compiler treats patched code as a function call, so the contents of the registers x0-x18 are not reusable after vhe/nvhe call.

Current code:
arch/arm64/kvm/hyp/switch.o: file format elf64-littleaarch64

Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00000000 0000000000000000 0000000000000000 00000040 2**0
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 00000000 0000000000000000 0000000000000000 00000040 2**0
CONTENTS, ALLOC, LOAD, DATA
2 .bss 00000000 0000000000000000 0000000000000000 00000040 2**0
ALLOC
3 .hyp.text 00000550 0000000000000000 0000000000000000 00000040 2**3
CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE

New code:
arch/arm64/kvm/hyp/switch.o: file format elf64-littleaarch64

Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00000000 0000000000000000 0000000000000000 00000040 2**0
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .data 00000000 0000000000000000 0000000000000000 00000040 2**0
CONTENTS, ALLOC, LOAD, DATA
2 .bss 00000000 0000000000000000 0000000000000000 00000040 2**0
ALLOC
3 .hyp.text 00000488 0000000000000000 0000000000000000 00000040 2**3
CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE


Signed-off-by: Shanker Donthineni <shankerd@xxxxxxxxxxxxxx>
---
v2: removed 'Change-Id: Ia8084189833f2081ff13c392deb5070c46a64038' from commit

arch/arm64/kvm/hyp/debug-sr.c | 12 ++++++----
arch/arm64/kvm/hyp/switch.c | 50 +++++++++++++++++++-----------------------
arch/arm64/kvm/hyp/sysreg-sr.c | 23 +++++++++----------
3 files changed, 43 insertions(+), 42 deletions(-)

diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index f5154ed..e5642c2 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -109,9 +109,13 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1)
dsb(nsh);
}
-static hyp_alternate_select(__debug_save_spe,
- __debug_save_spe_nvhe, __debug_save_spe_vhe,
- ARM64_HAS_VIRT_HOST_EXTN);
+static void __hyp_text __debug_save_spe(u64 *pmscr_el1)
+{
+ if (has_vhe())
+ __debug_save_spe_vhe(pmscr_el1);
+ else
+ __debug_save_spe_nvhe(pmscr_el1);
+}
I have two worries about this kind of thing:
- Not all compilers do support jump labels, leading to a memory access
on each static key (GCC 4.8, for example). This would immediately
introduce a pretty big regression
- The hyp_alternate_select() method doesn't introduce a fast/slow path
duality. Each path has the exact same cost. I'm not keen on choosing
what is supposed to be the fast path, really.
Yes, it'll require a runtime check if the compiler doesn't support ASM GOTO labels.
Agree, hyp_alternate_select() has a constant branch over head but it might cause a branch prediction penality.

Thanks,

M.

--
Shanker Donthineni
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.