Re: [PATCH] x86/asm: Force native_apic_mem_read to use mov

[H. Peter Anvin]

On August 11, 2022 1:03:11 PM PDT, Sean Christopherson <seanjc@xxxxxxxxxx> wrote:
>On Thu, Aug 11, 2022, H. Peter Anvin wrote:
>> On August 11, 2022 12:27:10 PM PDT, Sean Christopherson <seanjc@xxxxxxxxxx> wrote:
>> >On Thu, Aug 11, 2022, Adam Dunlap wrote:
>> >> Previously, when compiled with clang, native_apic_mem_read gets inlined
>> >> into __xapic_wait_icr_idle and optimized to a testl instruction. When
>> >> run in a VM with SEV-ES enabled, it attempts to emulate this
>> >> instruction, but the emulator does not support it. Instead, use inline
>> >> assembly to force native_apic_mem_read to use the mov instruction which
>> >> is supported by the emulator.
>> >> 
>> >> Signed-off-by: Adam Dunlap <acdunlap@xxxxxxxxxx>
>> >> Reviewed-by: Marc Orr <marcorr@xxxxxxxxxx>
>> >> Reviewed-by: Jacob Xu <jacobhxu@xxxxxxxxxx>
>> >> ---
>> >>  arch/x86/include/asm/apic.h | 13 ++++++++++++-
>> >>  1 file changed, 12 insertions(+), 1 deletion(-)
>> >> 
>> >> diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
>> >> index 3415321c8240..281db79e76a9 100644
>> >> --- a/arch/x86/include/asm/apic.h
>> >> +++ b/arch/x86/include/asm/apic.h
>> >> @@ -109,7 +109,18 @@ static inline void native_apic_mem_write(u32 reg, u32 v)
>> >>  
>> >>  static inline u32 native_apic_mem_read(u32 reg)
>> >>  {
>> >> -	return *((volatile u32 *)(APIC_BASE + reg));
>> >> +	volatile u32 *addr = (volatile u32 *)(APIC_BASE + reg);
>> >> +	u32 out;
>> >> +
>> >> +	/*
>> >> +	 * Functionally, what we want to do is simply return *addr. However,
>> >> +	 * this accesses an MMIO which may need to be emulated in some cases.
>> >> +	 * The emulator doesn't necessarily support all instructions, so we
>> >> +	 * force the read from addr to use a mov instruction.
>> >> +	 */
>> >> +	asm_inline("movl %1, %0" : "=r"(out) : "m"(*addr));
>> >> +
>> >> +	return out;
>> >
>> >Can't this just be:
>> >
>> >	return readl((void __iomem *)(APIC_BASE + reg));
>> 
>> The very point of the patch is to force a specific instruction sequence.
>
>Yes, and that specific emulator-friendly instruction also has to be forced for all
>of the core x86 read/write MMIO helpers.  And it's also possible for MMIO read/write
>to be enlightened to skip the MOV and go straight to #VMGEXIT, i.e. the xAPIC code
>shouldn't assume MOV is the best/only option (ignoring the handling of the P54C
>erratum in the write path).

That's not reasonable... but xAPIC is "special" enough.