[PATCH 5.4 18/51] x86/speculation: Change FILL_RETURN_BUFFER to work with objtool

From: Greg Kroah-Hartman
Date: Wed Oct 05 2022 - 07:37:15 EST

From: Peter Zijlstra <peterz@xxxxxxxxxxxxx>

commit 089dd8e53126ebaf506e2dc0bf89d652c36bfc12 upstream.

Change FILL_RETURN_BUFFER so that objtool groks it and can generate
correct ORC unwind information.

- Since ORC is alternative invariant; that is, all alternatives
should have the same ORC entries, the __FILL_RETURN_BUFFER body
can not be part of an alternative.

Therefore, move it out of the alternative and keep the alternative
as a sort of jump_label around it.

- Use the ANNOTATE_INTRA_FUNCTION_CALL annotation to white-list
these 'funny' call instructions to nowhere.

- Use UNWIND_HINT_EMPTY to 'fill' the speculation traps, otherwise
objtool will consider them unreachable.

- Move the RSP adjustment into the loop, such that the loop has a
deterministic stack layout.

Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
Reviewed-by: Alexandre Chartre <alexandre.chartre@xxxxxxxxxx>
Acked-by: Josh Poimboeuf <jpoimboe@xxxxxxxxxx>
Link: https://lkml.kernel.org/r/20200428191700.032079304@xxxxxxxxxxxxx
[cascardo: fixup because of backport of ba6e31af2be96c4d0536f2152ed6f7b6c11bca47 ("x86/speculation: Add LFENCE to RSB fill sequence")]
[cascardo: no intra-function call validation support]
[cascardo: avoid UNWIND_HINT_EMPTY because of svm]
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@xxxxxxxxxxxxx>
Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
arch/x86/include/asm/nospec-branch.h | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)

--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -4,11 +4,13 @@

#include <linux/static_key.h>
+#include <linux/frame.h>

#include <asm/alternative.h>
#include <asm/alternative-asm.h>
#include <asm/cpufeatures.h>
#include <asm/msr-index.h>
+#include <asm/unwind_hints.h>

* This should be used immediately before a retpoline alternative. It tells
@@ -60,9 +62,9 @@
lfence; \
jmp 775b; \
774: \
+ add $(BITS_PER_LONG/8) * 2, sp; \
dec reg; \
jnz 771b; \
- add $(BITS_PER_LONG/8) * nr, sp; \
/* barrier for jnz misprediction */ \
@@ -154,10 +156,8 @@
.macro FILL_RETURN_BUFFER reg:req nr:req ftr:req
- ALTERNATIVE "jmp .Lskip_rsb_\@", \
- __stringify(__FILL_RETURN_BUFFER(\reg,\nr,%_ASM_SP)) \
- \ftr
+ ALTERNATIVE "jmp .Lskip_rsb_\@", "", \ftr