Re: [PATCH 4/7] objtool: Add support for return trampoline call

[Alexandre Chartre]

On 4/4/20 5:51 PM, Peter Zijlstra wrote:
> On Sat, Apr 04, 2020 at 09:22:32AM -0500, Josh Poimboeuf wrote:
> > On Sat, Apr 04, 2020 at 03:32:18PM +0200, Peter Zijlstra wrote:
> > > On Fri, Apr 03, 2020 at 05:46:20PM +0200, Peter Zijlstra wrote:
> > > > On Fri, Apr 03, 2020 at 10:17:57AM -0500, Josh Poimboeuf wrote:
> > > > > Peter, I think my previous idea for UNWIND_HINT_ADJUST stack_add=8 would
> > > > > work here?
> > > >
> > > > Yes, it would.
> > >
> > > Sorry, I have reconsidered. While it will shut up objtool, it will not
> > > 'work'. That is, the ORC data generated will not correctly unwind.
> > >
> > > I'll try and write a longer email tonight.
> >
> > Right, that's what I've been trying to say.  The ORC data will be
> > non-deterministic unless we unroll the loop.  Or did you mean something
> > else?
>
> The below should result in deterministic code.
>
> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
> index 07e95dcb40ad..109ee65f4a11 100644
> --- a/arch/x86/include/asm/nospec-branch.h
> +++ b/arch/x86/include/asm/nospec-branch.h
> @@ -59,8 +59,8 @@
>   	jmp	775b;				\
>   774:						\
>   	dec	reg;				\
> -	jnz	771b;				\
> -	add	$(BITS_PER_LONG/8) * nr, sp;
> +	add	$(BITS_PER_LONG/8) * $2, sp;	\
> +	jnz	771b;
>   
>   #ifdef __ASSEMBLY__

Nice. This works fine and allows to remove ANNOTATE_NOSPEC_ALTERNATIVE when
using __FILL_RETURN_BUFFER. However this is probably less performant because
we now have nr/2 add instructions instead of just 1.

Here is a variant where I unroll half of the loop. This way we have 2
add+dec+jnz instruction instructions instead of nr/2 dec+jnz and 1 add
instruction.

#define __FILL_RETURN_BUFFER(reg, nr, sp)	\
	mov	$1, reg;			\
771:						\
	.rept (nr/2);				\
	call	772f;				\
773:	/* speculation trap */			\
	pause;					\
	lfence;					\
	jmp	773b;				\
772:	;					\
	.endr;					\
	add	$(BITS_PER_LONG/8) * (nr/2), sp; \
	dec	reg;				\
	jnz	771b;


Note that we can't unroll the entire loop: it won't work with nr=32 because
the code is then too large to fit into an alternative (the alternative size
is encoded on only one byte so this allows a maximum size of 255, while with
nr=32 the size is around 390).

alex.