Re: [PATCH 09/10] ftrace/x86: Add register_ftrace_direct() for custom trampolines

[Steven Rostedt]

On Thu, 14 Nov 2019 16:34:58 +0100 (CET)
Miroslav Benes <mbenes@xxxxxxx> wrote:

> On Fri, 8 Nov 2019, Steven Rostedt wrote:
> 
> > From: "Steven Rostedt (VMware)" <rostedt@xxxxxxxxxxx>
> > 
> > Enable x86 to allow for register_ftrace_direct(), where a custom trampoline
> > may be called directly from an ftrace mcount/fentry location.
> > 
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@xxxxxxxxxxx>  
> 
> [...]
> 
> > +++ b/arch/x86/kernel/ftrace_64.S
> > @@ -88,6 +88,7 @@ EXPORT_SYMBOL(__fentry__)
> >  	movq %rdi, RDI(%rsp)
> >  	movq %r8, R8(%rsp)
> >  	movq %r9, R9(%rsp)
> > +	movq $0, ORIG_RAX(%rsp)
> >  	/*
> >  	 * Save the original RBP. Even though the mcount ABI does not
> >  	 * require this, it helps out callers.
> > @@ -114,7 +115,8 @@ EXPORT_SYMBOL(__fentry__)
> >  	subq $MCOUNT_INSN_SIZE, %rdi
> >  	.endm
> >  
> > -.macro restore_mcount_regs
> > +.macro restore_mcount_regs save=0
> > +
> >  	movq R9(%rsp), %r9
> >  	movq R8(%rsp), %r8
> >  	movq RDI(%rsp), %rdi
> > @@ -123,10 +125,7 @@ EXPORT_SYMBOL(__fentry__)
> >  	movq RCX(%rsp), %rcx
> >  	movq RAX(%rsp), %rax
> >  
> > -	/* ftrace_regs_caller can modify %rbp */
> > -	movq RBP(%rsp), %rbp
> > -
> > -	addq $MCOUNT_REG_SIZE, %rsp
> > +	addq $MCOUNT_REG_SIZE-\save, %rsp
> >  
> >  	.endm
> >  
> > @@ -228,10 +227,30 @@ GLOBAL(ftrace_regs_call)
> >  	movq R10(%rsp), %r10
> >  	movq RBX(%rsp), %rbx
> >  
> > -	restore_mcount_regs
> > +	movq RBP(%rsp), %rbp
> > +
> > +	movq ORIG_RAX(%rsp), %rax
> > +	movq %rax, MCOUNT_REG_SIZE-8(%rsp)
> > +
> > +	/* If ORIG_RAX is anything but zero, make this a call to that */
> > +	movq ORIG_RAX(%rsp), %rax
> > +	cmpq	$0, %rax
> > +	je	1f
> > +
> > +	/* Swap the flags with orig_rax */
> > +	movq MCOUNT_REG_SIZE(%rsp), %rdi
> > +	movq %rdi, MCOUNT_REG_SIZE-8(%rsp)
> > +	movq %rax, MCOUNT_REG_SIZE(%rsp)
> > +
> > +	restore_mcount_regs 8
> > +
> > +	jmp	2f
> > +
> > +1:	restore_mcount_regs
> > +
> >  
> >  	/* Restore flags */
> > -	popfq
> > +2:	popfq  
> 
> If I am reading the code correctly (and I was confused couple of times, so 
> maybe I am not), this is what makes the direct fops incompatible with 
> ipmodify and livepatching for now. Is that correct?

Actually, it's the fact that the return goes to some unknown trampoline
that may do something else as well.

> 
> What are your plans regarding this?

I wanted to see what the eBPF folks were doing, and then perhaps allow
the ip modify occur too. I could let it happen as well now, and then we
can see what the fallout is later ;-)

> 
> Moreover, we could replace ftrace_regs_caller with direct fops for live 
> patching when this is merged with all arch support we need. After all, all 

Note, direct call is currently only available for x86_64.

> we need is to change the rip, which we could do easily in the direct 
> trampoline. On the other hand, it would exclude coexistence of a live 
> patch and a BPF filter (both direct now) on one function.

It may also end up being more complex, and not much of a performance
benefit. I believe the BPF is injecting programs into the start of
functions, but your trampoline for live patching may be not much
different than what ftrace gives you today.

-- Steve