Re: kCFI && patchable-function-entry=M,N

[Fangrui Song]

 On Mon, Oct 24, 2022 at 4:24 AM Mark Rutland <mark.rutland@xxxxxxx> wrote:
>
> On Sat, Oct 22, 2022 at 04:57:18PM +0200, Peter Zijlstra wrote:
> > On Fri, Oct 21, 2022 at 04:56:20PM +0100, Mark Rutland wrote:
> > > Hi,
> > >
> > > For arm64, I'd like to use -fatchable-function-entry=M,N (where N > 0), for our
> > > ftrace implementation, which instruments *some* but not all functions.
> > > Unfortuntately, this doesn't play nicely with -fsanitize=kcfi, as instrumented
> > > and non-instrumented functions don't agree on where the type hash should live
> > > relative to the function entry point, making them incompatible with one another.
> > > AFAICT, there's no mechanism today to get them to agree.
> > >
> > > Today we use -fatchable-function-entry=2, which happens to avoid this.
> >
> > > ... but I understand that for x86, folk want the pre-function NOPs to
> > > fall-through into the body of the function.
> >
> > Yep.
> >
> > > Is there any mechanism today that we could use to solve this, or could we
> > > extend clang to have some options to control this behaviour?
> >
> > So the main pain-point for you is differentiating between function with
> > notrace and those without it, right?
> >
> > That is; suppose you (like x86) globally do:
> > -fpatchable-function-entry=4,2 to get a consistent function signature,
> > you're up a creek because you use the __patchable_function_entries
> > section to drive ftrace and now every function will have it.
> >
> > So perhaps something like:
> >
> >  -fpatchable-function-entry=N,M,sectionname
> >
> > would help, then you can have notrace be the same layout, except a
> > different section. Eg. something like:
> >
> >  #define notrace __attribute__((patchable_function_entry(4,2,__notrace_function_entries)))
>
> FWIW, I think that'd work for me, and that was roughly my original proposal on
> IRC. My only concern with this approach is code size, since all uninstrumented
> functions gain some point less prefix NOPs.
>
> We can make that slghtly better as:
>
> #define notrace __attribute__((patchable_function_entry(2,2,__notrace_function_entries)))
>
> ... since we don't care about placing NOPs *within* the function
>
> > It does make the whole: CFLAGS_REMOVE_file.o = $(CC_FLAGS_FTRACE)
> > a bit of a pain, but I've long favoured removing all that and having
> > explitic notrace attributes on all relevant functions.
> >
> > Then again; perhaps it could be made to work by ensuring CFLAGS starts
> > with:
> >
> >  -fpatchable-function-entry=4,2,__notrace_function_entries
> >
> > and have CC_FLAGS_FTRACE include (and hence override with)
> >
> >  -fpatchable-function-entry=4,2,__ftrace_function_entries
> >
> > assuming that with duplicate argument the last is effective.
>
> TBH, it'd be nice to move ftrace to the `CFLAGS_WHATEVER_obj.o := n` approach
> the other instrumentation uses, which IIUC would allow us to define different
> flags for the two cases (though I'll need to go check that).
>
> Thanks,
> Mark.

Hi Mark and Peter,

Sami asked my opinion (as the main -fpatchable-function-entry=
implementer on the llvm-project side) on this extension
(-fpatchable-function-entry=4,2,__ftrace_function_entries).
I think this is fine.

You may consider bringing this up as a GCC feature request
(https://gcc.gnu.org/bugzilla/show_bug.cgi) and CCing the author/the
committer of https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=417ca0117a1a9a8aaf5bc5ca530adfd68cb00399
(original -fpatchable-function-entry= support) and the author of
https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=c23b5006d3ffeda1a9edf5fd817765a6da3696ca
(powerpc64 ELFv2 support).
On the feature request, a summary (so that toolchain people don't have
to read every message in this thread) will help:)



-- 
宋方睿