Re: [PATCH 2/4] arm64: prevent instrumentation of LL/SC atomics

[Will Deacon]

On Tue, Oct 17, 2017 at 12:10:33PM +0100, Mark Rutland wrote:
> On Tue, Oct 17, 2017 at 11:58:58AM +0100, Will Deacon wrote:
> > On Tue, Oct 17, 2017 at 11:54:54AM +0100, Mark Rutland wrote:
> > > On Tue, Oct 17, 2017 at 11:03:15AM +0100, Will Deacon wrote:
> > > > On Mon, Oct 16, 2017 at 02:24:38PM +0100, Mark Rutland wrote:
> > > > > While we build the LL/SC atomics as a C object file, this does not
> > > > > follow the AAPCS. This does not interoperate with other C code, and can
> > > > > only be called from special wrapper assembly.
> > > > > 
> > > > > Bulding a kernel with CONFIG_KCOV and CONFIG_ARM64_LSE_ATOMICS results
> > > > > in the cmopiler inserting calls to __sanitizer_cov_trace_pc within the
> > > > > LL/SC atomics. As __sanitizer_cov_trace_pc is built per the AAPCS, these
> > > > > calls corrupt register values, resulting in failures at boot time.
> > > > > 
> > > > > Avoid this (and other similar issues) by opting out of all compiler
> > > > > instrumentation. We can opt-in to specific instrumentation in future if
> > > > > we want to.
> 
> > > > > diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
> > > > > index a0abc142c92b..af77516f71b2 100644
> > > > > --- a/arch/arm64/lib/Makefile
> > > > > +++ b/arch/arm64/lib/Makefile
> > > > > @@ -17,5 +17,6 @@ CFLAGS_atomic_ll_sc.o	:= -fcall-used-x0 -ffixed-x1 -ffixed-x2		\
> > > > >  		   -fcall-saved-x10 -fcall-saved-x11 -fcall-saved-x12	\
> > > > >  		   -fcall-saved-x13 -fcall-saved-x14 -fcall-saved-x15	\
> > > > >  		   -fcall-saved-x18
> > > > > +CC_INSTRUMENT_atomic_ll_sc.o := n
> > > > 
> > > > Does this mean we can lose the "notrace" definition of __LL_SC_INLINE
> > > > when generating the out-of-line atomics?
> > > 
> > > Unfortunately not.
> > > 
> > > I'd missed -pg, since that isn't handled in scripts/Makefile.lib, and
> > > doesn't seem to have a makefile-level disable.
> > > 
> > > I'll see if that can be remedied.
> > 
> > Thanks. It's a real shame to have a "just use this option to avoid
> > instrumentation" if it doesn't actually catch everything. 
> 
> Agreed; it defeats the purpose of the exercise.
> 
> > We probably need to think about kprobes too, but not really sure what
> > you can do there on a per-file basis.
> 
> Ugh; that's a much more painful one, yes. :(
> 
> Does that rely on any compiler options at all? I thought was all a
> runtime thing.
> 
> Arguably it is somewhat separate for compiler instrumentation, and it
> might make sense for that to be a separate option.

Yes, I suppose the problem here is that opting out of dynamic tracing
requires function attributes such as notrace and __kprobes, rather than a
compiler flag.  If there's no way to say to the compiler "act as though
every function in this compilation unit is tagged with this attribute" then
we probably can't do anything to solve this easily.

We should probably add __kprobes to __LL_SC_INLINE though.

Will