Re: [PATCH -next V12 3/7] riscv: entry: Add noinstr to prevent instrumentation inserted

[Mark Rutland]

On Wed, Jan 04, 2023 at 09:40:38AM +0800, Guo Ren wrote:
> On Tue, Jan 3, 2023 at 5:12 PM Alexandre Ghiti <alex@xxxxxxxx> wrote:
> >
> > Hi Guo,
> >
> > On 1/3/23 04:35, guoren@xxxxxxxxxx wrote:
> > > From: Guo Ren <guoren@xxxxxxxxxxxxxxxxx>
> > >
> > > Without noinstr the compiler is free to insert instrumentation (think
> > > all the k*SAN, KCov, GCov, ftrace etc..) which can call code we're not
> > > yet ready to run this early in the entry path, for instance it could
> > > rely on RCU which isn't on yet, or expect lockdep state. (by peterz)
> > >
> > > Link: https://lore.kernel.org/linux-riscv/YxcQ6NoPf3AH0EXe@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/
> > > Reviewed-by: Björn Töpel <bjorn@xxxxxxxxxxxx>
> > > Suggested-by: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> > > Tested-by: Jisheng Zhang <jszhang@xxxxxxxxxx>
> > > Signed-off-by: Guo Ren <guoren@xxxxxxxxxxxxxxxxx>
> > > Signed-off-by: Guo Ren <guoren@xxxxxxxxxx>
> > > ---
> > >   arch/riscv/kernel/traps.c | 4 ++--
> > >   arch/riscv/mm/fault.c     | 2 +-
> > >   2 files changed, 3 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> > > index 549bde5c970a..96ec76c54ff2 100644
> > > --- a/arch/riscv/kernel/traps.c
> > > +++ b/arch/riscv/kernel/traps.c
> > > @@ -95,9 +95,9 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code,
> > >   }
> > >
> > >   #if defined(CONFIG_XIP_KERNEL) && defined(CONFIG_RISCV_ALTERNATIVE)
> > > -#define __trap_section               __section(".xip.traps")
> > > +#define __trap_section __noinstr_section(".xip.traps")
> > >   #else
> > > -#define __trap_section
> > > +#define __trap_section noinstr
> > >   #endif
> > >   #define DO_ERROR_INFO(name, signo, code, str)                               \
> > >   asmlinkage __visible __trap_section void name(struct pt_regs *regs) \
> > > diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
> > > index d86f7cebd4a7..b26f68eac61c 100644
> > > --- a/arch/riscv/mm/fault.c
> > > +++ b/arch/riscv/mm/fault.c
> > > @@ -204,7 +204,7 @@ static inline bool access_error(unsigned long cause, struct vm_area_struct *vma)
> > >    * This routine handles page faults.  It determines the address and the
> > >    * problem, and then passes it off to one of the appropriate routines.
> > >    */
> > > -asmlinkage void do_page_fault(struct pt_regs *regs)
> > > +asmlinkage void noinstr do_page_fault(struct pt_regs *regs)
> >
> >
> > (I dug the archive but can't find the series before v4, so sorry if it
> > was already answered)
> >
> > I think we should not disable the instrumentation of those trap handlers
> > as at least profiling them with ftrace would provide valuable
> > information (and gcov would be nice too): why do we need to do that? A
> > trap very early in the boot process is not recoverable anyway.
> Everything that calls irqentry_enter() should be noinstr, and this
> patch prepares for the next generic_entry convert.
> 
> eg:
> asmlinkage void noinstr do_page_fault(struct pt_regs *regs)
> {
>         irqentry_state_t state = irqentry_enter(regs);
> 
>         __do_page_fault(regs);
> 
>         local_irq_disable();
> 
>         irqentry_exit(regs, state);
> }
> NOKPROBE_SYMBOL(do_page_fault);
> 
> You still could profile __do_page_fault.
> 
> >
> > And I took a look at other architectures, none of them disables the
> > instrumentation on do_page_fault.
> That's not true, have a look at power & arm64. All of them have some
> limitations at the entry of page_fault.

Well, arm64's can't be kprobed, but is *can* be traced with ftrace, and *can*
be instrumented with KASAN and friends. I'm not sure that we actually need to
inhibit kprobes for do_page_fault, and we might be able to relax that.

As a general thing, we've tried to centralize all the necesarily-noinstr bits
in arch/arm64/kernel/entry-common.c, and keep everything else as instrumentable
as possible.

I'd recommend doing similar, and have a central file for any entry bits which
can't live in the generic entry code, and keep the rest instrumentable. That
will make it easier to maintain and verify.

Thanks,
Mark.