Re: linux-next: Tree for Aug 27 (objtool)

[Masami Hiramatsu]

Hi Josh,

On Thu, 29 Aug 2019 12:59:31 -0500
Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:

> On Thu, Aug 29, 2019 at 10:53:56AM +0900, Masami Hiramatsu wrote:
> > Hi Josh,
> > 
> > On Wed, 28 Aug 2019 11:34:33 -0500
> > Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:
> > 
> > > On Wed, Aug 28, 2019 at 11:13:31AM -0500, Josh Poimboeuf wrote:
> > > > Turns out this patch does break something:
> > > > 
> > > >   arch/x86/xen/enlighten_pv.o: warning: objtool: xen_cpuid()+0x25: can't find jump dest instruction at .text+0x9c
> > > > 
> > > > I'll need to figure out a better way to whitelist that
> > > > XEN_EMULATE_PREFIX fake instruction thing.  I'll probably just teach
> > > > the objtool decoder about it.
> > > 
> > > Hi Masami,
> > > 
> > > Is it possible for the kernel x86 decoder to recognize the
> > > XEN_EMULATE_PREFIX prefix?
> > > 
> > >         asm(XEN_EMULATE_PREFIX "cpuid"
> > >                 : "=a" (*ax),
> > >                   "=b" (*bx),
> > >                   "=c" (*cx),
> > >                   "=d" (*dx)
> > >                 : "0" (*ax), "2" (*cx));
> > > 
> > > is disassembled to:
> > > 
> > >       33:       0f 0b                   ud2
> > >       35:       78 65                   js     9c <xen_store_tr+0xc>
> > >       37:       6e                      outsb  %ds:(%rsi),(%dx)
> > >       38:       0f a2                   cpuid
> > > 
> > > which confuses objtool.  Presumably that would confuse other users of
> > > the decoder as well.
> > 
> > Good catch! It should be problematic, since x86 decoder sanity test is
> > based on objtool.
> 
> I think you mean the decoder test is based on objdump, not objtool?

Yes, it was my mistake. It depends on objdump.

> Actually I wonder if X86_DECODER_SELFTEST is even still needed these
> days, since objtool is enabled on default configs.  Objtool already uses
> the decoder to disassemble every instruction in the kernel (except for a
> few whitelisted files).

Sometimes it have found bugs, so I would like to keep it. That test runs
build time and in-kernel decoder is somewhat critical. It is better to
run a test before install it.

> 
> > But I don't want to change the test code itself,
> > because this problem is highly depending on Xen.
> > 
> > > That's a highly unlikely sequence of instructions, maybe the kernel
> > > decoder should recognize it as a single instruction.
> > 
> > OK, it is better to be done in decoder (only for CONFIG_XEN_PVHVM)
> > 
> > BTW, could you also share what test case would you using?
> 
> Enable CONFIG_XEN_PV and CONFIG_STACK_VALIDATION, and remove the
> STACK_FRAME_NON_STANDARD(xen_cpuid) line from
> arch/x86/xen/enlighten_pv.c.  objtool will complain:
> 
>   arch/x86/xen/enlighten_pv.o: warning: objtool: xen_cpuid()+0x25: can't find jump dest instruction at .text+0x9c

Ah, OK, so that is for objtool, not for in-kernel decoder (anyway both
need the fix.)

> Basing it on CONFIG_XEN_PVHVM may be problematic.  The decoder is
> duplicated in the tools directory so objtool can use it.  But the tools
> don't know about kernel configs.

Yes, in that case you need enable it always.

> BTW, I'm not sure if you're aware of this, but both objtool and perf
> have identical copies of the decoder.  The makefiles warn if they get
> out of sync with the kernel version.
> 
> We will always need at least one copy of the decoder in tools, because
> the tools subdir is supposed to be standalone from the rest of the
> kernel.  Still, I may look at combining the perf and objtool copies into
> a single shared copy.

Yes, we need to fix both.

> 
> > And what about attached patch? (just compile checked with/without CONFIG_XEN_PVHVM)
> 
> I copied the decoder to objtool, removed the CONFIG_XEN_PVHVM ifdef, and
> played a bit with the includes, and got it to compile with objtool, but
> it still fails:
> 
>   $ make arch/x86/xen/enlighten_pv.o
>   arch/x86/xen/enlighten_pv.o: warning: objtool: xen_cpuid()+0x25: can't find jump dest instruction at .text+0x9c

[...]
> @@ -58,6 +60,30 @@ void insn_init(struct insn *insn, const void *kaddr, int buf_len, int x86_64)
>  		insn->addr_bytes = 4;
>  }
>  
> +static const insn_byte_t xen_prefix[] = { XEN_EMULATE_PREFIX };

Oops, this must be __XEN_EMULATE_PREFIX. Mine is also have same bug.
since insn_byte_t is char, that makes no error, but it should be
initialized with __XEN_EMULATE_PREFIX, not XEN_EMULATE_PREFIX.

Thank you,

-- 
Masami Hiramatsu <mhiramat@xxxxxxxxxx>