Re: [PATCH 1/4] glibc: Perform rseq(2) registration at C startup and thread creation (v7)

[Paul Burton]

Hi Mathieu,

Just following up on a couple of things here.

On Wed, Apr 24, 2019 at 11:05:42AM -0400, Mathieu Desnoyers wrote:
> >>> 2a. A uncommon TRAP hopefully with some immediate data encoded (maybe uncommon)
> >> 
> >> Our break instruction has a 19b immediate in nanoMIPS (20b for microMIPS
> >> & classic MIPS) so that could be something like:
> >> 
> >>  break 0x7273       # ASCII 'rs'
> >> 
> > 
> > I like this uncommon break instruction as signature choice.
> > 
> > However, if I try to compile assembler with a break 0x7273 instruction
> > with mips64 and mips32 toolchains (gcc version 8.2.0 (Ubuntu
> > 8.2.0-1ubuntu2~18.04))
> > I get:
> > 
> > /tmp/ccVh9F7T.s: Assembler messages:
> > /tmp/ccVh9F7T.s:24: Error: operand 1 out of range `break 0x7273'
> > 
> > It works up to the value 0x3FF, which seems to use the top 10
> > code bits:
> > 
> >   a:	03ff 0007 	break	0x3ff
> > 
> > Would a "break 0x350" be a good choice as well ?

The "break 0x350" instruction seems good to me - it's still going to be
rare.

> > Any idea why 0x7273 is not accepted by my assembler ?

I don't know why the assembler wants a smaller immediate than the
instruction encoding allows... There's a comment in the binutils file
include/opcode/mips.h that reads:

> A breakpoint instruction uses OP, CODE and SPEC (10 bits of the
> breakpoint instruction are not defined; Kane says the breakpoint code
> field in BREAK is 20 bits; yet MIPS assemblers and debuggers only use
> ten bits).  An optional two-operand form of break/sdbbp allows the
> lower ten bits to be set too, and MIPS32 and later architectures allow
> 20 bits to be set with a signal operand (using CODE20).

I suspect there's some history here that predates my involvement (or
possibly just predates me).

> > I also tried crafting the assembler with values between 0x3FF and 0x7273
> > in the 20 code bits. It seems fine from an objdump perspective:
> > 
> > ".long 0x03FFFC7\n\t"
> > 
> > generates:
> > 
> >  10:	003f ffc7 	break	0x3f,0x3ff
> > 
> > What I don't understand is why the instruction generated by my
> > toolchain ends with the last 6 bits "000111", whereas the mips32
> > instruction set specifies break as ending with "001101" [1].
> > What am I missing ?

Were you targeting microMIPS by any chance? There the break32
instructions ends with 000111.

> > Also, the nanomips break code [2] has a completely different
> > instruction layout. Should we use a different signature when
> > compiling for nanomips ? What #ifdef should we use ?

Yes, and __nanomips__. I included the encoding in my reply to your RFC
patch.

> > Do I need a special toolchain to generate nanomips binaries ?

Yes, you can find it here:

  https://codescape.mips.com/components/toolchain/nanomips/latest/index.html

We don't have the nanoMIPS kernel support in mainline yet, I've gotten
various things applied in preparation but also been swamped with other
things so it's taking a while. If you want to see a working downstream
kernel though you can find it here:

  git://git.linux-mips.org/pub/scm/linux-mti.git nanomips-v4.15

> Hi Paul, I'm still waiting for feedback on the MIPS front.
> 
> Meanwhile, I plan to use #define RSEQ_SIG 0x0350000d which maps to:
> 
>    0350000d 	break	0x350
> 
> and use RSEQ_SIG in assembly with:
> 
>    ".word " __rseq_str(RSEQ_SIG) "\n\t"
> 
> on big and little endian MIPS, for MIPS32 and MIPS64, based on
> code generated with gcc version 8.2.0 (Ubuntu 8.2.0-1ubuntu2~18.04).
> 
> Let me know if it needs to be tweaked.

That's fine for the classic MIPS ISA, but won't decode as a break for
microMIPS or nanoMIPS. See my reply to your RFC for valid encodings for
both of those.

Thanks,
    Paul