Re: [RFC PATCH 2/4] rseq: Allow extending struct rseq

[Mathieu Desnoyers]

----- On Jul 14, 2020, at 10:34 PM, Chris Kennelly ckennelly@xxxxxxxxxx wrote:

> On Tue, Jul 14, 2020 at 2:33 PM Peter Oskolkov <posk@xxxxxxxxxx> wrote:
>>
>> On Tue, Jul 14, 2020 at 10:43 AM Mathieu Desnoyers
>> <mathieu.desnoyers@xxxxxxxxxxxx> wrote:
>> >
>> > ----- On Jul 14, 2020, at 1:24 PM, Peter Oskolkov posk@xxxxxxx wrote:
>> >
>> > > At Google, we actually extended struct rseq (I will post the patches
>> > > here once they are fully deployed and we have specific
>> > > benefits/improvements to report). We did this by adding several fields
>> > > below __u32 flags (the last field currently), and correspondingly
>> > > increasing rseq_len in rseq() syscall. If the kernel does not know of
>> > > this extension, it will return -EINVAL due to an unexpected rseq_len;
>> > > then the application can either fall-back to the standard/upstream
>> > > rseq, or bail. If the kernel does know of this extension, it accepts
>> > > it. If the application passes the old rseq_len (32), the kernel knows
>> > > that this is an old application and treats it as such.
>> > >
>> > > I looked through the archives, but I did not find specifically why the
>> > > pretty standard approach described above is considered inferior to the
>> > > one taken in this patch (freeze rseq_len at 32, add additional length
>> > > fields to struct rseq). Can these be summarized?
>> >
>> > I think you don't face the issues I'm facing with libc rseq integration
>> > because you control the entire user-space software ecosystem at Google.
>> >
>> > The main issue we face is that the library responsible for registering
>> > rseq (either glibc 2.32+, an early-adopter librseq library, or the
>> > application) may very well not be the same library defining the __rseq_abi
>> > symbol used in the global symbol table. Interposition with ld preload or
>> > by defining the __rseq_abi in the program's executable are good examples
>> > of this kind of scenario, and those use-cases are supported.
> 
> Does this work if/when we run out of bytes in the current sizeof(__rseq_abi)?

Only if all libraries/programs involved (including glibc) expect that the size
of the __rseq_abi can be the smallest possible subset, and only consider it
to be "extended" if specific information in the ABI tells them it is the case.

> 
> Which library provides the TLS symbol (and N bytes of storage) seems
> sensitive to the choices the linker makes for us, once the symbol
> sizes diverge.

AFAIU, a symbol defined in the main executable will have precedence over
a preloaded library, which has precedence over shared library dependencies,
e.g. glibc.

Thanks,

Mathieu

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com