Re: [RFC PATCH 0/3] restartable sequences v2: fast user-space percpu critical sections

[Andy Lutomirski]

On Thu, Apr 7, 2016 at 8:24 AM, Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
> On Thu, Apr 07, 2016 at 07:35:26AM -0700, Andy Lutomirski wrote:
>> What I meant was: rather than shoving individual values into the TLABI
>> thing, shove in a pointer:
>>
>> struct commit_info {
>>   u64 post_commit_rip;
>>   u32 cpu;
>>   u64 *event;
>>   // whatever else;
>> };
>>
>> and then put a commit_info* in TLABI.
>>
>> This would save some bytes in the TLABI structure.
>
> But would cost us extra indirections. The whole point was getting this
> stuff at a constant offset from the TLS segment register.

I don't think the extra indirections would matter much.  The kernel
would have to chase the pointer, but only in the very rare case where
it resumes userspace during a commit or on the immediately following
instruction.

At the very least, post_commit_rip and the abort address (which I
forgot about) could both live in a static structure, and shoving a
pointer to *that* into TLABI space is one store instead of two.

Admittedly, this is just a minor tweak either way.  I'm just wondering
an extra indirect variant ends up being slightly faster.

>
>> > So letting the user manage the event is doable, but it would still be
>> > advisable to have the event in the same shared word.
>> >
>>
>> Why is a single load needed?  The event and CPU in the TLABI structure
>> are only ever accessed from the thread in question.
>
> Its not required, but being able to do a single load saves on cycles,
> less cycles is more good.
>
>> That way we could take an async signal, handle it, and resume, even in
>> the middle of a commit, without aborting.  Of course, if the signal
>> hander tried to access the same rseq-protected resource, it would bump
>> the event counter and cause an abort.
>
> Ah, so what happens if the signal happens before the commit but after
> the load of the seqcount?
>
> Then, even if the signal motifies the count, we'll not observe.
>

Where exactly?

In my scheme, nothing except the kernel ever loads the seqcount.  The
user code generates a fresh value, writes it to memory, and then, just
before commit, writes that same value to the TLABI area and then
double-checks that the value it wrote at the beginning is still there.

If the signal modifies the count, then the user code won't directly
notice, but prepare_exit_to_usermode on the way out of the signal will
notice that the (restored) TLABI state doesn't match the counter that
the signal handler changed and will just to the abort address.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC