Re: [RFC PATCH v3 for 4.15 08/24] Provide cpu_opv system call

[Mathieu Desnoyers]

----- On Nov 17, 2017, at 3:22 PM, Thomas Gleixner tglx@xxxxxxxxxxxxx wrote:

> On Fri, 17 Nov 2017, Mathieu Desnoyers wrote:
>> ----- On Nov 17, 2017, at 5:09 AM, Thomas Gleixner tglx@xxxxxxxxxxxxx wrote:
>> 7) Allow libraries with multi-part algorithms to work on same per-cpu
>>    data without affecting the allowed cpu mask
>> 
>> I stumbled on an interesting use-case within the lttng-ust tracer
>> per-cpu buffers: the algorithm needs to update a "reserve" counter,
>> serialize data into the buffer, and then update a "commit" counter
>> _on the same per-cpu buffer_. My goal is to use rseq for both reserve
>> and commit.
>> 
>> Clearly, if rseq reserve fails, the algorithm can retry on a different
>> per-cpu buffer. However, it's not that easy for the commit. It needs to
>> be performed on the same per-cpu buffer as the reserve.
>> 
>> The cpu_opv system call solves that problem by receiving the cpu number
>> on which the operation needs to be performed as argument. It can push
>> the task to the right CPU if needed, and perform the operations there
>> with preemption disabled.
> 
> If your transaction cannot be done in one go, then abusing that byte code
> interpreter for concluding it is just hillarious. That whole exercise is a
> gazillion times slower than the atomic operations which are neccesary to do
> it without all that.
> 
> I'm even more convinced now that this is overengineered beyond repair.

The fast-path (typical case) will execute on the right CPU, and rseq will
do both the reserve and the commit, and that is faster than atomic ops.

However, we need to handle migration between reserve and commit.
Unfortunately, concurrent rseq and atomic ops don't mix well on the
same per-cpu data, so we cannot fall-back on atomic ops, except in
very specific cases where we can use a split-counter strategy.

So cpu_opv handles migration for this use-case by ensuring
the slow-path is performed with preemption-off after migrating
the current task to the right CPU.

Thanks,

Mathieu


> 
> Thanks,
> 
> 	tglx

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com