Re: [RFC v2 5/6] x86: learning and patching indirect branch targets

[Nadav Amit]

> On Dec 31, 2018, at 12:05 PM, Andy Lutomirski <luto@xxxxxxxxxx> wrote:
> 
> On Sun, Dec 30, 2018 at 11:20 PM Nadav Amit <namit@xxxxxxxxxx> wrote:
>> During runtime, we collect the targets of indirect branch targets and
>> patch them in. Patching is done asynchronously, by modifying each of the
>> relpoline code-paths separately while diverting code execution to the
>> other path during patching. Preemption is disabled while the code runs,
>> and we wait for preemption to occur on each core to ensure no core is
>> executing the patched code.
>> 
>> To make use of relpolines, a worker goes over the experienced indirect
>> calls targets and sorts them according to frequency. The target that
>> was encountered most times is patched in.
>> 
>> Periodically, the indirect branches are set back into learning mode to
>> see whether the targets have changed. The current policy might be too
>> aggressive.
> 
> Can you put, in a comment somewhere, a clear description of the actual
> optpoline assembly sequence?  I'm finding this code very hard to
> follow as is.  Something like:
> 
> /*
> * An optpoline is:
> *
> * cmp something, something else
> * je somewhere
> * [repeats of the above]
> * RETPOLINE (i.e. call some thunk)
> */
> 

Sure. I will add it. The GCC plugin code [3/6] holds commented assembly
code, but I will add it to the commit log as well.

> And please make it correct.
> 
> Your comment says that preemption is disabled, but it's not obvious to
> me where this happens.
> 
> Also, you define REX_B and don't use it.  Are there other cases of that?

Yes, I was sloppy. The preemption is not disabled, and instead I used your
proposed approach of restartable sequences.

REX_B is used as KERNEL_RESTARTABLE_PREFIX in [3/6], [5/6] and [6/6]. I will
rename it.