Re: [RFC PATCH 1/3] static_call: Add static call infrastructure

[Ard Biesheuvel]

On 9 November 2018 at 16:10, Josh Poimboeuf <jpoimboe@xxxxxxxxxx> wrote:
> On Fri, Nov 09, 2018 at 02:39:17PM +0100, Ard Biesheuvel wrote:
>> > +       for (site = start; site < stop; site++) {
>> > +               struct static_call_key *key = static_call_key(site);
>> > +               unsigned long addr = static_call_addr(site);
>> > +
>> > +               if (list_empty(&key->site_mods)) {
>> > +                       struct static_call_mod *mod;
>> > +
>> > +                       mod = kzalloc(sizeof(*mod), GFP_KERNEL);
>> > +                       if (!mod) {
>> > +                               WARN(1, "Failed to allocate memory for static calls");
>> > +                               return;
>> > +                       }
>> > +
>> > +                       mod->sites = site;
>> > +                       list_add_tail(&mod->list, &key->site_mods);
>> > +
>> > +                       /*
>> > +                        * The trampoline should no longer be used.  Poison it
>> > +                        * it with a BUG() to catch any stray callers.
>> > +                        */
>> > +                       arch_static_call_poison_tramp(addr);
>>
>> This patches the wrong thing: the trampoline is at key->func not addr.
>
> If you look at the x86 implementation, it actually does poison the
> trampoline.
>
> The address of the trampoline isn't actually known here.  key->func
> isn't the trampoline address; it's the destination func address.
>
> So instead I passed the address of the call instruction.  The arch code
> then reads the instruction to find the callee (the trampoline).
>
> The code is a bit confusing.  To make it more obvious, maybe we should
> add another arch function to read the call destination.  Then this code
> can pass that into arch_static_call_poison_tramp().
>

Ah right, so I am basically missing a dereference in my
arch_static_call_poison_tramp() code if this breaks.

>> However, patching it here means we poison it before all users are
>> patched. I added this on top
>>
>> diff --git a/kernel/static_call.c b/kernel/static_call.c
>> index 599ebc6fc4f1..d9562329bec6 100644
>> --- a/kernel/static_call.c
>> +++ b/kernel/static_call.c
>> @@ -248,6 +248,7 @@ static void __init static_call_init(void)
>>         struct static_call_site *start = __start_static_call_sites;
>>         struct static_call_site *stop  = __stop_static_call_sites;
>>         struct static_call_site *site;
>> +       struct static_call_key *prev_key = NULL;
>>
>>         if (start == stop) {
>>                 pr_warn("WARNING: empty static call table\n");
>> @@ -279,7 +280,9 @@ static void __init static_call_init(void)
>>                          * The trampoline should no longer be used.  Poison it
>>                          * it with a BUG() to catch any stray callers.
>>                          */
>> -                       arch_static_call_poison_tramp(addr);
>> +                       if (prev_key)
>> +
>> arch_static_call_poison_tramp((unsigned long)prev_key->func);
>> +                       prev_key = key;
>>                 }
>>
>>                 arch_static_call_transform(addr, key->func);
>
> While it does indeed poison the trampoline before all users are patched,
> I had been thinking that it didn't really matter because this is before
> the other CPUs have been booted.
>
> But I believe interrupts are enabled at this point during the boot, so
> it would indeed be wise to poison it afterwards, in case an irq handler
> makes a static call.
>

And kmalloc(GFP_KERNEL) itself could cascade into lots of other code as well.