Re: NULL ptr deref in perf/filter_match

[Vegard Nossum]

On 27 July 2016 at 16:15, Vegard Nossum <vegard.nossum@xxxxxxxxx> wrote:
> Hi,
>
> I'm seeing this on latest linus/master:
>
> kasan: CONFIG_KASAN_INLINE enabled
> kasan: GPF could be caused by NULL-ptr deref or user memory access
> general protection fault: 0000 [#1] PREEMPT SMP KASAN
> Dumping ftrace buffer:
>   (ftrace buffer empty)
> CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.7.0+ #50
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
> Ubuntu-1.8.2-1ubuntu1 04/01/2014
> task: ffff880119d05400 ti: ffff880119d38000 task.ti: ffff880119d38000
> RIP: 0010:[<ffffffff81327820>]  [<ffffffff81327820>] perf_iterate_sb+0x1b0/0x6a0
> RSP: 0018:ffff880119d3fc30  EFLAGS: 00010046
> RAX: 0000000000000000 RBX: ffff880080af8530 RCX: 0000000000000000
> RDX: 1ffff100235f3465 RSI: ffffffff8376a900 RDI: ffff880080af8730
> RBP: ffff880119d3fc70 R08: 0000000000000000 R09: 0000000000000000
> R10: ffff8800abbfe200 R11: 0000000000000000 R12: ffffffff8131b8e0
> R13: ffff880119d3fcf0 R14: dffffc0000000000 R15: 0000000000000000
> FS:  0000000000000000(0000) GS:ffff88011af80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007fad10e87b10 CR3: 00000000a89d5000 CR4: 00000000000006e0
> DR0: 00007fad1114b000 DR1: 00007fad0f4a7000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600
> Stack:
> ffff88011af9f580 ffff880119d05400 ffff88011af9a328 ffff880119d05400
> ffff880119d3fd30 0000000000000003 1ffff100233a7f9a ffff8800abbfe200
> ffff880119d3fd58 ffffffff81334670 0000000041b58ab3 ffffffff83bc294e
> Call Trace:
> [<ffffffff81334670>] __perf_event_task_sched_out+0x2a0/0xec0
> [<ffffffff813343d0>] ? perf_event_update_userpage+0x630/0x630
> [<ffffffff8115f2bd>] ? finish_task_switch+0x12d/0x580
> [<ffffffff8351a881>] __schedule+0x9a1/0x16c0
> [<ffffffff83519ee0>] ? pci_mmcfg_check_reserved+0x110/0x110
> [<ffffffff81058e37>] ? dump_trace+0x117/0x300
> [<ffffffff810771a6>] ? save_stack_trace+0x26/0x50
> [<ffffffff8351b86a>] schedule+0x9a/0x1c0
> [<ffffffff8351b9e3>] schedule_preempt_disabled+0x13/0x20
> [<ffffffff811c35fd>] cpu_startup_entry+0x1cd/0x5a0
> [<ffffffff83525d7f>] ? _raw_spin_unlock_irqrestore+0x1f/0x40
> [<ffffffff810a76e7>] start_secondary+0x247/0x2d0
> Code: 5f ff ff ff 48 8d bb 00 02 00 00 48 89 f8 48 c1 e8 03 42 80 3c
> 30 00 0f 85 57 04 00 00 4c 8b bb 00 02 00 00 4c 89 f8 48 c1 e8 03 <42>
> 80 3c 30 00 0f 85 31 04 00 00 4d 8b 3f 49 8d 7f 40 48 89 f8
> RIP  [<ffffffff81327820>] perf_iterate_sb+0x1b0/0x6a0
> RSP <ffff880119d3fc30>
> ---[ end trace fc2135c1ac1bf1e9 ]---
>
> That seems to be roughly:
>
> kernel/events/core.c:145
> kernel/events/core.c:547 perf_cgroup_match
> kernel/events/core.c:1720 event_filter_match
> kernel/events/core.c:5950 perf_iterate_sb_cpu
> kernel/events/core.c:5982 perf_iterate_sb
> kernel/events/core.c:6794 perf_event_switch
> kernel/events/core.c:2857 __perf_event_task_sched_out
>
> In particular, it looks to me like event->ctx is NULL.

Digging a bit deeper into this, it seems the event itself is getting
created by perf_event_open() and it gets added to the pmu_event_list
through:

perf_event_open()
 - perf_event_alloc()
    - account_event()
       - account_pmu_sb_event()
          - attach_sb_event()

so at this point the event is being attached but its ->ctx is still
NULL. It seems like ->ctx is set just a bit later in
perf_event_open(), though.

But before that, __schedule() comes along and creates a stack trace
similar to the one above:

__schedule()
 - __perf_event_task_sched_out()
   - perf_iterate_sb()
     - perf_iterate_sb_cpu()
        - event_filter_match()
          - perf_cgroup_match()
            - __get_cpu_context()
              - (dereference ctx which is NULL)

So I guess the question is... should the event be attached (= put on
the list) before ->ctx gets set? Or should the cgroup code check for a
NULL ->ctx?

I'm seeing the NUL ptr deref in __perf_event_task_sched_in() as well, btw.

I'm thinking this is probably where the bug was introduced:

commit f2fb6bef92514432398a653df1c2f1041d79ac46
Author: Kan Liang <kan.liang@xxxxxxxxx>
Date:   Wed Mar 23 11:24:37 2016 -0700

   perf/core: Optimize side-band event delivery


Vegard