Re: kvm: GPF in irqfd_shutdown/eventfd_ctx_remove_wait_queue

From: Dmitry Vyukov
Date: Fri Nov 18 2016 - 03:17:34 EST

Next message: Ingo Molnar: "Re: [PATCH 0/5] x86: remove idle notifier"
Previous message: Ingo Molnar: "Re: [PATCH v12 2/7] x86/arch_prctl/64: Rename do_arch_prctl to do_arch_prctl_64"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Sat, Nov 12, 2016 at 10:48 PM, Dmitry Vyukov <dvyukov@xxxxxxxxxx> wrote:
> On Sat, Nov 12, 2016 at 1:27 PM, Dmitry Vyukov <dvyukov@xxxxxxxxxx> wrote:
>> Hello,
>>
>> I've got the following crash while running syzkaller fuzzer.
>> On commit 015ed9433be2b476ec7e2e6a9a411a56e3b5b035 (Nov 11).
>> Unfortunately it is not reproducible.
>> May be related to:
>> https://groups.google.com/forum/#!topic/syzkaller/NKlClJzOOww
>> https://groups.google.com/forum/#!topic/syzkaller/Dz__GySpVr8
>>
>> general protection fault: 0000 [#1] SMP KASAN
>> Dumping ftrace buffer:
>> (ftrace buffer empty)
>> Modules linked in:
>> CPU: 0 PID: 17194 Comm: kworker/0:2 Not tainted 4.9.0-rc4+ #49
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Workqueue: kvm-irqfd-cleanup irqfd_shutdown
>> task: ffff88003b2bdb40 task.stack: ffff88003bcd8000
>> RIP: 0010:[<ffffffff8175b8e9>] [< inline >] __list_del
>> include/linux/list.h:89
>> RIP: 0010:[<ffffffff8175b8e9>] [< inline >] list_del
>> include/linux/list.h:107
>> RIP: 0010:[<ffffffff8175b8e9>] [< inline >]
>> __remove_wait_queue include/linux/wait.h:196
>> RIP: 0010:[<ffffffff8175b8e9>] [<ffffffff8175b8e9>]
>> eventfd_ctx_remove_wait_queue+0x139/0x2d0 fs/eventfd.c:201
>> RSP: 0018:ffff88003bcdfb50 EFLAGS: 00010002
>> RAX: 0000000000000000 RBX: ffff88003cefbd40 RCX: dffffc0000000000
>> RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000008
>> RBP: ffff88003bcdfb90 R08: 0000000090a58b45 R09: 0000000000000001
>> R10: ffffffff84da2600 R11: 1ffff1000779bf36 R12: ffff88003b986b88
>> R13: ffff88003bcdfbc8 R14: ffff88003cefbd48 R15: 0000000000000081
>> FS: 0000000000000000(0000) GS:ffff88003ec00000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> CR2: 0000000000813000 CR3: 0000000065fd4000 CR4: 00000000000026f0
>> Stack:
>> ffff88003bcdfb60 ffff88003cefbd90 0000000000000296 ffff88003b986b80
>> ffff88003b986ca8 1ffff1000779bf75 ffff88003b986c80 dffffc0000000000
>> ffff88003bcdfc30 ffffffff8106dd86 0000000000000000 0000000041b58ab3
>> Call Trace:
>> [<ffffffff8106dd86>] irqfd_shutdown+0x96/0x1a0
>> arch/x86/kvm/../../../virt/kvm/eventfd.c:128
>> [<ffffffff8129175c>] process_one_work+0x9fc/0x1900 kernel/workqueue.c:2096
>> [<ffffffff8129274f>] worker_thread+0xef/0x1480 kernel/workqueue.c:2230
>> [<ffffffff812a5a94>] kthread+0x244/0x2d0 kernel/kthread.c:209
>> [<ffffffff831f102a>] ret_from_fork+0x2a/0x40 arch/x86/entry/entry_64.S:433
>> Code: 48 89 f9 48 c1 e9 03 80 3c 01 00 0f 85 83 01 00 00 48 8d 7a 08
>> 48 b9 00 00 00 00 00 fc ff df 49 8b 44 24 20 48 89 fe 48 c1 ee 03 <80>
>> 3c 0e 00 0f 85 45 01 00 00 48 89 c6 48 b9 00 00 00 00 00 fc
>> RIP [< inline >] __list_del include/linux/list.h:89
>> RIP [< inline >] list_del include/linux/list.h:107
>> RIP [< inline >] __remove_wait_queue include/linux/wait.h:196
>> RIP [<ffffffff8175b8e9>] eventfd_ctx_remove_wait_queue+0x139/0x2d0
>> fs/eventfd.c:201
>> RSP <ffff88003bcdfb50>
>> ---[ end trace 9772f974e210aab6 ]---
>> Kernel panic - not syncing: Fatal exception
>> Shutting down cpus with NMI
>> Dumping ftrace buffer:
>> (ftrace buffer empty)
>> Kernel Offset: disabled
>> reboot: cpu_has_vmx: ecx=80a02021 1
>
>
> Another use-after-free that looks relevant:
>
> BUG: KASAN: use-after-free in insert_work+0x24a/0x2e0 at addr ffff88003d87bcd8
> Read of size 8 by task syz-executor/28580
> CPU: 0 PID: 28580 Comm: syz-executor Not tainted 4.9.0-rc4+ #49
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> ffff88003c60f920 ffffffff81c2e46b ffff88003e80ccc0 ffff88003d87bc38
> ffff88003d87bf70 0000000000000000 ffff88003c60f948 ffffffff8165ab9c
> ffffed0007b0f79b ffffed0007b0f79b ffff88003e80ccc0 ffff88003c60f9c8
> Call Trace:
> [< inline >] __dump_stack lib/dump_stack.c:15
> [<ffffffff81c2e46b>] dump_stack+0xb3/0x118 lib/dump_stack.c:51
> [<ffffffff8165ab9c>] kasan_object_err+0x1c/0x70 mm/kasan/report.c:156
> [< inline >] print_address_description mm/kasan/report.c:194
> [< inline >] kasan_report_error mm/kasan/report.c:283
> [<ffffffff8165aed1>] kasan_report+0x231/0x500 mm/kasan/report.c:303
> [<ffffffff8165b214>] __asan_report_load8_noabort+0x14/0x20
> mm/kasan/report.c:329
> [< inline >] constant_test_bit arch/x86/include/asm/bitops.h:311
> [< inline >] set_work_data kernel/workqueue.c:614
> [< inline >] set_work_pwq kernel/workqueue.c:621
> [<ffffffff8128ca1a>] insert_work+0x24a/0x2e0 kernel/workqueue.c:1297
> [<ffffffff8128cfa1>] __queue_work+0x4f1/0xed0 kernel/workqueue.c:1459
> [<ffffffff8128e7f7>] queue_work_on+0x97/0xa0 kernel/workqueue.c:1484
> [< inline >] queue_work include/linux/workqueue.h:474
> [< inline >] schedule_work include/linux/workqueue.h:532
> [< inline >] kvm_irqfd_assign
> arch/x86/kvm/../../../virt/kvm/eventfd.c:403
> [<ffffffff8106f18f>] kvm_irqfd+0x117f/0x18a0
> arch/x86/kvm/../../../virt/kvm/eventfd.c:572
> [<ffffffff81065fe7>] kvm_vm_ioctl+0x2e7/0x1670
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2996
> [< inline >] vfs_ioctl fs/ioctl.c:43
> [<ffffffff816b03cc>] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
> [< inline >] SYSC_ioctl fs/ioctl.c:694
> [<ffffffff816b130f>] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
> [<ffffffff831f0dc1>] entry_SYSCALL_64_fastpath+0x1f/0xc2
> Object at ffff88003d87bc38, in cache kmalloc-512 size: 512
> Allocated:
> PID = 28580
> [ 956.676797] [<ffffffff811abb36>] save_stack_trace+0x16/0x20
> arch/x86/kernel/stacktrace.c:57
> [ 956.676797] [<ffffffff81659ee6>] save_stack+0x46/0xd0 mm/kasan/kasan.c:495
> [ 956.676797] [< inline >] set_track mm/kasan/kasan.c:507
> [ 956.676797] [<ffffffff8165a15d>] kasan_kmalloc+0xad/0xe0
> mm/kasan/kasan.c:598
> [ 956.676797] [<ffffffff816557f8>] kmem_cache_alloc_trace+0xf8/0x280
> mm/slub.c:2735
> [ 956.676797] [< inline >] kmalloc include/linux/slab.h:490
> [ 956.676797] [< inline >] kzalloc include/linux/slab.h:636
> [ 956.676797] [< inline >] kvm_irqfd_assign
> arch/x86/kvm/../../../virt/kvm/eventfd.c:296
> [ 956.676797] [<ffffffff8106e0b7>] kvm_irqfd+0xa7/0x18a0
> arch/x86/kvm/../../../virt/kvm/eventfd.c:572
> [ 956.774974] [<ffffffff81065fe7>] kvm_vm_ioctl+0x2e7/0x1670
> arch/x86/kvm/../../../virt/kvm/kvm_main.c:2996
> [ 956.779053] [< inline >] vfs_ioctl fs/ioctl.c:43
> [ 956.779053] [<ffffffff816b03cc>] do_vfs_ioctl+0x18c/0x1040 fs/ioctl.c:679
> [ 956.779053] [< inline >] SYSC_ioctl fs/ioctl.c:694
> [ 956.779053] [<ffffffff816b130f>] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:685
> [ 956.779053] [<ffffffff831f0dc1>] entry_SYSCALL_64_fastpath+0x1f/0xc2
> Freed:
> PID = 12240
> [ 956.779053] [<ffffffff811abb36>] save_stack_trace+0x16/0x20
> arch/x86/kernel/stacktrace.c:57
> [ 956.779053] [<ffffffff81659ee6>] save_stack+0x46/0xd0 mm/kasan/kasan.c:495
> [ 956.779053] [< inline >] set_track mm/kasan/kasan.c:507
> [ 956.779053] [<ffffffff8165a741>] kasan_slab_free+0x71/0xb0
> mm/kasan/kasan.c:571
> [ 956.779053] [< inline >] slab_free_hook mm/slub.c:1352
> [ 956.779053] [< inline >] slab_free_freelist_hook mm/slub.c:1374
> [ 956.779053] [< inline >] slab_free mm/slub.c:2951
> [ 956.779053] [<ffffffff8165696a>] kfree+0xea/0x2c0 mm/slub.c:3871
> [ 956.807057] [<ffffffff8106de2d>] irqfd_shutdown+0x13d/0x1a0
> arch/x86/kvm/../../../virt/kvm/eventfd.c:148
> [ 956.807057] [<ffffffff8129175c>] process_one_work+0x9fc/0x1900
> kernel/workqueue.c:2096
> [ 956.807057] [<ffffffff8129274f>] worker_thread+0xef/0x1480
> kernel/workqueue.c:2230
> [ 956.807057] [<ffffffff812a5a94>] kthread+0x244/0x2d0 kernel/kthread.c:209
> [ 956.807057] [<ffffffff831f102a>] ret_from_fork+0x2a/0x40
> arch/x86/entry/entry_64.S:433

Another one:

BUG: KASAN: use-after-free in kvm_irqfd+0x188e/0x1940 at addr ffff880060af9100
Read of size 8 by task syz-executor/17548
CPU: 3 PID: 17548 Comm: syz-executor Not tainted 4.9.0-rc5+ #43
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
ffff88005f9ef5f0 ffffffff834c2a19 ffffffff00000003 1ffff1000bf3de51
ffffed000bf3de49 0000000041b58ab3 ffffffff895758b0 ffffffff834c272b
ffffffff894cff00 ffffffff81565d50 ffffffff894cff00 ffffffff81565d50
Call Trace:
[< inline >] __dump_stack lib/dump_stack.c:15
[<ffffffff834c2a19>] dump_stack+0x2ee/0x3f5 lib/dump_stack.c:51
[<ffffffff819f09b1>] kasan_object_err+0x21/0x70 mm/kasan/report.c:159
[< inline >] print_address_description mm/kasan/report.c:197
[< inline >] kasan_report_error mm/kasan/report.c:286
[<ffffffff819f0c9b>] kasan_report+0x1eb/0x4c0 mm/kasan/report.c:306
[<ffffffff819f0fe9>] __asan_report_load8_noabort+0x19/0x20
mm/kasan/report.c:332
[< inline >] kvm_irqfd_assign
arch/x86/kvm/../../../virt/kvm/eventfd.c:412
[<ffffffff8108ee7e>] kvm_irqfd+0x188e/0x1940
arch/x86/kvm/../../../virt/kvm/eventfd.c:572
[<ffffffff810839aa>] kvm_vm_ioctl+0x105a/0x1a70
arch/x86/kvm/../../../virt/kvm/kvm_main.c:2998
[< inline >] vfs_ioctl fs/ioctl.c:43
[<ffffffff81abcc04>] do_vfs_ioctl+0x1c4/0x1630 fs/ioctl.c:679
[< inline >] SYSC_ioctl fs/ioctl.c:694
[<ffffffff81abe104>] SyS_ioctl+0x94/0xc0 fs/ioctl.c:685
[<ffffffff88147905>] entry_SYSCALL_64_fastpath+0x23/0xc6
Object at ffff880060af9000, in cache kmalloc-512 size: 512
Allocated:
PID = 17548
[ 514.149774] [<ffffffff8127101b>] save_stack_trace+0x1b/0x20
arch/x86/kernel/stacktrace.c:57
[ 514.149774] [<ffffffff819efce3>] save_stack+0x43/0xd0 mm/kasan/kasan.c:495
[ 514.149774] [< inline >] set_track mm/kasan/kasan.c:507
[ 514.149774] [<ffffffff819eff6d>] kasan_kmalloc+0xad/0xe0
mm/kasan/kasan.c:598
[ 514.149774] [<ffffffff819ec32c>]
kmem_cache_alloc_trace+0x12c/0x710 mm/slab.c:3635
[ 514.149774] [< inline >] kmalloc include/linux/slab.h:490
[ 514.149774] [< inline >] kzalloc include/linux/slab.h:636
[ 514.149774] [< inline >] kvm_irqfd_assign
arch/x86/kvm/../../../virt/kvm/eventfd.c:296
[ 514.149774] [<ffffffff8108d737>] kvm_irqfd+0x147/0x1940
arch/x86/kvm/../../../virt/kvm/eventfd.c:572
[ 514.149774] [<ffffffff810839aa>] kvm_vm_ioctl+0x105a/0x1a70
arch/x86/kvm/../../../virt/kvm/kvm_main.c:2998
[ 514.149774] [< inline >] vfs_ioctl fs/ioctl.c:43
[ 514.149774] [<ffffffff81abcc04>] do_vfs_ioctl+0x1c4/0x1630 fs/ioctl.c:679
[ 514.149774] [< inline >] SYSC_ioctl fs/ioctl.c:694
[ 514.149774] [<ffffffff81abe104>] SyS_ioctl+0x94/0xc0 fs/ioctl.c:685
[ 514.149774] [<ffffffff88147905>] entry_SYSCALL_64_fastpath+0x23/0xc6
Freed:
PID = 4
[ 514.149774] [<ffffffff8127101b>] save_stack_trace+0x1b/0x20
arch/x86/kernel/stacktrace.c:57
[ 514.149774] [<ffffffff819efce3>] save_stack+0x43/0xd0 mm/kasan/kasan.c:495
[ 514.149774] [< inline >] set_track mm/kasan/kasan.c:507
[ 514.149774] [<ffffffff819f05c2>] kasan_slab_free+0x72/0xc0
mm/kasan/kasan.c:571
[ 514.149774] [< inline >] __cache_free mm/slab.c:3511
[ 514.149774] [<ffffffff819ee228>] kfree+0xc8/0x2a0 mm/slab.c:3828
[ 514.149774] [<ffffffff8108d2c5>] irqfd_shutdown+0x155/0x1d0
arch/x86/kvm/../../../virt/kvm/eventfd.c:148
[ 514.149774] [<ffffffff81473050>] process_one_work+0xb40/0x1ba0
kernel/workqueue.c:2096
[ 514.149774] [<ffffffff814742c4>] worker_thread+0x214/0x18a0
kernel/workqueue.c:2230
[ 514.149774] [<ffffffff81489ee8>] kthread+0x328/0x3e0 kernel/kthread.c:209
[ 514.149774] [<ffffffff88147b9a>] ret_from_fork+0x2a/0x40
arch/x86/entry/entry_64.S:433
Memory state around the buggy address:
ffff880060af9000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff880060af9080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>ffff880060af9100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
^
ffff880060af9180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
ffff880060af9200: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
==================================================================

Next message: Ingo Molnar: "Re: [PATCH 0/5] x86: remove idle notifier"
Previous message: Ingo Molnar: "Re: [PATCH v12 2/7] x86/arch_prctl/64: Rename do_arch_prctl to do_arch_prctl_64"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]