Re: KASAN: use-after-free Write in irq_bypass_register_consumer

From: Dmitry Vyukov
Date: Sat May 26 2018 - 05:24:30 EST

Next message: Dmitry Vyukov: "Re: KASAN: stack-out-of-bounds Read in do_general_protection"
Previous message: Dmitry Vyukov: "Re: general protection fault in eventfd_ctx_remove_wait_queue"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Sun, May 13, 2018 at 8:21 AM, Eric Biggers <ebiggers3@xxxxxxxxx> wrote:
> On Thu, Apr 05, 2018 at 08:15:24PM -0700, Eric Biggers wrote:
>> On Mon, Jan 29, 2018 at 01:29:48PM +0800, Tianyu Lan wrote:
>> >
>> >
>> > On 1/27/2018 7:27 AM, Eric Biggers wrote:
>> > > On Sat, Dec 16, 2017 at 04:37:02PM +0800, Lan, Tianyu wrote:
>> > > > The root cause is that kvm_irqfd_assign() and kvm_irqfd_deassign() can't
>> > > > be run in parallel. Some data structure(e.g, irqfd->consumer) will be
>> > > > crashed because irqfd may be freed in deassign path before they are used
>> > > > in assign path. The other data maybe used in deassign path before
>> > > > initialization. Syzbot test hit such case. Add mutx between
>> > > > kvm_irqfd_assign() and kvm_irqfd_deassign() can fix such issue. Will
>> > > > send patch to fix it.
>> > > >
>> > > > On 12/16/2017 12:53 PM, Tianyu Lan wrote:
>> > > > > I reproduced the issue. Will have a look.
>> > > > >
>> > > > > -- Best regards Tianyu Lan 2017-12-15 18:14 GMT+08:00 syzbot
>> > > > > <bot+7ad82526b98c64c90062275162dacfa48c20dbd3@xxxxxxxxxxxxxxxxxxxxxxxxx>:
>> > > > > > syzkaller has found reproducer for the following crash on
>> > > > > > 82bcf1def3b5f1251177ad47c44f7e17af039b4b
>> > > > > > git://git.cmpxchg.org/linux-mmots.git/master
>> > > > > > compiler: gcc (GCC) 7.1.1 20170620
>> > > > > > .config is attached
>> > > > > > Raw console output is attached.
>> > > > > > C reproducer is attached
>> > > > > > syzkaller reproducer is attached. Seehttps://goo.gl/kgGztJ
>> > > > > > for information about syzkaller reproducers
>> > > > > >
>> > > > > >
>> > > > > > ==================================================================
>> > > > > > BUG: KASAN: use-after-free in __list_add include/linux/list.h:64 [inline]
>> > > > > > BUG: KASAN: use-after-free in list_add include/linux/list.h:79 [inline]
>> > > > > > BUG: KASAN: use-after-free in irq_bypass_register_consumer+0x4b4/0x500
>> > > > > > virt/lib/irqbypass.c:217
>> > > > > > Write of size 8 at addr ffff8801cdf51180 by task syzkaller436086/15031
>> > > > > >
>> > > > > > CPU: 1 PID: 15031 Comm: syzkaller436086 Not tainted 4.15.0-rc2-mm1+ #39
>> > > > > > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
>> > > > > > Google 01/01/2011
>> > > > > > Call Trace:
>> > > > > > __dump_stack lib/dump_stack.c:17 [inline]
>> > > > > > dump_stack+0x194/0x257 lib/dump_stack.c:53
>> > > > > > print_address_description+0x73/0x250 mm/kasan/report.c:252
>> > > > > > kasan_report_error mm/kasan/report.c:351 [inline]
>> > > > > > kasan_report+0x25b/0x340 mm/kasan/report.c:409
>> > > > > > __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:435
>> > > > > > __list_add include/linux/list.h:64 [inline]
>> > > > > > list_add include/linux/list.h:79 [inline]
>> > > > > > irq_bypass_register_consumer+0x4b4/0x500 virt/lib/irqbypass.c:217
>> > > > > > kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:417 [inline]
>> > > > > > kvm_irqfd+0x137f/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572
>> > > > > > kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2992
>> > > > > > vfs_ioctl fs/ioctl.c:46 [inline]
>> > > > > > do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:686
>> > > > > > SYSC_ioctl fs/ioctl.c:701 [inline]
>> > > > > > SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
>> > > > > > entry_SYSCALL_64_fastpath+0x1f/0x96
>> > > > > > RIP: 0033:0x44d379
>> > > > > > RSP: 002b:00007fc5ff9a9d08 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
>> > > > > > RAX: ffffffffffffffda RBX: 00007fc5ff9aa700 RCX: 000000000044d379
>> > > > > > RDX: 0000000020080fe0 RSI: 000000004020ae76 RDI: 0000000000000005
>> > > > > > RBP: 00000000007ff900 R08: 00007fc5ff9aa700 R09: 00007fc5ff9aa700
>> > > > > > R10: 00007fc5ff9aa700 R11: 0000000000000246 R12: 0000000000000000
>> > > > > > R13: 00000000007ff8ff R14: 00007fc5ff9aa9c0 R15: 0000000000000000
>> > > > > >
>> > > > > > Allocated by task 15031:
>> > > > > > save_stack+0x43/0xd0 mm/kasan/kasan.c:447
>> > > > > > set_track mm/kasan/kasan.c:459 [inline]
>> > > > > > kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:551
>> > > > > > kmem_cache_alloc_trace+0x136/0x750 mm/slab.c:3614
>> > > > > > kmalloc include/linux/slab.h:516 [inline]
>> > > > > > kzalloc include/linux/slab.h:705 [inline]
>> > > > > > kvm_irqfd_assign arch/x86/kvm/../../../virt/kvm/eventfd.c:296 [inline]
>> > > > > > kvm_irqfd+0x16c/0x1d50 arch/x86/kvm/../../../virt/kvm/eventfd.c:572
>> > > > > > kvm_vm_ioctl+0x1079/0x1c40 arch/x86/kvm/../../../virt/kvm/kvm_main.c:2992
>> > > > > > vfs_ioctl fs/ioctl.c:46 [inline]
>> > > > > > do_vfs_ioctl+0x1b1/0x1530 fs/ioctl.c:686
>> > > > > > SYSC_ioctl fs/ioctl.c:701 [inline]
>> > > > > > SyS_ioctl+0x8f/0xc0 fs/ioctl.c:692
>> > > > > > entry_SYSCALL_64_fastpath+0x1f/0x96
>> > > > > >
>> > > > > > Freed by task 1402:
>> > > > > > save_stack+0x43/0xd0 mm/kasan/kasan.c:447
>> > > > > > set_track mm/kasan/kasan.c:459 [inline]
>> > > > > > kasan_slab_free+0x71/0xc0 mm/kasan/kasan.c:524
>> > > > > > __cache_free mm/slab.c:3492 [inline]
>> > > > > > kfree+0xca/0x250 mm/slab.c:3807
>> > > > > > irqfd_shutdown+0x13c/0x1a0 arch/x86/kvm/../../../virt/kvm/eventfd.c:148
>> > > > > > process_one_work+0xbfd/0x1bc0 kernel/workqueue.c:2113
>> > > > > > worker_thread+0x223/0x1990 kernel/workqueue.c:2247
>> > > > > > kthread+0x37a/0x440 kernel/kthread.c:238
>> > > > > > ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:524
>> > > > > >
>> > > > > > The buggy address belongs to the object at ffff8801cdf51000
>> > > > > > which belongs to the cache kmalloc-512 of size 512
>> > > > > > The buggy address is located 384 bytes inside of
>> > > > > > 512-byte region [ffff8801cdf51000, ffff8801cdf51200)
>> > > > > > The buggy address belongs to the page:
>> > > > > > page:00000000d08a0d63 count:1 mapcount:0 mapping:00000000d54c7be6 index:0x0
>> > > > > > flags: 0x2fffc0000000100(slab)
>> > > > > > raw: 02fffc0000000100 ffff8801cdf51000 0000000000000000 0000000100000006
>> > > > > > raw: ffffea00073a7660 ffffea000737f3a0 ffff8801dac00940 0000000000000000
>> > > > > > page dumped because: kasan: bad access detected
>> > > > > >
>> > > > > > Memory state around the buggy address:
>> > > > > > ffff8801cdf51080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>> > > > > > ffff8801cdf51100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>> > > > > > > ffff8801cdf51180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>> > > > > > ^
>> > > > > > ffff8801cdf51200: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>> > > > > > ffff8801cdf51280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>> > > > > > ==================================================================
>> > > > > >
>> > > Tianyu, are you still planning to send a patch for this? This is still
>> > > happening.
>> > >
>> > > - Eric
>> > Yes, I sent out fix patch and wait for review.
>> > https://patchwork.kernel.org/patch/10128925/
>> >
>>
>> This is still happening. Can you please resend your patch?
>>
>
> Still happening; it's pretty easily reproducible, using the program below.
> (Sometimes it crashes in different ways too. It also spams the log, ignore
> that.) Tianyu's patch at https://patchwork.kernel.org/patch/10128925/ fixes it,
> though the locking is difficult to understand so I'm not sure it's correct yet.
> Probably the comment that says "do not drop the file until the irqfd is fully
> initialized, otherwise we might race against the EPOLLHUP" would be obsoleted
> and should be removed? Can the KVM maintainers please review the patch too?
>
> #include <fcntl.h>
> #include <linux/kvm.h>
> #include <pthread.h>
> #include <sys/eventfd.h>
> #include <sys/ioctl.h>
>
> static int dev, vm, event;
>
> static void *irqfd_thrproc(void *_flags)
> {
> struct kvm_irqfd irqfd = { .fd = event, .flags = (long)_flags };
>
> for (;;)
> ioctl(vm, KVM_IRQFD, &irqfd);
> }
>
> int main()
> {
> pthread_t t;
>
> event = eventfd(0, 0);
> dev = open("/dev/kvm", 0);
> vm = ioctl(dev, KVM_CREATE_VM, 0);
> ioctl(vm, KVM_CREATE_IRQCHIP, 0);
> pthread_create(&t, NULL, irqfd_thrproc, (void *)(long)0);
> irqfd_thrproc((void *)(long)KVM_IRQFD_FLAG_DEASSIGN);
> }

Tentative duplicates of this bug:

BUG: unable to handle kernel NULL pointer dereference in
irq_bypass_register_consumer
KASAN: use-after-free Read in irq_bypass_unregister_consumer
general protection fault in eventfd_ctx_remove_wait_queue
KASAN: use-after-free Read in irq_bypass_register_consumer

https://syzkaller.appspot.com/bug?id=45591ae3053c59fb50169401fb61cb596735f9d1

Next message: Dmitry Vyukov: "Re: KASAN: stack-out-of-bounds Read in do_general_protection"
Previous message: Dmitry Vyukov: "Re: general protection fault in eventfd_ctx_remove_wait_queue"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]