Re: [PATCH] kprobes, x86/alternatives: use text_mutex to protect smp_alt_modules

[zhouchengming]

On 2017/10/27 19:15, Borislav Petkov wrote:
> On Fri, Oct 27, 2017 at 05:34:44PM +0800, Zhou Chengming wrote:
> > Fixes: 2cfa197 "ftrace/alternatives: Introducing *_text_reserved
> > functions"
> >
> > We use alternatives_text_reserved() to check if the address is in
> > the fixed pieces of alternative reserved, but the problem is that
> > we don't hold the smp_alt mutex when call this function. So the list
> > traversal may encounter a deleted list_head if another path is doing
> > alternatives_smp_module_del().
> Is this something you've triggered on a real machine or is this just
> from code staring?

Hi, thanks for your reply.
This is a real bug happened on one of our machines, below is the calltrace.
We can see the trigger is at alternatives_text_reserved+0x20/0x80, and
encounter a deleted (poisoned) list_head.

[   14.016190] general protection fault: 0000 [#1] SMP
[   14.016988] CPU 0
[   14.017287] Modules linked in: mlx4_ib(O+) mlx4_en(O+) ib_sa(O) ib_mad(O) ib_core(O) ib_addr(O) ipv6 mlx4_core(O) compat(O) igb(O) rtc_cmos dca button ata_piix ahci libahci libata ext3 jbd mbcache usbhid hid uhci_hcd ehci_hcd processor thermal_sys hwmon usbcore usb_common sd_mod crc_t10dif virtio_console(O) virtio_pci(O) virtio_net(O) kvm_ivshmem(O) virtio_scsi(O) scsi_mod virtio_blk(O) virtio(O) virtio_ring(O) pv_channel(O)
[   14.020005]
[   14.020005] Pid: 1979, comm: modprobe Tainted: G           O 3.4.24.19-0.11-default #1 QEMU Standard PC (i440FX + PIIX, 1996)
[   14.020005] RIP: 0010:[<ffffffff81007eb0>]  [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[   14.020005] RSP: 0018:ffff880ea355bcb8  EFLAGS: 00010283
[   14.020005] RAX: dead000000000100 RBX: ffffffffa02af720 RCX: dead0000000000d0
[   14.020005] RDX: ffffffffa02f0588 RSI: ffffffffa02d2fc0 RDI: ffffffffa02d2fc0
[   14.020005] RBP: ffff880ea355bcb8 R08: ffffffffa02f3b68 R09: 00017f4ae12d2fc0
[   14.020005] R10: 00000000000000e8 R11: ffffffffa02bb9d7 R12: 0000000000000000
[   14.020005] R13: ffffffffa02af720 R14: ffffffffa0307140 R15: ffffffffa02af730
[   14.020005] FS:  00007f26c6acc700(0000) GS:ffff880fff200000(0000) knlGS:0000000000000000
[   14.020005] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[   14.020005] CR2: 00007fd4adc3b000 CR3: 0000000ea40ea000 CR4: 00000000001407f0
[   14.020005] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   14.020005] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[   14.020005] Process modprobe (pid: 1979, threadinfo ffff880ea355a000, task ffff880e9eb9e600)
[   14.020005] Stack:
[   14.020005]  ffff880ea355bcd8 ffffffff8145b148 ffffffffa02af720 ffffffffa02af720
[   14.020005]  ffff880ea355bd18 ffffffff8145ed38 0000000000000000 0000000000000000
[   14.041015]  ffff880ea355bd90 ffffffffa02af720 0000000000000001 ffff880ea355bd90
[   14.041015] Call Trace:
[   14.041015]  [<ffffffff8145b148>] arch_prepare_kprobe+0x18/0x80
[   14.042982]  [<ffffffff8145ed38>] register_kprobe+0x338/0x4c0
[   14.042982]  [<ffffffff8145f658>] register_jprobes+0x98/0xc0
[   14.042982]  [<ffffffff8145f69a>] register_jprobe+0x1a/0x20
[   14.042982]  [<ffffffffa02a5f5d>] mlx4_stats_sysfs_create+0x2d/0x150 [mlx4_en]
[   14.042982]  [<ffffffffa02a4087>] mlx4_en_init_netdev+0xc77/0xe50 [mlx4_en]
[   14.042982]  [<ffffffffa029263a>] mlx4_en_add+0x44a/0x550 [mlx4_en]
[   14.042982]  [<ffffffffa02d2d1c>] mlx4_add_device+0x4c/0xe0 [mlx4_core]
[   14.042982]  [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[   14.042982]  [<ffffffffa02d2e23>] mlx4_register_interface+0x73/0xb0 [mlx4_core]
[   14.042982]  [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[   14.042982]  [<ffffffffa02b6031>] __init_backport+0x31/0x1000 [mlx4_en]
[   14.042982]  [<ffffffff810002b2>] do_one_initcall+0x122/0x180
[   14.042982]  [<ffffffff8109ed2d>] sys_init_module+0xbd/0x220
[   14.042982]  [<ffffffff81460c99>] system_call_fastpath+0x16/0x1b
[   14.042982] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 48 8b 05 49 e9 80 00 55 48 89 e5 48 3d e0 67 81 81 48 8d 48 d0 74 59 66 0f 1f 84 00 00 00 00 00 <48> 3b 71 20 72 3a 48 3b 79 28 77 34 48 8b 41 10 4c 8b 41 18 4c
[   14.042982] RIP  [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[   14.042982]  RSP <ffff880ea355bcb8>