Trying to understand General protection fault/hrtimer_active

From: Artem S. Tashkinov
Date: Fri Aug 11 2017 - 08:20:49 EST

Hello,

After stopping mariadb on our database server, the server physically crashed and required a hard reset in order to get back online.

Fortunately the system was able to dump the kernel error:

Aug 11 09:22:44 mariadb mysqld[1229]: 2017-08-11 9:22:44 140417868658432 [ERROR] mysqld: Deadlock found when trying to get lock; try restarting transaction
Aug 11 09:24:03 mariadb kernel: [225113.038696] general protection fault: 0000 [#1] SMP
Aug 11 09:24:03 mariadb kernel: [225113.038709] Modules linked in: ppdev intel_rapl x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm irqbypass crct10dif_pc
lmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul joydev input_leds glue_helper ablk_helper cryptd serio_raw shpchp lpc_ich parport_pc
8250_fintek parport tpm_infineon mac_hid nct6775 hwmon_vid coretemp autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx x
or hid_generic usbhid hid raid6_pq libcrc32c raid0 multipath linear raid1 mxm_wmi ahci psmouse r8169 libahci mii wmi video fjes
Aug 11 09:24:03 mariadb kernel: [225113.038836] CPU: 3 PID: 3570 Comm: mysqld Not tainted 4.4.0-89-generic #112-Ubuntu
Aug 11 09:24:03 mariadb kernel: [225113.038853] Hardware name: MSI MS-7816/H87-G43 (MS-7816), BIOS V2.14B6 08/23/2013
Aug 11 09:24:03 mariadb kernel: [225113.038868] task: ffff8807f6f88e00 ti: ffff8807f6534000 task.ti: ffff8807f6534000
Aug 11 09:24:03 mariadb kernel: [225113.038881] RIP: 0010:[<ffffffff810ef659>] [<ffffffff810ef659>] hrtimer_active+0x9/0x60
Aug 11 09:24:03 mariadb kernel: [225113.038899] RSP: 0018:ffff8807f65379e0 EFLAGS: 00010246
Aug 11 09:24:03 mariadb kernel: [225113.038909] RAX: 0000000000000000 RBX: ffbf8807f6537a30 RCX: 0000000000000000
Aug 11 09:24:03 mariadb kernel: [225113.038922] RDX: 00000000ffffffff RSI: ffff8807f6f88e00 RDI: ffbf8807f6537a30
Aug 11 09:24:03 mariadb kernel: [225113.038947] RBP: ffff8807f65379e0 R08: ffff8807f6534000 R09: 0000000000000000
Aug 11 09:24:03 mariadb kernel: [225113.038982] R10: 0000000103599c14 R11: 0000000000000000 R12: 0000000000000000
Aug 11 09:24:03 mariadb kernel: [225113.039018] R13: 0000000000000001 R14: ffff8807f6537b58 R15: 0000000000000000
Aug 11 09:24:03 mariadb kernel: [225113.039053] FS: 00007fb69edc5700(0000) GS:ffff88081eac0000(0000) knlGS:0000000000000000
Aug 11 09:24:03 mariadb kernel: [225113.039091] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Aug 11 09:24:03 mariadb kernel: [225113.039112] CR2: 00007fb59e1e7e88 CR3: 00000007f943f000 CR4: 00000000001406e0
Aug 11 09:24:03 mariadb kernel: [225113.039148] Stack:
Aug 11 09:24:03 mariadb kernel: [225113.039164] ffff8807f6537a18 ffffffff810efba9 ffff8807f6537b58 2cf88ace51220a81
Aug 11 09:24:03 mariadb kernel: [225113.039202] ffbf8807f6537a30 0000000000000000 0000000000000001 ffff8807f6537ac0
Aug 11 09:24:03 mariadb kernel: [225113.039240] ffffffff81841341 0000000005f5e100 ffff88071ab63a30 0000000000000000
Aug 11 09:24:03 mariadb kernel: [225113.039278] Call Trace:
Aug 11 09:24:03 mariadb kernel: [225113.039297] [<ffffffff810efba9>] hrtimer_try_to_cancel+0x29/0x130
Aug 11 09:24:03 mariadb kernel: [225113.039321] [<ffffffff81841341>] schedule_hrtimeout_range_clock+0xd1/0x1b0
Aug 11 09:24:03 mariadb kernel: [225113.039346] [<ffffffff810ef970>] ? __hrtimer_init+0x90/0x90
Aug 11 09:24:03 mariadb kernel: [225113.039369] [<ffffffff81841329>] ? schedule_hrtimeout_range_clock+0xb9/0x1b0
Aug 11 09:24:03 mariadb kernel: [225113.039405] [<ffffffff81841433>] schedule_hrtimeout_range+0x13/0x20
Aug 11 09:24:03 mariadb kernel: [225113.039430] [<ffffffff81223f24>] poll_schedule_timeout+0x44/0x70
Aug 11 09:24:03 mariadb kernel: [225113.039453] [<ffffffff812255df>] do_sys_poll+0x4af/0x560
Aug 11 09:24:03 mariadb kernel: [225113.039477] [<ffffffff817226ab>] ? __alloc_skb+0x5b/0x1f0
Aug 11 09:24:03 mariadb kernel: [225113.039500] [<ffffffff811f1799>] ? __kmalloc_node_track_caller+0x249/0x310
Aug 11 09:24:03 mariadb kernel: [225113.039525] [<ffffffff817226d7>] ? __alloc_skb+0x87/0x1f0
Aug 11 09:24:03 mariadb kernel: [225113.039548] [<ffffffff812240c0>] ? poll_select_copy_remaining+0x140/0x140
Aug 11 09:24:03 mariadb kernel: [225113.039572] [<ffffffff81841a7e>] ? _raw_spin_unlock_bh+0x1e/0x20
Aug 11 09:24:03 mariadb kernel: [225113.039596] [<ffffffff8171dcf1>] ? release_sock+0x111/0x160
Aug 11 09:24:03 mariadb kernel: [225113.039620] [<ffffffff81782eac>] ? tcp_recvmsg+0x3fc/0xbe0
Aug 11 09:24:03 mariadb kernel: [225113.039644] [<ffffffff817b147e>] ? inet_recvmsg+0x7e/0xb0
Aug 11 09:24:03 mariadb kernel: [225113.039666] [<ffffffff817195cd>] ? sock_recvmsg+0x3d/0x50
Aug 11 09:24:03 mariadb kernel: [225113.039688] [<ffffffff8171986d>] ? SYSC_recvfrom+0x13d/0x150
Aug 11 09:24:03 mariadb kernel: [225113.039711] [<ffffffff8183d766>] ? __schedule+0x3b6/0xa30
Aug 11 09:24:03 mariadb kernel: [225113.039734] [<ffffffff810f54a9>] ? ktime_get_ts64+0x49/0xf0
Aug 11 09:24:03 mariadb kernel: [225113.039756] [<ffffffff81225781>] SyS_poll+0x71/0x130
Aug 11 09:24:03 mariadb kernel: [225113.039778] [<ffffffff81841f32>] entry_SYSCALL_64_fastpath+0x16/0x71
Aug 11 09:24:03 mariadb kernel: [225113.039801] Code: 00 00 0f 1f 44 00 00 55 48 c7 47 28 70 f9 0e 81 48 89 77 58 48 89 e5 5d c3 66 0f 1f 84 00 00 00 00 00 0f
1f 44 00 00 55 48 89 e5 <48> 8b 57 30 eb 1d 80 7f 38 00 75 32 48 3b 78 08 74 2c 39 50 04
Aug 11 09:24:03 mariadb kernel: [225113.039909] RIP [<ffffffff810ef659>] hrtimer_active+0x9/0x60
Aug 11 09:24:03 mariadb kernel: [225113.039933] RSP <ffff8807f65379e0>
Aug 11 09:24:03 mariadb kernel: [225113.040228] ---[ end trace a4cd8a286c3c9557 ]---

I've tried Googling for this error however the only relevant result seems to be this: https://access.redhat.com/solutions/1392343 but I cannot access it.

We're running linux 4.4.0-89.112 generic x86-64 Ubuntu 16.04 LTS.

Our system specs:

Motherboard: H87-G43 (MS-7816), unknown beta BIOS release from 2013/08
CPU: Intel Core i7-4770
RAM: 32GB (DDR3 1600MHz 4*8GB)
SSD: 2 x INTEL SSDSC2CW24 (software raid level 1)

Regards,

--
Best regards,

Artem