[PATCH] nvme: fix use-after-free during booting

From: Tong Zhang
Date: Wed Sep 16 2020 - 13:57:21 EST

if a nvme controller is not responding during probing, a use-after-free
condition could happen

[ 215.396884] nvme nvme0: Identify Controller failed (-4)
[ 215.397239] nvme nvme0: Removing after probe failure status: -5
[ 215.409079] Buffer I/O error on dev nvme0n1, logical block 0, async page read
[ 215.409526] Buffer I/O error on dev nvme0n1, logical block 1, async page read
[ 215.409924] Buffer I/O error on dev nvme0n1, logical block 2, async page read
[ 215.410317] Buffer I/O error on dev nvme0n1, logical block 3, async page read
[ 215.410706] Buffer I/O error on dev nvme0n1, logical block 4, async page read
[ 215.411103] Buffer I/O error on dev nvme0n1, logical block 5, async page read
[ 215.411498] Buffer I/O error on dev nvme0n1, logical block 6, async page read
[ 215.411893] Buffer I/O error on dev nvme0n1, logical block 7, async page read
[ 215.412272] nvme0n1: unable to read partition table
[ 215.412581] nvme0n1: partition table beyond EOD, truncated
[ 215.966867] ------------[ cut here ]------------
[ 215.967167] WARNING: CPU: 1 PID: 7 at block/genhd.c:838 __device_add_disk+0x79c/0x7c0
[ 215.967558] Modules linked in:
[ 215.967717] CPU: 1 PID: 7 Comm: kworker/u4:0 Not tainted 5.8.0+ #59
[ 215.968618] Workqueue: nvme-wq nvme_scan_work
[ 215.968849] RIP: 0010:__device_add_disk+0x79c/0x7c0
[ 215.969098] Code: 85 30 04 00 00 48 89 44 24 20 e9 41 fa ff ff 48 89 df e8 07 e5 ca ff 80 a5 dc 00 00 00 ef e9 88 fc ff ff 0f 0b e9 81 fc ff ff <0f>
0b e9 a1 fc ff ff 0f 0b e9 b5 fa ff ff 0f 0b e9 51 ff ff ff e8
[ 215.970032] RSP: 0000:ffff88806c06f9f0 EFLAGS: 00010246
[ 215.970297] RAX: 0000000000000000 RBX: ffff88806c233400 RCX: ffffffff9ce3e74e
[ 215.970653] RDX: dffffc0000000000 RSI: 0000000000000008 RDI: ffff888066135560
[ 215.971015] RBP: ffff88806b2ce800 R08: 0000000000000001 R09: ffffed100cc26aad
[ 215.971373] R10: ffff888066135567 R11: ffffed100cc26aac R12: ffff88806c40f940
[ 215.971731] R13: ffff88806b2ce8a0 R14: ffff88806b2ce808 R15: ffff88806b2cebd8
[ 215.972094] FS: 0000000000000000(0000) GS:ffff88806d100000(0000) knlGS:0000000000000000
[ 215.972497] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 215.972789] CR2: 0000000000000000 CR3: 0000000024e0e000 CR4: 00000000000006e0
[ 215.973153] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 215.973513] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 215.973875] Call Trace:
[ 215.974007] ? blk_alloc_devt+0x140/0x140
[ 215.974213] ? __hrtimer_init+0xb6/0xf0
[ 215.974411] ? rwsem_down_read_slowpath+0x7d0/0x7d0
[ 215.974659] ? __nvme_revalidate_disk+0x1ba/0x420
[ 215.974905] nvme_validate_ns+0x54c/0xe70
[ 215.975113] ? nvme_dev_ioctl+0x190/0x190
[ 215.975318] ? __blk_mq_free_request+0xe3/0x130
[ 215.975549] ? __nvme_submit_sync_cmd+0x153/0x300
[ 215.975789] ? kasan_unpoison_shadow+0x33/0x40
[ 215.976022] nvme_scan_work+0x20f/0x35f
[ 215.976220] ? nvme_fw_act_work+0x210/0x210
[ 215.976434] ? free_object+0x50/0x50
[ 215.976619] ? try_to_wake_up+0x37c/0x900
[ 215.976619] ? try_to_wake_up+0x37c/0x900
[ 215.976843] ? read_word_at_a_time+0xe/0x20
[ 215.977057] ? strscpy+0xbf/0x190
[ 215.977229] process_one_work+0x4ad/0x7e0
[ 215.977435] worker_thread+0x73/0x690
[ 215.977623] ? process_one_work+0x7e0/0x7e0
[ 215.977842] kthread+0x199/0x1f0
[ 215.978011] ? kthread_create_on_node+0xd0/0xd0
[ 215.978240] ret_from_fork+0x22/0x30
[ 215.978423] ---[ end trace e8b38966135fe74b ]---
[ 215.987504] refcount_t: underflow; use-after-free.
[ 215.987772] WARNING: CPU: 1 PID: 7 at lib/refcount.c:28 refcount_warn_saturate+0xc5/0x110
[ 215.988641] Modules linked in:
[ 215.988827] CPU: 1 PID: 7 Comm: kworker/u4:0 Tainted: G W 5.8.0+ #59
[ 215.989784] Workqueue: nvme-wq nvme_scan_work
[ 215.990013] RIP: 0010:refcount_warn_saturate+0xc5/0x110
0b e9 76 ff ff ff 80 3d cc 0f f6 01 00 0f 85 69 ff ff ff 48 c7
[ 215.991220] RSP: 0000:ffff88806c06fb60 EFLAGS: 00010282
[ 215.991485] RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000000
[ 215.991850] RDX: 0000000000000001 RSI: 0000000000000004 RDI: ffffed100d80df5e
[ 215.992209] RBP: ffff8880661355b0 R08: 0000000000000001 R09: ffffed100d80df32
[ 215.992568] R10: 0000000000000003 R11: ffffed100d80df31 R12: ffff88806b2ce830
[ 215.992937] R13: ffff88806b2ce800 R14: ffff88806bf47a78 R15: ffff88806bdb6a88
[ 215.993294] FS: 0000000000000000(0000) GS:ffff88806d100000(0000) knlGS:0000000000000000
[ 215.993700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 215.993997] CR2: 0000000000000000 CR3: 0000000024e0e000 CR4: 00000000000006e0
[ 215.994356] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 215.994714] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 215.995077] Call Trace:
[ 215.995208] disk_release+0x114/0x130
[ 215.995397] device_release+0x3c/0xd0
[ 215.995586] kobject_put+0xa5/0x120
[ 215.995767] nvme_put_ns+0x40/0xa0
[ 215.995950] nvme_remove_invalid_namespaces+0x1bd/0x210
[ 215.996214] ? nvme_ns_remove+0x2c0/0x2c0
[ 215.996420] nvme_scan_work+0x258/0x35f
[ 215.996618] ? nvme_fw_act_work+0x210/0x210
[ 215.996841] ? free_object+0x50/0x50
[ 215.997027] ? try_to_wake_up+0x37c/0x900
[ 215.997233] ? read_word_at_a_time+0xe/0x20
[ 215.997446] ? strscpy+0xbf/0x190
[ 215.997617] process_one_work+0x4ad/0x7e0
[ 215.997830] worker_thread+0x73/0x690
[ 215.998021] ? process_one_work+0x7e0/0x7e0
[ 215.998234] kthread+0x199/0x1f0
[ 215.998401] ? kthread_create_on_node+0xd0/0xd0
[ 215.998632] ret_from_fork+0x22/0x30
[ 215.998822] ---[ end trace e8b38966135fe74c ]---

Signed-off-by: Tong Zhang <ztong0001@xxxxxxxxx>
---
drivers/nvme/host/core.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d543bc1747fd..cdd40aab10e9 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -481,6 +481,8 @@ static void nvme_free_ns(struct kref *kref)
if (ns->ndev)
nvme_nvm_unregister(ns);

+ /* prevent double queue cleanup */
+ ns->disk->queue = NULL;
put_disk(ns->disk);
nvme_put_ns_head(ns->head);
nvme_put_ctrl(ns->ctrl);
--
2.25.1