Re: [PATCH 00/17] fs, btrfs refcount conversions
From: Qu Wenruo
Date: Tue Mar 07 2017 - 03:18:38 EST
At 03/07/2017 03:41 PM, Reshetova, Elena wrote:
At 03/06/2017 05:43 PM, Reshetova, Elena wrote:
At 03/03/2017 04:55 PM, Elena Reshetova wrote:
Now when new refcount_t type and API are finally merged
(see include/linux/refcount.h), the following
patches convert various refcounters in the btrfs filesystem from atomic_t
to refcount_t. By doing this we prevent intentional or accidental
underflows or overflows that can led to use-after-free vulnerabilities.
The below patches are fully independent and can be cherry-picked separately.
Since we convert all kernel subsystems in the same fashion, resulting
in about 300 patches, we have to group them for sending at least in some
fashion to be manageable. Please excuse the long cc list.
These patches have been tested with xfstests by running btrfs-related tests.
btrfs debug was enabled, warns on refcount errors, too. No output related to
refcount errors produced. However, the following errors were during the run:
* tests btrfs/078, btrfs/114, btrfs/115, no errors anywhere in dmesg, but
process hangs. They all seem to be around qgroup, sometimes error visible
such as qgroup scan failed -4 before it blocks, but not always.
How reproducible of the hang?
Always in my environment, but I would not much go into investigating why it
happens, if it works for you.
My test environment is far from ideal: I am testing in VM with rather old
userspace and couple of additional changes in,
so there are many things that can potentially go wrong. Anyway the strace for
078 is in the attachment.
Thanks for the strace.
However no "-f" is passed to strace, so it doesn't contain much useful info.
If the patches pass all tests on your side, could you please take them in and
propagate further?
I will continue with other kernel subsystems.
The patchset itself looks like a common cleanup, while I did encounter
several cases (almost all scrub tests) causing kernel warning due to
underflow.
Oh, could you please send me the warning outputs? I can hopefully analyze and fix them.
Attached. Which is the generated by running btrfs/070 test case.
And I canceled the case almost instantly, so output is not much, but
still contains enough info.
Both refcount_inc() and refcount_sub_and_test() are causing warning.
So now I'm not sure which is the cause, btrfs or bad use of refcount?
Thanks,
Qu
Best Regards,
Elena.
So I'm afraid the patchset will not be merged until we fix all the
underflows.
But thanks for the patchset, it helps us to expose a lot of problem.
Thanks,
Qu
Best Regards,
Elena.
I also see the -EINTR output, but that seems to be designed for
btrfs/11[45].
btrfs/078 is unrelated to qgroup, and all these three test pass in my
test environment, which is v4.11-rc1 with your patches applied.
I ran these 3 tests in a row with default and space_cache=v2 mount
options, and 5 times for each mount option, no hang at all.
It would help much if more info can be provided, from blocked process
backtrace to test mount option to base commit.
Thanks,
Qu
* test btrfs/104 dmesg has additional error output:
BTRFS warning (device vdc): qgroup 258 reserved space underflow, have: 0,
to free: 4096
I tried looking at the code on what causes the failure, but could not figure
it out. It doesn't seem to be related to any refcount changes at least IMO.
The above test failures are hard for me to understand and interpreted, but
they don't seem to relate to refcount conversions.
Elena Reshetova (17):
fs, btrfs: convert btrfs_bio.refs from atomic_t to refcount_t
fs, btrfs: convert btrfs_transaction.use_count from atomic_t to
refcount_t
fs, btrfs: convert extent_map.refs from atomic_t to refcount_t
fs, btrfs: convert btrfs_ordered_extent.refs from atomic_t to
refcount_t
fs, btrfs: convert btrfs_caching_control.count from atomic_t to
refcount_t
fs, btrfs: convert btrfs_delayed_ref_node.refs from atomic_t to
refcount_t
fs, btrfs: convert btrfs_delayed_node.refs from atomic_t to refcount_t
fs, btrfs: convert btrfs_delayed_item.refs from atomic_t to refcount_t
fs, btrfs: convert btrfs_root.refs from atomic_t to refcount_t
fs, btrfs: convert extent_state.refs from atomic_t to refcount_t
fs, btrfs: convert compressed_bio.pending_bios from atomic_t to
refcount_t
fs, btrfs: convert scrub_recover.refs from atomic_t to refcount_t
fs, btrfs: convert scrub_page.refs from atomic_t to refcount_t
fs, btrfs: convert scrub_block.refs from atomic_t to refcount_t
fs, btrfs: convert scrub_parity.refs from atomic_t to refcount_t
fs, btrfs: convert scrub_ctx.refs from atomic_t to refcount_t
fs, btrfs: convert btrfs_raid_bio.refs from atomic_t to refcount_t
fs/btrfs/backref.c | 2 +-
fs/btrfs/compression.c | 18 ++++++++---------
fs/btrfs/ctree.h | 5 +++--
fs/btrfs/delayed-inode.c | 46 ++++++++++++++++++++++----------------------
fs/btrfs/delayed-inode.h | 5 +++--
fs/btrfs/delayed-ref.c | 8 ++++----
fs/btrfs/delayed-ref.h | 8 +++++---
fs/btrfs/disk-io.c | 6 +++---
fs/btrfs/disk-io.h | 4 ++--
fs/btrfs/extent-tree.c | 20 +++++++++----------
fs/btrfs/extent_io.c | 18 ++++++++---------
fs/btrfs/extent_io.h | 3 ++-
fs/btrfs/extent_map.c | 10 +++++-----
fs/btrfs/extent_map.h | 3 ++-
fs/btrfs/ordered-data.c | 20 +++++++++----------
fs/btrfs/ordered-data.h | 2 +-
fs/btrfs/raid56.c | 19 +++++++++---------
fs/btrfs/scrub.c | 42 ++++++++++++++++++++--------------------
fs/btrfs/transaction.c | 20 +++++++++----------
fs/btrfs/transaction.h | 3 ++-
fs/btrfs/tree-log.c | 2 +-
fs/btrfs/volumes.c | 10 +++++-----
fs/btrfs/volumes.h | 2 +-
include/trace/events/btrfs.h | 4 ++--
24 files changed, 143 insertions(+), 137 deletions(-)
[17805.205289] run fstests btrfs/070 at 2017-03-07 14:02:47
[17805.765749] BTRFS: device fsid ef047ab3-6c98-4d0c-b76c-f10631b58a51 devid 1 transid 2 /dev/vdb6
[17805.776084] BTRFS: device fsid ef047ab3-6c98-4d0c-b76c-f10631b58a51 devid 2 transid 3 /dev/vdb7
[17805.779794] BTRFS: device fsid ef047ab3-6c98-4d0c-b76c-f10631b58a51 devid 3 transid 3 /dev/vdb8
[17805.783487] BTRFS: device fsid ef047ab3-6c98-4d0c-b76c-f10631b58a51 devid 4 transid 3 /dev/vdb9
[17806.124656] BTRFS info (device vdb9): disk space caching is enabled
[17806.125164] BTRFS info (device vdb9): has skinny extents
[17806.125528] BTRFS info (device vdb9): flagging fs with big metadata feature
[17806.129091] BTRFS info (device vdb9): creating UUID tree
[17807.145227] BTRFS info (device vdb9): dev_replace from /dev/vdb7 (devid 2) to /dev/vdb10 started
[17808.335771] ------------[ cut here ]------------
[17808.336144] WARNING: CPU: 0 PID: 9448 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.336786] refcount_t: increment on 0; use-after-free.
[17808.337143] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.337753] CPU: 0 PID: 9448 Comm: btrfs Tainted: G W O 4.11.0-rc1+ #71
[17808.338134] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.338134] Call Trace:
[17808.338134] dump_stack+0x68/0x93
[17808.338134] __warn+0xcb/0xf0
[17808.338134] warn_slowpath_fmt+0x4f/0x60
[17808.338134] refcount_inc+0x27/0x30
[17808.338134] scrub_pages+0x118/0x410 [btrfs]
[17808.338134] scrub_stripe+0x851/0x1170 [btrfs]
[17808.338134] ? trace_hardirqs_on+0xd/0x10
[17808.338134] scrub_chunk+0x109/0x150 [btrfs]
[17808.338134] scrub_enumerate_chunks+0x297/0x660 [btrfs]
[17808.338134] btrfs_scrub_dev+0x1f6/0x670 [btrfs]
[17808.338134] btrfs_dev_replace_start+0x2f1/0x4f0 [btrfs]
[17808.338134] btrfs_dev_replace_by_ioctl+0x3e/0x70 [btrfs]
[17808.338134] btrfs_ioctl+0x1e40/0x27f0 [btrfs]
[17808.338134] ? do_sigaction+0x69/0x1b0
[17808.338134] do_vfs_ioctl+0x94/0x710
[17808.338134] ? do_sigaction+0x193/0x1b0
[17808.338134] SyS_ioctl+0x79/0x90
[17808.338134] entry_SYSCALL_64_fastpath+0x18/0xad
[17808.338134] RIP: 0033:0x7fe702de1787
[17808.338134] RSP: 002b:00007ffd3dbf0758 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[17808.338134] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007fe702de1787
[17808.338134] RDX: 00007ffd3dbf0b90 RSI: 00000000ca289435 RDI: 0000000000000003
[17808.338134] RBP: 0000000000000004 R08: 0000000000000000 R09: 0000000000000000
[17808.338134] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000561120154050
[17808.338134] R13: 00007ffd3dbf2e6f R14: 0000000000000000 R15: 0000000000000001
[17808.347969] ---[ end trace 42fc4dedec8f6207 ]---
[17808.349374] ------------[ cut here ]------------
[17808.349716] WARNING: CPU: 0 PID: 9448 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.351339] refcount_t: increment on 0; use-after-free.
[17808.351667] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.352383] CPU: 0 PID: 9448 Comm: btrfs Tainted: G W O 4.11.0-rc1+ #71
[17808.353121] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.353371] Call Trace:
[17808.353371] dump_stack+0x68/0x93
[17808.353371] __warn+0xcb/0xf0
[17808.353371] warn_slowpath_fmt+0x4f/0x60
[17808.353371] refcount_inc+0x27/0x30
[17808.355021] scrub_pages+0x118/0x410 [btrfs]
[17808.355021] scrub_stripe+0x851/0x1170 [btrfs]
[17808.355021] ? trace_hardirqs_on+0xd/0x10
[17808.355021] scrub_chunk+0x109/0x150 [btrfs]
[17808.355021] scrub_enumerate_chunks+0x297/0x660 [btrfs]
[17808.359021] btrfs_scrub_dev+0x1f6/0x670 [btrfs]
[17808.363022] btrfs_dev_replace_start+0x2f1/0x4f0 [btrfs]
[17808.363022] btrfs_dev_replace_by_ioctl+0x3e/0x70 [btrfs]
[17808.363022] btrfs_ioctl+0x1e40/0x27f0 [btrfs]
[17808.363022] ? do_sigaction+0x69/0x1b0
[17808.363022] do_vfs_ioctl+0x94/0x710
[17808.363022] ? do_sigaction+0x193/0x1b0
[17808.363022] SyS_ioctl+0x79/0x90
[17808.363022] entry_SYSCALL_64_fastpath+0x18/0xad
[17808.363022] RIP: 0033:0x7fe702de1787
[17808.363022] RSP: 002b:00007ffd3dbf0758 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[17808.363022] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007fe702de1787
[17808.367025] RDX: 00007ffd3dbf0b90 RSI: 00000000ca289435 RDI: 0000000000000003
[17808.367025] RBP: 0000000000000004 R08: 0000000000000000 R09: 0000000000000000
[17808.367025] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000561120154050
[17808.367025] R13: 00007ffd3dbf2e6f R14: 0000000000000000 R15: 0000000000000001
[17808.369634] ---[ end trace 42fc4dedec8f6208 ]---
[17808.375435] ------------[ cut here ]------------
[17808.381855] WARNING: CPU: 0 PID: 9448 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.385910] refcount_t: increment on 0; use-after-free.
[17808.387980] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.388588] CPU: 0 PID: 9448 Comm: btrfs Tainted: G W O 4.11.0-rc1+ #71
[17808.389166] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.389578] Call Trace:
[17808.389865] dump_stack+0x68/0x93
[17808.389865] __warn+0xcb/0xf0
[17808.389865] warn_slowpath_fmt+0x4f/0x60
[17808.389865] refcount_inc+0x27/0x30
[17808.389865] scrub_pages+0x118/0x410 [btrfs]
[17808.389865] scrub_stripe+0x851/0x1170 [btrfs]
[17808.389865] ? trace_hardirqs_on+0xd/0x10
[17808.389865] scrub_chunk+0x109/0x150 [btrfs]
[17808.389865] scrub_enumerate_chunks+0x297/0x660 [btrfs]
[17808.389865] btrfs_scrub_dev+0x1f6/0x670 [btrfs]
[17808.389865] btrfs_dev_replace_start+0x2f1/0x4f0 [btrfs]
[17808.389865] btrfs_dev_replace_by_ioctl+0x3e/0x70 [btrfs]
[17808.389865] btrfs_ioctl+0x1e40/0x27f0 [btrfs]
[17808.389865] ? do_sigaction+0x69/0x1b0
[17808.389865] do_vfs_ioctl+0x94/0x710
[17808.389865] ? do_sigaction+0x193/0x1b0
[17808.389865] SyS_ioctl+0x79/0x90
[17808.389865] entry_SYSCALL_64_fastpath+0x18/0xad
[17808.389865] RIP: 0033:0x7fe702de1787
[17808.389865] RSP: 002b:00007ffd3dbf0758 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[17808.389865] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007fe702de1787
[17808.389865] RDX: 00007ffd3dbf0b90 RSI: 00000000ca289435 RDI: 0000000000000003
[17808.389865] RBP: 0000000000000004 R08: 0000000000000000 R09: 0000000000000000
[17808.389865] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000561120154050
[17808.389865] R13: 00007ffd3dbf2e6f R14: 0000000000000000 R15: 0000000000000001
[17808.401890] ---[ end trace 42fc4dedec8f6209 ]---
[17808.402282] ------------[ cut here ]------------
[17808.402623] WARNING: CPU: 0 PID: 9448 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.403358] refcount_t: increment on 0; use-after-free.
[17808.403701] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.404494] CPU: 0 PID: 9448 Comm: btrfs Tainted: G W O 4.11.0-rc1+ #71
[17808.405069] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.405455] Call Trace:
[17808.405455] dump_stack+0x68/0x93
[17808.405455] __warn+0xcb/0xf0
[17808.405455] warn_slowpath_fmt+0x4f/0x60
[17808.405455] refcount_inc+0x27/0x30
[17808.405455] scrub_pages+0x118/0x410 [btrfs]
[17808.405455] scrub_stripe+0x851/0x1170 [btrfs]
[17808.405455] ? trace_hardirqs_on+0xd/0x10
[17808.405455] scrub_chunk+0x109/0x150 [btrfs]
[17808.405455] scrub_enumerate_chunks+0x297/0x660 [btrfs]
[17808.405455] btrfs_scrub_dev+0x1f6/0x670 [btrfs]
[17808.405455] btrfs_dev_replace_start+0x2f1/0x4f0 [btrfs]
[17808.405455] btrfs_dev_replace_by_ioctl+0x3e/0x70 [btrfs]
[17808.405455] btrfs_ioctl+0x1e40/0x27f0 [btrfs]
[17808.405455] ? do_sigaction+0x69/0x1b0
[17808.405455] do_vfs_ioctl+0x94/0x710
[17808.405455] ? do_sigaction+0x193/0x1b0
[17808.405455] SyS_ioctl+0x79/0x90
[17808.405455] entry_SYSCALL_64_fastpath+0x18/0xad
[17808.405455] RIP: 0033:0x7fe702de1787
[17808.405455] RSP: 002b:00007ffd3dbf0758 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[17808.405455] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007fe702de1787
[17808.405455] RDX: 00007ffd3dbf0b90 RSI: 00000000ca289435 RDI: 0000000000000003
[17808.405455] RBP: 0000000000000004 R08: 0000000000000000 R09: 0000000000000000
[17808.405455] R10: 00000000ffffffff R11: 0000000000000246 R12: 0000561120154050
[17808.405455] R13: 00007ffd3dbf2e6f R14: 0000000000000000 R15: 0000000000000001
[17808.415683] ---[ end trace 42fc4dedec8f620a ]---
[17808.416261] ------------[ cut here ]------------
[17808.417080] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.417824] refcount_t: increment on 0; use-after-free.
[17808.418425] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.419194] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.419932] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.420179] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.420179] Call Trace:
[17808.420179] dump_stack+0x68/0x93
[17808.420179] __warn+0xcb/0xf0
[17808.420179] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.420179] warn_slowpath_fmt+0x4f/0x60
[17808.420179] refcount_inc+0x27/0x30
[17808.420179] scrub_write_page_to_dev_replace+0x1a1/0x2a0 [btrfs]
[17808.420179] scrub_write_block_to_dev_replace+0x40/0x60 [btrfs]
[17808.420179] scrub_bio_end_io_worker+0x36a/0x520 [btrfs]
[17808.420179] ? debug_object_deactivate+0x56/0x130
[17808.420179] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.420179] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.420179] process_one_work+0x2af/0x720
[17808.420179] ? process_one_work+0x22b/0x720
[17808.420179] worker_thread+0x4b/0x4f0
[17808.420179] kthread+0x10f/0x150
[17808.420179] ? process_one_work+0x720/0x720
[17808.420179] ? kthread_create_on_node+0x40/0x40
[17808.420179] ret_from_fork+0x2e/0x40
[17808.427447] ---[ end trace 42fc4dedec8f620b ]---
[17808.428357] ------------[ cut here ]------------
[17808.428828] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.429542] refcount_t: increment on 0; use-after-free.
[17808.430117] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.430766] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.431012] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.431012] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.431012] Call Trace:
[17808.431012] dump_stack+0x68/0x93
[17808.431012] __warn+0xcb/0xf0
[17808.431012] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.431012] warn_slowpath_fmt+0x4f/0x60
[17808.431012] refcount_inc+0x27/0x30
[17808.431012] scrub_write_page_to_dev_replace+0x1a1/0x2a0 [btrfs]
[17808.431012] scrub_write_block_to_dev_replace+0x40/0x60 [btrfs]
[17808.436040] scrub_bio_end_io_worker+0x36a/0x520 [btrfs]
[17808.436040] ? debug_object_deactivate+0x56/0x130
[17808.436040] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.436040] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.436040] process_one_work+0x2af/0x720
[17808.436040] ? process_one_work+0x22b/0x720
[17808.436040] worker_thread+0x4b/0x4f0
[17808.436040] kthread+0x10f/0x150
[17808.436040] ? process_one_work+0x720/0x720
[17808.436040] ? kthread_create_on_node+0x40/0x40
[17808.436040] ret_from_fork+0x2e/0x40
[17808.440016] ---[ end trace 42fc4dedec8f620c ]---
[17808.440464] ------------[ cut here ]------------
[17808.440951] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.441632] refcount_t: increment on 0; use-after-free.
[17808.442151] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.442884] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.443033] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.443033] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.443033] Call Trace:
[17808.443033] dump_stack+0x68/0x93
[17808.443033] __warn+0xcb/0xf0
[17808.443033] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.443033] warn_slowpath_fmt+0x4f/0x60
[17808.443033] refcount_inc+0x27/0x30
[17808.443033] scrub_write_page_to_dev_replace+0x1a1/0x2a0 [btrfs]
[17808.443033] scrub_write_block_to_dev_replace+0x40/0x60 [btrfs]
[17808.443033] scrub_bio_end_io_worker+0x36a/0x520 [btrfs]
[17808.443033] ? debug_object_deactivate+0x56/0x130
[17808.443033] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.443033] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.443033] process_one_work+0x2af/0x720
[17808.443033] ? process_one_work+0x22b/0x720
[17808.443033] worker_thread+0x4b/0x4f0
[17808.443033] kthread+0x10f/0x150
[17808.443033] ? process_one_work+0x720/0x720
[17808.443033] ? kthread_create_on_node+0x40/0x40
[17808.443033] ret_from_fork+0x2e/0x40
[17808.456507] ---[ end trace 42fc4dedec8f620d ]---
[17808.457696] ------------[ cut here ]------------
[17808.457998] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:114 refcount_inc+0x27/0x30
[17808.458681] refcount_t: increment on 0; use-after-free.
[17808.459075] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.459685] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.460006] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.460006] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.460006] Call Trace:
[17808.460006] dump_stack+0x68/0x93
[17808.460006] __warn+0xcb/0xf0
[17808.460006] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.460006] warn_slowpath_fmt+0x4f/0x60
[17808.460006] refcount_inc+0x27/0x30
[17808.460006] scrub_write_page_to_dev_replace+0x1a1/0x2a0 [btrfs]
[17808.460006] scrub_write_block_to_dev_replace+0x40/0x60 [btrfs]
[17808.460006] scrub_bio_end_io_worker+0x36a/0x520 [btrfs]
[17808.460006] ? debug_object_deactivate+0x56/0x130
[17808.460006] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.460006] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.460006] process_one_work+0x2af/0x720
[17808.460006] ? process_one_work+0x22b/0x720
[17808.460006] worker_thread+0x4b/0x4f0
[17808.460006] kthread+0x10f/0x150
[17808.460006] ? process_one_work+0x720/0x720
[17808.460006] ? kthread_create_on_node+0x40/0x40
[17808.460006] ret_from_fork+0x2e/0x40
[17808.467673] ---[ end trace 42fc4dedec8f620e ]---
[17808.469434] ------------[ cut here ]------------
[17808.469913] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.470600] refcount_t: underflow; use-after-free.
[17808.471125] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.471836] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.472005] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.472005] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.472005] Call Trace:
[17808.472005] dump_stack+0x68/0x93
[17808.472005] __warn+0xcb/0xf0
[17808.472005] warn_slowpath_fmt+0x4f/0x60
[17808.472005] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.472005] refcount_sub_and_test+0x60/0x70
[17808.472005] refcount_dec_and_test+0x11/0x20
[17808.472005] scrub_page_put+0x16/0x40 [btrfs]
[17808.472005] scrub_block_put+0x5a/0x80 [btrfs]
[17808.472005] scrub_bio_end_io_worker+0x77/0x520 [btrfs]
[17808.472005] ? debug_object_deactivate+0x56/0x130
[17808.472005] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.472005] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.472005] process_one_work+0x2af/0x720
[17808.472005] ? process_one_work+0x22b/0x720
[17808.472005] worker_thread+0x4b/0x4f0
[17808.472005] kthread+0x10f/0x150
[17808.472005] ? process_one_work+0x720/0x720
[17808.472005] ? kthread_create_on_node+0x40/0x40
[17808.472005] ret_from_fork+0x2e/0x40
[17808.479703] ---[ end trace 42fc4dedec8f620f ]---
[17808.480326] ------------[ cut here ]------------
[17808.480875] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.481584] refcount_t: underflow; use-after-free.
[17808.481885] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.482802] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.483383] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.483680] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.483680] Call Trace:
[17808.483680] dump_stack+0x68/0x93
[17808.483680] __warn+0xcb/0xf0
[17808.483680] warn_slowpath_fmt+0x4f/0x60
[17808.483680] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.483680] refcount_sub_and_test+0x60/0x70
[17808.483680] refcount_dec_and_test+0x11/0x20
[17808.483680] scrub_page_put+0x16/0x40 [btrfs]
[17808.483680] scrub_block_put+0x5a/0x80 [btrfs]
[17808.487038] scrub_bio_end_io_worker+0x77/0x520 [btrfs]
[17808.487038] ? debug_object_deactivate+0x56/0x130
[17808.487038] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.487038] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.487038] process_one_work+0x2af/0x720
[17808.487038] ? process_one_work+0x22b/0x720
[17808.487038] worker_thread+0x4b/0x4f0
[17808.487038] kthread+0x10f/0x150
[17808.487038] ? process_one_work+0x720/0x720
[17808.487038] ? kthread_create_on_node+0x40/0x40
[17808.487038] ret_from_fork+0x2e/0x40
[17808.490757] ---[ end trace 42fc4dedec8f6210 ]---
[17808.491117] ------------[ cut here ]------------
[17808.491448] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.492062] refcount_t: underflow; use-after-free.
[17808.492393] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.493053] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.493648] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.494030] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.494030] Call Trace:
[17808.494030] dump_stack+0x68/0x93
[17808.494030] __warn+0xcb/0xf0
[17808.494030] warn_slowpath_fmt+0x4f/0x60
[17808.494030] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.494030] refcount_sub_and_test+0x60/0x70
[17808.494030] refcount_dec_and_test+0x11/0x20
[17808.494030] scrub_page_put+0x16/0x40 [btrfs]
[17808.494030] scrub_block_put+0x5a/0x80 [btrfs]
[17808.494030] scrub_bio_end_io_worker+0x77/0x520 [btrfs]
[17808.494030] ? debug_object_deactivate+0x56/0x130
[17808.494030] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.494030] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.494030] process_one_work+0x2af/0x720
[17808.494030] ? process_one_work+0x22b/0x720
[17808.494030] worker_thread+0x4b/0x4f0
[17808.494030] kthread+0x10f/0x150
[17808.494030] ? process_one_work+0x720/0x720
[17808.494030] ? kthread_create_on_node+0x40/0x40
[17808.494030] ret_from_fork+0x2e/0x40
[17808.501354] ---[ end trace 42fc4dedec8f6211 ]---
[17808.502677] ------------[ cut here ]------------
[17808.503040] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.503639] refcount_t: underflow; use-after-free.
[17808.503954] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.504592] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.505187] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.505569] Workqueue: btrfs-scrub btrfs_scrub_helper [btrfs]
[17808.505569] Call Trace:
[17808.505569] dump_stack+0x68/0x93
[17808.505569] __warn+0xcb/0xf0
[17808.505569] warn_slowpath_fmt+0x4f/0x60
[17808.505569] ? scrub_workers_put+0x80/0x80 [btrfs]
[17808.505569] refcount_sub_and_test+0x60/0x70
[17808.505569] refcount_dec_and_test+0x11/0x20
[17808.505569] scrub_page_put+0x16/0x40 [btrfs]
[17808.505569] scrub_block_put+0x5a/0x80 [btrfs]
[17808.505569] scrub_bio_end_io_worker+0x77/0x520 [btrfs]
[17808.505569] ? debug_object_deactivate+0x56/0x130
[17808.505569] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.505569] btrfs_scrub_helper+0xe/0x10 [btrfs]
[17808.505569] process_one_work+0x2af/0x720
[17808.505569] ? process_one_work+0x22b/0x720
[17808.505569] worker_thread+0x4b/0x4f0
[17808.505569] kthread+0x10f/0x150
[17808.505569] ? process_one_work+0x720/0x720
[17808.505569] ? kthread_create_on_node+0x40/0x40
[17808.505569] ret_from_fork+0x2e/0x40
[17808.512438] ---[ end trace 42fc4dedec8f6212 ]---
[17808.512962] ------------[ cut here ]------------
[17808.513288] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.513895] refcount_t: underflow; use-after-free.
[17808.514241] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.514865] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.515220] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.515220] Workqueue: btrfs-scrubwrc btrfs_scrubwrc_helper [btrfs]
[17808.515220] Call Trace:
[17808.515220] dump_stack+0x68/0x93
[17808.515220] __warn+0xcb/0xf0
[17808.515220] warn_slowpath_fmt+0x4f/0x60
[17808.515220] ? debug_object_deactivate+0x56/0x130
[17808.515220] refcount_sub_and_test+0x60/0x70
[17808.515220] refcount_dec_and_test+0x11/0x20
[17808.515220] scrub_page_put+0x16/0x40 [btrfs]
[17808.515220] scrub_wr_bio_end_io_worker+0x93/0xe0 [btrfs]
[17808.515220] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.515220] btrfs_scrubwrc_helper+0xe/0x10 [btrfs]
[17808.515220] process_one_work+0x2af/0x720
[17808.515220] ? process_one_work+0x22b/0x720
[17808.515220] worker_thread+0x4b/0x4f0
[17808.515220] kthread+0x10f/0x150
[17808.515220] ? process_one_work+0x720/0x720
[17808.515220] ? kthread_create_on_node+0x40/0x40
[17808.515220] ret_from_fork+0x2e/0x40
[17808.528894] ---[ end trace 42fc4dedec8f6213 ]---
[17808.529383] ------------[ cut here ]------------
[17808.529760] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.530390] refcount_t: underflow; use-after-free.
[17808.530718] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.531354] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.531961] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.532329] Workqueue: btrfs-scrubwrc btrfs_scrubwrc_helper [btrfs]
[17808.532329] Call Trace:
[17808.532329] dump_stack+0x68/0x93
[17808.532329] __warn+0xcb/0xf0
[17808.532329] warn_slowpath_fmt+0x4f/0x60
[17808.532329] ? debug_object_deactivate+0x56/0x130
[17808.532329] refcount_sub_and_test+0x60/0x70
[17808.532329] refcount_dec_and_test+0x11/0x20
[17808.532329] scrub_page_put+0x16/0x40 [btrfs]
[17808.532329] scrub_wr_bio_end_io_worker+0x93/0xe0 [btrfs]
[17808.532329] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.532329] btrfs_scrubwrc_helper+0xe/0x10 [btrfs]
[17808.532329] process_one_work+0x2af/0x720
[17808.532329] ? process_one_work+0x22b/0x720
[17808.532329] worker_thread+0x4b/0x4f0
[17808.532329] kthread+0x10f/0x150
[17808.532329] ? process_one_work+0x720/0x720
[17808.538039] ? kthread_create_on_node+0x40/0x40
[17808.538039] ret_from_fork+0x2e/0x40
[17808.538830] ---[ end trace 42fc4dedec8f6214 ]---
[17808.539179] ------------[ cut here ]------------
[17808.539520] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.540144] refcount_t: underflow; use-after-free.
[17808.540472] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.541110] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.541704] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.542087] Workqueue: btrfs-scrubwrc btrfs_scrubwrc_helper [btrfs]
[17808.542087] Call Trace:
[17808.542087] dump_stack+0x68/0x93
[17808.542087] __warn+0xcb/0xf0
[17808.542087] warn_slowpath_fmt+0x4f/0x60
[17808.542087] ? debug_object_deactivate+0x56/0x130
[17808.542087] refcount_sub_and_test+0x60/0x70
[17808.542087] refcount_dec_and_test+0x11/0x20
[17808.542087] scrub_page_put+0x16/0x40 [btrfs]
[17808.542087] scrub_wr_bio_end_io_worker+0x93/0xe0 [btrfs]
[17808.542087] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.542087] btrfs_scrubwrc_helper+0xe/0x10 [btrfs]
[17808.542087] process_one_work+0x2af/0x720
[17808.542087] ? process_one_work+0x22b/0x720
[17808.542087] worker_thread+0x4b/0x4f0
[17808.542087] kthread+0x10f/0x150
[17808.542087] ? process_one_work+0x720/0x720
[17808.542087] ? kthread_create_on_node+0x40/0x40
[17808.542087] ret_from_fork+0x2e/0x40
[17808.548374] ---[ end trace 42fc4dedec8f6215 ]---
[17808.548700] ------------[ cut here ]------------
[17808.549074] WARNING: CPU: 1 PID: 9508 at lib/refcount.c:128 refcount_sub_and_test+0x60/0x70
[17808.549702] refcount_t: underflow; use-after-free.
[17808.550101] Modules linked in: btrfs(O) ext4 jbd2 mbcache xor raid6_pq netconsole xfs [last unloaded: btrfs]
[17808.550726] CPU: 1 PID: 9508 Comm: kworker/u4:6 Tainted: G W O 4.11.0-rc1+ #71
[17808.551079] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.1-20161122_114906-anatol 04/01/2014
[17808.551079] Workqueue: btrfs-scrubwrc btrfs_scrubwrc_helper [btrfs]
[17808.551079] Call Trace:
[17808.551079] dump_stack+0x68/0x93
[17808.551079] __warn+0xcb/0xf0
[17808.551079] warn_slowpath_fmt+0x4f/0x60
[17808.551079] ? debug_object_deactivate+0x56/0x130
[17808.551079] refcount_sub_and_test+0x60/0x70
[17808.551079] refcount_dec_and_test+0x11/0x20
[17808.551079] scrub_page_put+0x16/0x40 [btrfs]
[17808.551079] scrub_wr_bio_end_io_worker+0x93/0xe0 [btrfs]
[17808.551079] btrfs_scrubparity_helper+0xef/0x620 [btrfs]
[17808.551079] btrfs_scrubwrc_helper+0xe/0x10 [btrfs]
[17808.551079] process_one_work+0x2af/0x720
[17808.551079] ? process_one_work+0x22b/0x720
[17808.551079] worker_thread+0x4b/0x4f0
[17808.551079] kthread+0x10f/0x150
[17808.551079] ? process_one_work+0x720/0x720
[17808.551079] ? kthread_create_on_node+0x40/0x40
[17808.551079] ret_from_fork+0x2e/0x40
[17808.559351] ---[ end trace 42fc4dedec8f6216 ]---
[17809.477360] BTRFS info (device vdb9): dev_replace from /dev/vdb7 (devid 2) to /dev/vdb10 finished
[17810.229098] BTRFS info (device vdb9): dev_replace from /dev/vdb8 (devid 3) to /dev/vdb7 started
[17810.518980] BTRFS info (device vdb9): dev_replace from /dev/vdb8 (devid 3) to /dev/vdb7 finished