Re: [PATCH] btrfs: raid56: fix use-after-free when waking up the rbio I/O waiter
From: Qu Wenruo
Date: Sun Nov 15 2026 - 18:57:28 EST
在 2026/11/16 09:53, Miroslav Cimerman 写道:
The end_io functions of the RAID56 read and write bios decrement
rbio->stripes_pending and call wake_up(&rbio->io_wait) when it drops
to zero. The submitter waits with wait_event(rbio->io_wait,
atomic_read(&rbio->stripes_pending) == 0).
wait_event() checks the condition before it sleeps, without the wait
queue's lock. If the submitter checks it after the last decrement and
before the wake_up(), it does not sleep, finishes the operation and
can drop the last reference to the rbio, which frees it. Even when
the submitter sleeps, the wake-up removes it from the wait queue, so
finish_wait() returns without taking the queue's lock while wake_up()
may still hold it. Either way, wake_up() then uses a wait queue in
freed memory, in interrupt context. On 6.12.117 (bio_endio() and the
end_io function are tail calls, so they do not show in the trace):
The analyze looks good to me. Although I'd prefer an ASCII chart showing the race window more clearly, e.g:
CPU0 (endio, irq) CPU1 (submitter, rmw_rbio)
raid_wait_write_end_io() rmw_rbio()
| |-submit_write_bios()
| | \-submit_bio()
| |
| |-wait_event()
|-bio_put() | ^^^ checks the condition without
| | taking io_wait.lock
|-atomic_dec_and_test() |
| ^^^ stripes_pending hits 0 |
| | ^^^ sees 0, doesn't sleep
| |
| \-rbio_orig_end_io()
| \-kfree()
| ^^^ last rbio reference, rbio
| freed
|
\-wake_up()
\-_raw_spin_lock_irqsave()
^^^ io_wait.lock is in freed memory
Too late.
And the waiter sleeps case:
CPU0 (endio, irq) CPU1 (submitter, rmw_rbio)
raid_wait_write_end_io() rmw_rbio()
| \-wait_event()
| |-prepare_to_wait_event()
| \-schedule()
|-atomic_dec_and_test()
| ^^^ stripes_pending hits 0
|
\-wake_up()
\-__wake_up_common_lock()
|-_raw_spin_lock_irqsave()
|-__wake_up_common()
| ^^^ autoremove_wake_function()
| takes the waiter off the
| queue and wakes CPU1 finish_wait()
| |-list_empty_careful()
| | ^^^ entry already removed,
| | returns without taking
| | io_wait.lock
| |
| \-rbio_orig_end_io()
| \-kfree()
| ^^^ rbio freed
|
\-_raw_spin_unlock_irqrestore()
^^^ io_wait.lock is in freed memory
Too late.
BUG: kernel NULL pointer dereference, address: 0000000000000000
RIP: 0010:__wake_up_common+0x25/0xa0
Call Trace:
<IRQ>
__wake_up+0x31/0x60
blk_update_request+0xfd/0x410
blk_mq_end_request+0x1d/0x1a0
Kernel panic - not syncing: Fatal exception in interrupt
With KASAN on v7.4-rc2 and a test-only delay between the decrement
and the wake-up:
BUG: KASAN: slab-use-after-free in _raw_spin_lock_irqsave+0x76/0xe0
__wake_up+0x1c/0x60
raid_wait_read_end_io+0x132/0x10f0
blk_update_request+0x21e/0xea0
Allocated by task 82:
alloc_rbio.isra.0+0xa8/0xbf0
raid56_parity_recover+0x10a/0x9c0
Freed by task 84:
kfree+0x154/0x420
rbio_orig_end_io+0x1c4/0x340
recover_rbio+0x393/0x590
Wait with a completion instead. complete() wakes the waiter under the
completion's lock and does not touch it afterwards, and
wait_for_completion() returns only after taking that lock, so the
rbio may be freed as soon as the wait returns. The parity scrub waits
for its writes only when finish_parity_scrub() submitted them.
Found with FuzzyRAID, a fault-injection tester for RAID.
Since it's not a public tool, I think you can skip this sentence.>
Fixes: d817ce35d24a ("btrfs: raid56: switch recovery path to a single function")
Fixes: 5eb30ee26fa4 ("btrfs: raid56: introduce the main entrance for RMW path")
Fixes: 6bfd0133bee2 ("btrfs: raid56: switch scrub path to use a single function")
Cc: stable@xxxxxxxxxxxxxxx # 6.2+
Signed-off-by: Miroslav Cimerman <mc@xxxxxxxxxxx>
Reviewed-by: Qu Wenruo <wqu@xxxxxxxx>
Just a nitpick below related the header.
[...]
diff --git a/fs/btrfs/raid56.h b/fs/btrfs/raid56.h
index 8700910b8dc9..6c82a975eb68 100644
--- a/fs/btrfs/raid56.h
+++ b/fs/btrfs/raid56.h
@@ -12,6 +12,7 @@
#include <linux/spinlock.h>
#include <linux/bio.h>
#include <linux/refcount.h>
+#include <linux/completion.h>
You can skip this header, "volumes.h" has already pulled in this header.
Thanks for catching and fixing the bug,
Qu
#include <linux/workqueue.h>
#include "volumes.h"
@@ -175,7 +176,8 @@ struct btrfs_raid_bio {
atomic_t stripes_pending;
- wait_queue_head_t io_wait;
+ /* Completed by the last end_io when stripes_pending drops to zero. */
+ struct completion io_done;
/* Bitmap to record which horizontal stripe has data */
unsigned long dbitmap;
base-commit: 3aefd67f08f8ed59ed0fa90619b25856df415031