Re: [PATCH] md/raid5: fix wrong read data on RAID-4/5 with two failed devices
From: yu kuai
Date: Fri Nov 20 2026 - 23:01:33 EST
在 2026/11/15 20:38, Miroslav Cimerman 写道:
> On RAID-4/5, a read of a block on a working member can return wrong
> data, without reading the member, when the stripe has two failed
> devices: the array has failed (MD_BROKEN only stops writes), or it is
> degraded and another member has a bad block in the stripe, for
> example after a write to it failed.
>
> fetch_block() computes two missing blocks at once when all but two
> blocks of the stripe are up to date and two devices have failed. That
> needs P and Q, but nothing limits the branch to RAID-6. On RAID-4/5,
> ops_run_compute5() computes sh->ops.target as the XOR of all the other
> blocks, including the second target, which was never read, and
> ops_complete_compute() marks both targets up to date. The read returns
> the XOR of the cached blocks and the unread page, and a later read of
> the second target returns that page instead of an error. The page is
> not cleared, so it can hold data of another block.
>
> Commit 93b3dbce6456 ("md/raid5: unite fetch_block5 and fetch_block6")
> brought the branch to RAID-4/5. fetch_block5() computed only one block.
>
> Take the branch on RAID-6 only, with the sh->qd_idx >= 0 test that the
> branch above already uses. On RAID-4/5 the block is then read from its
> member, and reads of failed members are failed by
> handle_failed_stripe() as before.
>
> For example, take a degraded 4-device RAID-5 (left-symmetric, 4 KiB
> chunks, device 1 missing) where block v holds the byte 0x10 + v. A
> write to block 3, on device 3, does a read-modify-write. Its write to
> device 3 fails: md records a bad block, keeps the device and fails the
> write. A read of block 4, on device 0, then returns 0x14 ^ 0x15 ^ the
> unread page (0x01 if it is zero) instead of 0x14, and a read of block
> 5 returns the unread page instead of EIO.
>
> Found with FuzzyRAID, a fault-injection tester for RAID.
>
> Fixes: 93b3dbce6456 ("md/raid5: unite fetch_block5 and fetch_block6")
> Cc:stable@xxxxxxxxxxxxxxx
> Signed-off-by: Miroslav Cimerman<mc@xxxxxxxxxxx>
> ---
> FuzzyRAID is not public yet. The analysis, the fix, the commit message
> and the reproducer were written with the help of an LLM. I reviewed
> everything and ran the reproducer on md-7.4 with and without this
> patch.
>
> Reproducer (needs only standard tools):
>
> #!/bin/bash
> # md RAID 5 with two failed devices for a stripe: a read of a block on a
> # working member returns wrong data, without reading the member, when exactly
> # disks-2 blocks of the stripe are cached.
> # Needs root, mdadm, losetup and GNU coreutils; uses /dev/md99 and 3 loop devices.
> set -e
> D=$(mktemp -d)
> for i in 0 2 3; do
> truncate -s 64M $D/m$i
> L[i]=$(losetup -f --show $D/m$i)
> done
> blk() { head -c 4096 /dev/zero | tr '\0' "\\$(printf %o $1)"; }
> for v in $(seq 0 11); do blk $((0x10 + v)); done > $D/pattern # block v = 0x10+v
> blk 0xaa > $D/aa
>
> # degraded 4-device RAID 5, 4 KiB chunks;
> # stripe 1: block 3 on m3, block 4 on m0, block 5 on m1 (missing), P on m2
> mdadm -C /dev/md99 -R -l5 -n4 -c4 --assume-clean --bitmap=none \
> ${L[0]} missing ${L[2]} ${L[3]}
> dd if=$D/pattern of=/dev/md99 bs=48k oflag=direct status=none
> mdadm -S /dev/md99 # empty the stripe cache
> mdadm -A -R /dev/md99 ${L[0]} ${L[2]} ${L[3]}
>
> # read-modify-write of block 3: caches block 3 and P only
> dd if=$D/aa of=/dev/md99 bs=4k seek=3 oflag=direct status=none
> # second failure; md fails m3 and returns EBUSY, the array has failed
> echo faulty >/sys/block/md99/md/dev-${L[3]#/dev/}/state 2>/dev/null || true
>
> rd() { dd if=/dev/md99 bs=4k skip=$1 count=1 iflag=direct status=none | od -An -tx1 -N1 | tr -d " "; }
> echo "block 4 (m0, working), expected 14: $(rd 4)"
> echo "block 5 (m1, missing), expected I/O error: $(rd 5 2>&1)"
>
> mdadm -S /dev/md99
> for i in 0 2 3; do losetup -d ${L[i]}; rm $D/m$i; done
> rm $D/pattern $D/aa; rmdir $D
>
> drivers/md/raid5.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
Applied
--
Thanks,
Kuai