Re: Reading the same block via partition and non-partitioned device gives different content

[NeilBrown]

On Tue, Jan 05 2016, Andrei Borzenkov wrote:

> [Please Cc me on reply; thank you]
>
> QEMU KVM virtual machine with openSUSE Tumbleweed (kernel
> 4.3.3-3-default); MD RAID1 with 1.2 metadata on /dev/vdb1 and /dev/vdc1.
>
> If I do
>
> mdadm /dev/mdX --fail /dev/vdb1
> mdadm /dev/mdX --add /dev/vdd1
>
> and wait for synchronization to finish and then look directly on on-disk
> suportblock, I see different content whether I read from /dev/vdd or
> from /dev/vdd1.
>
> /dev/vdd1 claims new disk is still spare (i.e. it is apparently
> immediately after adding it).
>
> The *same* superblock when read from /dev/vdd (of course with
> appropriate offset) correctly marks /dev/vdd1 as RAID member. The same
> content also seen when looking from host.
>
> I hit this when debugging problem with grub2 that scans devices for
> Linux MD (it is using the same code both at boot and run time). It
> skipped replacement disk because it believed disk was spare.
>
> Is it expected behavior? Note that if we now replace /dev/vdc1 with
> something else we have "wrong" superblock on both partitions so grub
> fails to find array completely. Fortunately this is transient state, but
> it also means it is impossible to reconfigure grub until reboot.
> Alternatively shutting down and restarting array cleats it as well.
>
> Any trick we can use to force content to be the same in this state?

blockdev --flushbufs /dev/vdd*

or use O_DIRECT to access the devices.

NeilBrown

Attachment: signature.asc
Description: PGP signature