Re: [patch] ext2/3: document conditions when reliable operation is possible

[david]

On Wed, 26 Aug 2009, Neil Brown wrote:

> On Monday August 24, greg.freemyer@xxxxxxxxx wrote:
> > > +Don't damage the old data on a failed write (ATOMIC-WRITES)
> > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > +
> > > +Either whole sector is correctly written or nothing is written during
> > > +powerfail.
> > > +
> > > +       Because RAM tends to fail faster than rest of system during
> > > +       powerfail, special hw killing DMA transfers may be necessary;
> > > +       otherwise, disks may write garbage during powerfail.
> > > +       This may be quite common on generic PC machines.
> > > +
> > > +       Note that atomic write is very hard to guarantee for RAID-4/5/6,
> > > +       because it needs to write both changed data, and parity, to
> > > +       different disks. (But it will only really show up in degraded mode).
> > > +       UPS for RAID array should help.
> >
> > Can someone clarify if this is true in raid-6 with just a single disk
> > failure?  I don't see why it would be.
>
> It does affect raid6 with a single drive missing.
>
> After an unclean shutdown you cannot trust any Parity block as it
> is possible that some of the blocks in the stripe have been updated,
> but others have not.  So you must assume that all parity blocks are
> wrong and update them.  If you have a missing disk you cannot do that.
>
> To take a more concrete example, imagine a 5 device RAID6 with
> 3 data blocks D0 D1 D2 as well a P and Q on some stripe.
> Suppose that we crashed while updating D0, which would have involved
> writing out D0, P and Q.
> On restart, suppose D2 is missing. It is possible that 0, 1, 2, or 3
> of D0, P and Q have been updated and the others not.
> We can try to recompute D2 from D0 D1 and P, from
> D0 P and Q or from D1, P and Q.
>
> We could conceivably try each of those and if they all produce the
> same result we might be confident of it.
> If two produced the same result and the other was different we could
> use a voting process to choose the 'best'.  And in this particular
> case I think that would work.  If 0 or 3 had been updates, all would
> be the same.  If only 1 was updated, then the combinations that
> exclude it will match.  If 2 were updated, then the combinations that
> exclude the non-updated block will match.
>
> But if both D0 and D1 were being updated I think there would be too
> many combinations and it would be very possibly that all three
> computed values for D2 would be different.
>
> So yes: a singly degraded RAID6 cannot promise no data corruption
> after an unclean shutdown.  That is why "mdadm" will not assemble such
> an array unless you use "--force" to acknowledge that there has been a
> problem.

thanks for this detail, I would not have expected a partially degraded 
raid 6 array to be this sensitive to problems.

assuming that the degradation happens prior to the power failure, what 
could be done to make this safer and more predictable.

off the top of my head (and possibly an extreme performance hit, not 
nessasarily suitable for everyone) is there something that could be done 
with ordering the writes to the various drives?

David Lang