Re: [dm-devel] Barriers still not passing on simple dm devices...

From: Ric Wheeler
Date: Thu Apr 09 2009 - 06:48:51 EST


Mikulas Patocka wrote:
And I will restate that back at EMC we tested the original barriers (with
reiserfs mostly, a bit on ext3 and ext2) and saw significant reduction in file
system integrity issues after power loss.

You saw that barrier-enabled filesystem was worse than the same filesystem without barriers? And what kind of issues were that? Disks writing damaged sectors if powered-off in the middle of the writes? Or data corruptions due to bugs in ReiserFS?

No - I was not being clear. We saw a reduction in issues which is a confusing way to say that it was significantly better with barriers enabled, for both ext3 & reiserfs.

The vantage point I had at EMC while testing and deploying the original
barrier work done by Jens and Chris was pretty unique - full ability to do
root cause failures of any component when really needed, a huge installed base
which could send information home on a regular basis about crashes/fsck
instances/etc and the ability (with customer permission) to dial into any box
and diagnose issues remotely. Not to mention access to drive vendors to
pressure them to make the flushes more robust. The application was also able
to validate that all acknowledged writes were consistent.

Barriers do work as we have them, but as others have mentioned, it is not a
"free" win - fsync will actually move your data safely out to persistent
storage for a huge percentage of real users (including every ATA/S-ATA and SAS
drive I was able to test). The file systems I monitored in production use
without barriers were much less reliable.

With write cache or without write cache?
Write cache enabled.

Barriers are off when write cache is disabled - we probe the drives write cache and enable barriers at mount time if and only if the barriers are on.
With cache and without barriers the system is violating the specification. There just could be data corruption ... and it will eventually happen.

If you got corruption without cache and without barriers, there's a bug and it needs to be investigated.

As others have noted, some storage does not need barriers or flushed (high end
arrays, drives with no volatile write cache) and some need it but stink (low
cost USB flash sticks?) so warning is a good thing to do...

ric

Mikulas

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/