Re: Mis-Design of Btrfs?

From: Ric Wheeler
Date: Fri Jul 15 2011 - 12:51:35 EST


On 07/15/2011 05:23 PM, david@xxxxxxx wrote:
On Fri, 15 Jul 2011, Chris Mason wrote:

Excerpts from Ric Wheeler's message of 2011-07-15 08:58:04 -0400:
On 07/15/2011 12:34 PM, Chris Mason wrote:

By bubble up I mean that if you have multiple layers capable of doing
retries, the lowest levels would retry first. Basically by the time we
get an -EIO_ALREADY_RETRIED we know there's nothing that lower level can
do to help.

the problem with doing this is that it can end up stalling the box for significant amounts of time while all the retries happen.

we already see this happening today where a disk read failure is retried multiple times by the disk, multiple times by the raid controller, and then multiple times by Linux, resulting is multi-minute stalls when you hit a disk error in some cases.

having the lower layers do the retries automatically runs the risk of making this even worse.

This needs to be able to be throttled by some layer that can see the entire picture (either by cutting off the retries after a number, after some time, or by spacing out the retries to allow other queries to get in and let the box do useful work in the meantime)

David Lang


That should not be an issue - we have a "fast fail" path for IO that should avoid retrying just for those reasons (i.e., for multi-path or when recovering a flaky drive).

This is not a scheme for unbounded retries. If you have a 3 disk mirror in RAID1, you would read the data no more than 2 extra times and almost never more than once. That should be *much* faster than the multiple-second long timeout you see when waiting for SCSI timeout to fire, etc.

Ric

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/