Re: petabyte class archival filestore wanted/proposed

From: Jeff Anderson-Lee
Date: Thu Jun 22 2006 - 16:27:48 EST


Jeff Garzik wrote:

Jeff Anderson-Lee wrote:

I'm part of a project at University of California Berkeley that is trying to put together a predominantly archival file system for petabyte class data stores using Linux with clusters of commodity server hardware. We currently have multiple terabytes of hardware on top of which we intend to build such a system. However, our hope is that the end system would be useful for a wide range of users from someone with 3 large disk or three disk servers to groups with 3 or more distributed storage sites.

Main Goals/Features:
1) Tapeless: maintain multiple copies on disk (minimize backup/restore lag)
2) "Mirroring" across remote sites: for disaster recovery (we sit on top of the Hayward Fault)
3) Persistent snapshots: as archival copies instead of backup/restore scanning
4) Copy-On-Write: in support of snapshots/archives
5) Append-mostly log structured file system: make synchronization of remote mirrors easier (tail the log).
6) Avoid (insofar as possible) single point of failure and bottlenecks (for scalability)

I've looked into the existing file systems I know about, and none of them seem to fit the bill.

Parts of the Open Solaris ZFS file system looks interesting, except (a) it is not on Linux and (b) seems to mix together too many levels (volume manager and file system). I can see how using some of the concepts and implementing something like it on top of an append-mostly distributed logical device might work however. By splitting the project into two parts ((a) a robust, distributed logical block device and (b) a flexible file system with snapshots) it might make it easier to design and build.

Before we begin however, it is important to find out:
1) Is there anything sufficiently like this to either (a) use instead, or (b) start from.
2) Is there community support for insertion in the main kernel tree (without which it is just another toy project)?
3) Anyone care to join in (a) design, (b) implementation, or (c) testing?


I would recommend checking out Venti:
http://cm.bell-labs.com/sys/doc/venti.html

Yes, I've seen that and like some of the ideas. There is no GPL Linux implementation of Venti that I know of.

Jeff Anderson-Lee

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/