Re: [ANNOUNCE] Darkstar Development Project

From: Jamie Lokier (lk@tantalophile.demon.co.uk)
Date: Tue Sep 12 2000 - 04:53:16 EST


Larry McVoy wrote:
> No matter what you do with rsync, there is no bloody way you can even
> come close to a single 32K disk read and then a 6K over the wire transfer.
> At least, I can't think of one, can you?

rsync will transfer a bunch of file modification times and, where they
don't match, a bunch of checksums. This is still a few bytes of state
for every file.

The obvious extension to rsync would be to make the rolling checksum
hierarchical -- successively approximating smaller and smaller regions,
and treating the whole set of files as a single data stream for the
algorithm. "Null update" would then always reduce to transferring a
single cryptographic-level checksum -- better than 6K and still works if
you lose the Changeset file. :-)

> The fundemental observation is as the tree size/age grows, the amount of
> change you make to it stays relatively constant but the updates grow with
> tree size. One human can only make so much change, but many can make a
> lot. BK takes advantage of that and does the hard work when you do hard
> work, not every time you update.

Yes that's a good thing.

-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Sep 15 2000 - 21:00:17 EST