Re: partially encrypted filesystem

[Erez Zadok]

In message <20031205112050.GA29975@xxxxxxxxxxxxxxxxxxxx>, =?iso-8859-1?Q?J=F6rn?= Engel writes:
> On Thu, 4 December 2003 14:41:48 -0500, Erez Zadok wrote:
> > 
> > Part of our stackable f/s project (FiST) includes a Gzipfs stackable
> > compression f/s.  There was a paper on it in Usenix 2001 and there's code in
> > the latest fistgen package.  See
> > http://www1.cs.columbia.edu/~ezk/research/fist/
> > 
> > Performance of Gzipfs is another matter, esp. for writes in the middle of
> > files. :-)
> 
> You don't seriously treat big files as a single gzip stream, do you?
> ;)

Of course not.  We're not *that* bad.  :-)

We compress each chunk separately; currently chunk==PAGE_CACHE_SIZE.  For
each file foo we keep an index file foo.idx that records the offsets in the
main file of where you might find the decompressed data for page N.  Then we
hook it up into the page read/write ops of the VFS.  It works great for the
most common file access patterns: small files, sequential/random reads, and
sequential writes.  But, it works poorly for random writes into large files,
b/c we have to decompress and re-compress the data past the point of
writing.  Our paper provides a lot of benchmarks results showing performance
and resulting space consumption under various scenarios.

We've got some ideas on how to improve performance for writes-in-the-middle,
but they may hurt performance for common cases.  Essentially we have to go
for some sort of O(log n)-like data structure, which'd make random writes
much better.  But since it may hurt performance for other access patterns,
we've been thinking about some way to support both modes and be able to
switch b/t the two modes on the fly (or at least let users "mark" a file as
one for which you'd expect a lot of random writes to happen).

If anyone has some comments or suggestions, we'd love to hear them.

BTW, our support in the stackable templates (described in the paper) is more
general than just for compression.  We support any (monotonically
continuous) size-changing algorithm: compression, encryption, etc.  In our
paper we also show a size-expanding file system called uuencodefs.

Cheers,
Erez.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/