Re: partially encrypted filesystem

From: Phillip Lougher
Date: Fri Dec 05 2003 - 20:00:51 EST

Matthew Wilcox wrote:
On Fri, Dec 05, 2003 at 02:47:56PM -0500, Erez Zadok wrote:

Thanks for the info, Matthew. Yes, clearly a scheme that keeps some "holes"
in compressed files can help; one of our ideas was to leave sparse holes
every N blocks, exactly for this kind of expansion, and to update the index
file's format to record where the spaces are (so we can efficiently
calculate how many holes we need to consume upon a new write).

FYI, Acorn's scheme was described in "Compressed Executables: An Exercise in Thinking Small" by Mark Taunton, in the Usenix Spring '91 conference, it doesn't seem to be online, but a search on google groups for "group:comp.unix.internals taunton compressed executables" brings up a description. I used to work with Mark Taunton at Acorn.

But the genius is that you don't need to calculate anything. If the
data block turns out to be incompressible (those damn .tar.bz2s!), you
just write the block in-place. If it is compressible, you write as much
into that block's entry as you need and leave a gap. The underlying
file system doesn't write any data there. There's no need for an index
file -- you know exactly where to start reading each block.

Of course this is all being done at the file level, which relies on proper support of holes in the underlying filesystem (which Acorn's BSD FFS filesystem did). FiST's scheme is much more how it would be implemented without hole support, where you *have* to pack the data, otherwise the "unused" space would physically consume disk blocks. In this case an index to find the start of each compressed block is essential.

I'm guessing that FiST lacks support for holes or data insertion in the filesystem model, which explains why on writing to the middle of a file, the entire file from that point has to be re-written.

Of course, all this is at the logical file level, and ignores the physical blocks on disk. All filesystems assume physical data blocks can be updated in place. With compression it is possible a new physical block has to be found, especially if blocks are highly packed and not aligned to block boundaries. I expect this is at least partially why JFFS2 is a log structured filesystem.


To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at