Conglomerate files - reiserfs (was: all sorts of things)

Hans Reiser (reiser@ceic.com)
Wed, 30 Jun 1999 16:09:15 +0000 (/etc/localtime)


Wesley Terpstra writes:
> Do I have this right?: You want to make the fs capable of providing
> structured data storage to application developers. You plan to accomplish
> this by making an extension to reiserfs which allows a directory/file
> structure to be read as a file for purposes like copying by the user and
> opened as a directory by the application to modify the innards.
>
> I really like this idea! I know I spend far too much time choosing an
> easily extensible structure for my records and how to
> organize/move/defragment/etc.
>
> However, what kind of disk space overhead am I looking at per file/dir?

If your data is dynamic in size, then less than you get with a custom
implemented method, see http://www.devlinux.org/namesys, we pack small
files into a single block efficiently.

>
> Why do we need the glob ability? If there is an efficient FS for the
> overhead problem, why not just use the FS as is and have the user use tar?

If /etc/passwd is a directory of one line files, it could be useful to
edit the /etc/passwd directory in one emacs buffer.

>
> If that's too much for the user to handle, chances are that they use a
> GUI. Then that could be extended to automatically treat directories with
> some extension as an object which it tars and untars as necessary.

This belongs in the namespace not the gui. It should not be dependent
on the existence of xwindows.

>
> Or if the file is not going to grow, mount a file with loopback?
>
> If you're set on writing this (as I suspect you are), may I suggest
> implementing some sort scheme that converts your dir/file hybrid
> automatically to/from a tarballish file for non-extended file-systems.
>
> Have some semi-portable flag on the tarballish file that identifies it as
> a dir/file thing. Then whenever a tarballish thing this is copied to an fs
> that supports your extension, untar it to a file/dir thing transparently.
> When you open the file/dir thing as a file, return a tarballish thing with
> the semi-portable flag. The flag could be anything - perhaps a magic
> number as the first few bytes.
>
I think it needs to be the reverse, it is a file/dir in my filesystem,
and a tarballish thing elsewhere.

> I admit this has problems since files are written with an open then write
> sequence and you would like to know what the file is at the open stage.
>
> However, it occures to me that this could be done on a file close.
>
> For instance, I have a tarballish file which origonally came from an fs
> supporting your extension, but is now on a plain old ext2fs. It has some
> magic number header. I initiate any sort of copying technique to the
> extended fs. After I close the file your fs extension notices the header
> now has the magic number and untars the file.
>
> I admit this is 2x slower than need be (write as a file - then write as a
> dir/file thing), but it is an easy way to implement this as a first step.
>
> When copied off the fs, it is accessed as a file and you do the conversion
> to the tarballish thing transparently as you planned.
>
> Alternately, watch for writes to a pure file at offset 0 and catch the
> magic header and then initiate the change. With good write caching this
> could be done without a single wasted seek except under the most bizare
> circumstances (cp writing from end to start).
>
> Off-topic: In a project I am about to start I wanted to know if there is a
> way one can provide good (well perfect and race free :-) ) file-locking
> with only access to a common block device.
>
> For example computer A shares a block device that is accessed by computers
> B, C, and D. They all access a file-system on the block device. Can
> locking be implemented with only read/writes to the disk? Do any existing
> file-systems do this? Are there any strategies to avoid problems from
> caching reads or caching writes? Do any existing file-systems do this?

Not my area, but I am sure you can make it work. Worry about caching on
the controllers. I think there are some controllers with protocols for
sharing disk drives, I really don't know much about it.

>
> Sorry about the side-track, but I thought you might know.
>
> ---
> E-mail: terpstra@interchange.ubc.ca Host: iota.dhs.org
> Phone #: 1-604-221-8018, voice-mail: 1-604-221-8087

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/