Re: file as a directory

From: Hans Reiser
Date: Thu Dec 16 2004 - 13:56:30 EST

David Masover wrote:

Hans Reiser wrote:
| Explain the value of caching executable output more please.

If executables become "plugins" (see below), there is a performance loss
to be made up by caching. For instance, if we have an executable which
calls "tar", which we attach to a tarball, we want to cache the
extracted files.

This requires writing the plugin to have explicit caching of output. It could be quite interesting actually.

It's even more urgent when people start writing
perl/python/ruby executables to deal with system configuration files,
such as /etc/passwd. It's allright for passwd to take half a second to
generate, so long as we cache that output, and thus only spend the half
second when passwd is modified, instead of every time a user logs in.
|> | The component objects themselves could be full objects, so they
|> | themselves could have sub-components.
|> Right.
|> Also, there should be an inverse. For instance, a file-as-directory type
|> object should have a "contents" object, usually a normal directory, but
|> which could conceivably be any type of object, including a code-ish
|> object which implements a filesystem. Accessing foo/ would be the same
|> as accessing foo/.../contents, only because "..." (or whatever we use
|> for meta-files) is outside the actual directory namespace,
|> foo/.../contents/... refers to the metas of object "contents", which are
|> different than the metas of object "foo".
| Are you sure that having more than one "...." is needed? Hmmm,
| interesting, must think about it.

Yes. foo is a file, foo/contents is another file (an executable). But
maybe the executable has contents too -- with an extra "...", you can do
things like "foo/.../contents/.../contents", where the first "contents"
controls what you get from "ls foo/", and the second "contents" controls
what you get from "ls foo/.../contents/".

|> These two steps essentially create userspace "plugins", and do away with
|> having to mount other kernel layers such as lufs (or whatever its
|> current implementation is).
| I don't follow this point above.

lufs is (or was) a kernel filesystem which talks to a user daemon. The
daemon does all the work, so you don't have to do things like windows
emulation in the kernel in order to get captive-ntfs to work.

The reason we don't do that for the whole filesystem is performance --
after all, userland things are easier to write, debug, upgrade, and
install/use than kernel things.

If an executable can generate a directory listing or a whole directory
structure, some files, and so on, then we only need a few reiser4 kernel
plugins, and these userland executables can implement all the
functionality of most kernel plugins people have thought of so far.

The only problem is, it would be much slower than a kernel plugin.
Caching solves that.

Of course, someone still might find a situation where people really need
to create a new kernel plugin, and that would be allowed. But I doubt
it would help that much, or we'd all be using TUX instead of Apache.


|> I think the issues with directory-as-a-file were the same problems you
|> get when you allow hardlinked directories -- that you'd eventually have
|> to ditch reference counting for a garbage collector, which is hellishly
|> impractical. I don't have a solution to that, other than dropping
|> hardlinks entirely.
| This issue is overblown. Other fields have solved this problem.

Nevertheless, it's an issue that I don't have the skills to solve, and
one that must be solved if we are ever to implement these concepts.

If you have a solution, I'd love to hear it.

My solution is to tell Nate Diller that there is extensive literature on it, that it isn't really the big problem it is made out to be, and leave it to him to go read the literature and code it up after he finishes the required tasks of our current darpa contract.;-)
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at