Streams design draft, draft 0.01

In a recent discussion in IRC, the subject of streams was brought up again, and
it was mentioned that the problem was that everybody talked and nobody did
anything at all. So, I decided to propose a simple API for streams, which
pushes the details like tar/cp/cpio compatibility to userspace. I believe this
is the simpler and easier way to do it.

This is a draft for a draft for a unofficial standard for Unix-like streams (I
think that we need a rfc-like standard for such a touchy subject, and also I
have been reading too many RFCs lately). So, I believe there should be
discussion on this (like one would do for a real rfc) -- I'm sure I did
something bogus somewhere.


Kernel design

Inside the kernel, the streams would be implemented as a hidden magic directory
pointed to by a hidden field in the inode. If the filesystem supports streams,
the virtual streams directory is always there, even if it doesn't exist (that
is, a non-existing streams directory is represented by a hidden "virtual" dir
which doesn't get written to the disk, and removing the last stream removes the
directory from the disk). Of course the filesystem isn't required to represent
the streams in the disk that way.

Notes:
1. There is no $DATA stream. The "data" stream is the own main file. If you
   think otherwise, use a userspace wrapper.
2. There can be a $DATA stream, represented as a hardlink to the main file. If
   you try to open its streams directory, you get the same one you are in (to
   prevent infinite recursion, much like /'s ".." entry).
   This is filesystem-specific; I'd recommend NOT doing so unless the legacy
   design already has it.
3. Streams can have their own stream directories, if the filesystem designer
   was nuts enough to allow that.
4. Streams directories can have normal directories inside them. No, I have no
   idea of why that would be useful.
5. Streams directories can't have their own stream directories -- I know they
   should to keep things logic, but I fear the kernel would explode if that was
   allowed (programs should *NOT* depend on this one!)
6. No special files allowed. No sockets, fifos, devices, or other
   strangenesses. Of course no program should depend on that either.

RATIONALES:
1. Keep it simple
2. Allow all weird cases you can without special casing, or else some random
   will make a filesystem that needs that.
3. Avoid namespace confusions. The foo/bar design is horrible. Imagine you had
   a foo file and you untar a tarball which has a "foo/bar" entry -- the
   results would *not* be what you expected.

The kernel API is a couple of syscalls:

(RATIONALE: I don't want to use a single syscall to do everything under the sun
 using the first parameter as a sub-syscall number)

int openstreamdir (int fd);

- The fd argument is a fd which would be valid for calls like fstat.
  (RATIONALE: It makes sense to allow one to attach streams to a directory,
   even if no sane person would do that -- because someone in the future might.
   I believe that having the same restrictions as glibc's fstat is sane)
- The returned fd is almost like a fd you get from glibc's opendir
  (RATIONALE: avoiding syscall explosion)
- Doing something insane like trying to attach a streamdir to a directory (even
  the API not forbidding it) deserves a -EISDIR
  (RATIONALE: allow future expansion but disallow crazyness until you need it.
   Programs shouldn't depend on it)

int openstream (int fd, const char * filename, int flags, int mode);

- This one is like sys_open, with an extra fd argument (which is the same one
  you would pass to openstreamdir).
- The filename is parsed as if the streamdir was the root of the VFS. That is,
  foo, /foo and //foo are the same stream.
- flags and mode are normal (as normal as flags and mode can be)


Usermode design

(to be done later -- userspace isn't my area, but it should be easy)


Thanks to surf (Daniel Phillips) for bringing back the subject of streams and
for forcing me to create an argument against the foo/bar design.