Re: Suggestion: a garbage-collected file system

From: David Madore (madore@clipper.ens.fr)
Date: Wed Jan 12 2000 - 17:41:40 EST


Towers, Tim (London) in litteris (linux-kernel:12131) scripsit :
> My first thoughts on a gc filesystem would be a tie into the unlink
> call, when the FS came back with ENOSPACE (or whatever) the inodes
> marked for unlinking would be reclaimed in some order and the write
> retried without the application having to know about it. That could
> be carried out by a daemon in userspace, but there are other problems:

I think we have very different visions of what a garbage-collected
filesystem would be.

I realize I should have been clearer in my initial post as to what the
goal is and what the consequences are. I am going to write a web
document explaining things in more detail and post its URL on
linux-kernel and linux-fsdevel soon. But maybe I should clear one
possible misunderstanding right now:

My idea is not to make a somehow more efficient Unixy filesystem. I
aim at something more profound: generalize the filesystem semantics
(or relax the constraints, if you prefer); and it so happens that some
of these generalizations (such as hard links on directories, arbitrary
moves on directories) require the replacement of the traditional
reference-counting way of keeping track of garbage by more general
methods.

> Following the huge digression you started with discussing the linking
> of directories (which I see as a red herring, and un-unixy to boot)...

As far as I'm concerned, this is not a digression, it is the whole
point.

It is un-unixy, I guess, but I think here we are witnessing one of the
perverse effects of Unix, namely that it has got us so accustomed to
thinking in terms of tree-like hierarchies rather than more general
structures that we find these more general structures unnatural.

It would also break a lot of things, admittedly. I'm mostly concerned
with trying this for the fun (and to see how far it would break
things), not for anything serious. But note that there are already a
lot of wrong assumptions floating around on filesystem semantics in
the first place. For example, notice how after a "rmdir ." a lot of
things start failing randomly...

(Actually, a curious point: does anyone know why Solaris returns
EINVAL when a process tries to unlink its current directory? This is
completely gratuitous. Fortunately Linux doesn't try to forbid this.)

> I think I can see where Sean maybe has the wrong end of the stick.
> I don't think you can get rid of '..' or reference counts but handling
> them would be tricky:

Indeed.

> A move is regarded as a relink and unlink - but with the garbage-collected
> file system the unlink wouldn't take place until later (when rules allow).
> so...

IMHO, you are confusing the unlink with the actual reclaim (garbage
collection). The unlink takes place immediately in the sense that the
directory entry ceases to be visible. The reclaim takes place later
on a garbage-collected filesystem.

>> mkdir a
>> mkdir a/b
>> ln a/b c
> now b and c are the same directory
>> mkdir c/d
>> mv a c/d
> I understand you to mean a link/copy of 'a' will be put into a/b/d
> but the root directory will still contain 'a' because it hasn't
> become empty.

My vision of the semantics is to allow removing non empty directories.

> If you 'ln -s . foo' currently, and export it though
> samba to a NT box you also get an infinite loop.
> Using loopback mounts inside an exported NFS
> mount can also cause infinite loops.

Indeed.

> What saves us in normal UNIX is that if this is done using symlinks
> there are a maximum number of symlink traversals carried out when
> resolving a path, when you use them up it pops out with an error.
> You may find that trying to keep track of how many times you've
> been through an individual directory is too time-consuming (list
> of [inode,count]?).
>
> More importantly though, applications can currently choose whether
> they see a link or resolve it - which is an option they will lose if
> all directories are equivalent. This currently allows applications to
> avoid recursion by avoiding all symlink traversals (e.g. tar)

Very good points. Yes, changing the filesystem semantics is likely to
break a lot of things. Some things can maybe be fixed, others cannot
and are just incompatible with the semantics I want. But I think this
is by no means a reason not to experiment with these semantics.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Sat Jan 15 2000 - 21:00:21 EST