Re: [ANNOUNCEMENT PATCH COW] proof of concept impementation of cowlinks

[Denis Vlasenko]

On Thursday 06 May 2004 16:17, Jörn Engel wrote:
> Couldn't sleep last night and finished a first complete version of
> cowlinks, code-named MAD COW.  It is still based on the stupid old
> design with a flag to distinguish between regular hard links and
> cowlinks.  Please don't comment on that design, it's just a proof of
> concept.
>
> Patches are against 2.6.5 but most things should apply to other 2.6
> kernel without too much trouble.
>
> 1 generic_sendpage	- allow sendfile with ext[23] files as target
> 2 sendfile		- introduce vfs_sendfile for in-kernel use
> 3 copyfile		- new copyfile() system call
> 4 lock_flags		- old cruft, just ignore it
> 5 madcow		- the MAD COW itself
>
> Patches 1-3 will stay, 4 will be remove and 5 replaced by a better
> design over time.  I've also set up a webpage for it:
> http://wohnheim.fh-wedel.de/~joern/cowlink/
>
> Maybe that should be converted into a wiki so someone with better
> taste than myself can improve it.

Hi,

Glad to see this happening. My filesystems are mostly reiser
these days, but if I have some time/occasion, I will try your patch.

Regarding semantics etc. I have read http://lwn.net/Articles/77972/.
I think most difficulties arose from inode numbers as a concept.

Usage of inode number is a historic UNIX misfeature.
AFAIK it is almost exclusively used for determining whether
two files are really the same: same (dev,ino) => same file.
Usage:
* diff wants this to avoid diffing file against itself.
* du wants this in order to not count file twice.
* cp/tar wants this in order to preserve hardlinks.
For cowlinks it is a bit different:
* diff still dont need to compare cowlinked file.
* cp/tar aren't required to detect cowlinks, may do this
  as an optimization.
* du: I am not sure... analogous to sparse files?

In essense, diff wants to ask kernel "are these files have
identical contents?" while tar/cp ask "are these files
hardlinked together?". du asks "are these share storage"?

Original UNIX folks had to make inode number only an advisory
indicator, meaning "if ino1 != ino2, files are _definitely_ not
linked", and a syscall is_hardlinked(fd1,fd2), intended
for use when inodes are equal.

That is probably unfixable now, but you can avoid making similar
error. Provide is_cowlinked(fd1,fd2) syscall. Pity you will
have to use different inode numbers for cowlinks (due to tar/cp),
and this won't fly:

diff.c:
...
if(ino1==ino2 && is_cowlinked(fd1,fd2))
	skip_diff();
...

diff will have to use if(is_cowlinked(fd1,fd2)), i.e. one
extra syscall, always. Anyway, for diff that's not tragic,
savings from not doing diff are still very substantial.
For du, that would hurt.

Per-block cow filesystems will open another can of worms
for diff and du. ;)

P.S. If is_hardlinked() is introduced too, tar/cp can be
updated to use that, and we can slowly drift into right
direction (of not assuming "equal inos => hardlinked file").
I think too much people will disagree with me, and this
won't happen.
--
vda

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/