Re: 2.4 Features

From: Theodore Y. Ts'o (tytso@MIT.EDU)
Date: Wed Feb 09 2000 - 15:00:17 EST


   Date: Wed, 9 Feb 2000 17:41:57 +0000 (GMT)
   From: Riley Williams <rhw@MemAlpha.CX>

   I don't by any means know all of the patches against the e2fs file
   system, but of the ones I do know, only e2comp is even close to being
   ready for inclusion in the standard kernel. However, it's long past
   time that e2comp was included therein in my opinion.

Well, see Al Viro's comments about the porting work needed to get it
ready for Linux 2.3.latest.

>> Note that the *reason* why most of these patches haven't been
>> merged into the e2fsprogs is that it's not yet 100% certain that
>> the format filesystem is stable yet.

   Ted: What exactly do you mean by "the format filesystem" ???

   If you are referring to "the format of the filesystem", then as far as
   the e2comp patch is concerned, that is essentially unchanged, as the
   ONLY change is that the file will take up less blocks than its
   UNCOMPRESSED size indicates. Were this not the case, then it would not
   have been possible to produce the patch in the first place.

I'm referring to the format of the filesystem change. E2compr makes
filesytem changes. Backwards compatible changes, yes, but filesystem
changes nonetheless.

Basically, I arbitrate ext2 format changes. If someone wants to use a
space in the ext2 inode, we need to make sure that it's a good use of a
very precious resource. For example, one of the ext2 extensions wanted
to use the space reserved for extended the uid to 32 bits. I suggested
that they not do that, so that when the 32 uid work got done, there was
space for it.

As another example, e2compr 0.3 used a number of extra fields in the
inode, and so I reviewed the format as suggeted by Peter Moulder, and
after some back and forth, we managed to keep the number of bits
reserved to e2compr to be 11 bits in i_flags.

If the *format* for e2compr is stable, then I'm much more willing to
include support it into the mainline e2fsprogs sources. (Of course,
then I will have code correctness issues. Currently the e2compr
e2fsprogs patches don't check the superblock compatibility flag before
they do there different thing. This is bad; they should only make
allowances for e2compr if the e2compr compatibility flag is set. That's
not a problem; I can fix most of that when I accept the patch. I also
don't want to support 0.3 users in the mainline, since it adds to the
grot and makes it harder to do long-term maintenance.)

Speaking of format, though, consider carefully. Before you merge into
the 2.3 tree, this is your last chance to drop grot that maybe *you*
don't want to support in the long term. Do you really want to support
all those many different compression algorithms, for example? It may be
painful to existing e2compr users if you have to force some fraction of
them to convert (although you could provide conversion programs), but
after it hits the mainline kernel, you'll *really* have a hard time
getting rid of some deprecated compression format.

The ability to add additional compression methods is good, for future
expansion --- but do you really expect everyone to use low these many
different types of compression algorithms? Each bit of kernel bloat
(and potential maintenance headache on your part) should be carefully
scrutinized and justified. This is your last chance to remove excess
code bloat!

>> If it turns out that some change is needed to more efficienctly
>> or robustly support ACL or Capabilities, there may not be
>> (almost certainly will not be) backwards compatibility with the
>> old format.

   Linus made it clear some time ago that ANY time a file system changed
   in ways that meant that it could no longer access older formats, it
   would be given a new name. That's the reason why ext became ext2 in
   the first place, and also why ext3 has its own name. As a result, that
   argument is a non-starter.

No. Ext3 is currently fully backwards compatible with ext2. Once you
unmount an ext3 filesystem cleanly, the journal bit is cleared, and a
standard ext2 filesystem code and standard ext2 utilities will work just
fine with the filesystem.

The reason why ext3 has its own name is more about making it easy to
have two different versions of the filesystem code in the tree at once;
one which is in production and is stable, and one which is in testing.
I generally use ext2dev as a module-loaded filesystem when I'm doing my
own testing, for example.

What I was talking about is the backwards incompatible changes. For
example, the change between e2compr 0.3 and e2compr 0.4. While that
sort of change is still a possibility, IMHO it shouldn't go into the
kernel, since we *don't* want to encourage widespread adoption of
something that we might need to change later. (And yes, we could put
lots of backwards compatibility support code into the kernel, but that
starts getting real ugly and it's a real maintenance headache).

   Better still, let's get it merged into e2fs where it belongs. That
   way, we can do precicely what Linus has said he wants: Get it in wide
   circulation so any problems can be located and ironed out.

If you're committed to spending the time necessary to port e2compr to
2.3.latest, and committed to tracking down the resulting bugs, great.
What would be a really bad thing would be if the code gets folded in,
and folks lose interest to fix the problems.

                                        - Ted

P.S. I just downloaded the e2compr patches, and some comments on the
actual e2compr code:

1) Suggestion: find the comment that's in French, and run it through
babelfish, and the put the comments in English in the ext2 code.
Long-term maintainability is your friend.

2) The code that checks for e2compr flags and does something different
if they are set should check to see if compression was enabled in the
superblock, and call ext2_error() if not. That way inodes that are
incorrectly set as using compression when the overall filesystem aren't
can be caught.

3) There is space to support at most 32 algorithms. Do you really need
to chew up that space with 9 different GZIP variants (for gzip -1
through gzip -9) and 3 different Lempel-Ziv variants? The gzip variants
are particularly interesting since you don't need to know the gzip level
to decompress, and it may be that a global system-wide gzip compression
level is sufficient --- at which point, you only need one gzip variant
instead of 9. (This is an example of a format change where now's the
time to make such changes if you're going to make them.)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Tue Feb 15 2000 - 21:00:15 EST