OK, you've misunderstood me, then. I'm not concerned about space
saving either. I'm also trying to avoid seeks. What I was getting at
was that if you pack the data together, then read-ahead will yield
more complete files, which translates to less transactions in the end.
I don't care about saving disc space in terms of fitting all your data
onto the disc: disc is cheap. If you run out of room, buy some more.
> : - do you see a benefit of tarfs over reiserfs
>
> First of all, I use "tar" to mean "a wad of files all jammed together,
> possibly with gaps". Thinking tar == tar(1) is going to lead you into
> details that I didn't mean. You caould use ar or gdbm or whatever,
> all I care is that one I/O gets you the whole wad.
OK, this leads me to think that you're focussed on getting all the
blocks for files in a directory in a contiguous chunk. Is this
correct?
> Anyway, if my understanding of reiserfs is correct, yes, I sure do
> see a benefit.
OK, we agree on this point (see, it can happen).
> The idea of making one I/O work by putting the data in the directory
> entries doesn't work when you limit the amount of data because there
> is no nice natural size where you would draw the line.
If you're referring to the proposed trick of putting data inside the
inode for ext2fs, I agree. That would help for really small files, but
wouldn't help for most files. It may be worth doing anyway, as it will
have a positive effect (say for a /bin/true shell script without the
copyright;-), but people shouldn't think it solves the general
problem.
> If resierfs could do stuff like "hmm, all the files in this
> directory put together are < 5MB total, I'll tar 'em up", then
> that's what I want.
Why is this tarring step important? If instead it puts all the inode
and data blocks for all the files in a 5 MByte contiguous chunk,
what's wrong with that?
Or do you see the "put inode & data in a chunk for all the files"
operation as "tar 'em up"? If that's the case, then we've been
violently agreeing but have confused each other with different
meanings for the same terms.
We need a glossary ;-)
> If resierfs was really, realy cool, it would have multiple tar files
> per directory - one tar file per uid.
How often do you get a mix of people writing to the same directory,
other than /tmp? From what I've seen, most people write to their home
directory or to personal directories on shared data discs. Only rarely
have I seen a common directory used for data (AIPS), and there the
files are huge anyway (10s or 100s of MBytes and up).
> : - a modest read-ahead (hundreds of kBytes) of the inode blocks will
> : halve the number of seeks, and will make the remaining seeks
> : generally less expensive (no need to seek between two different
> : areas on the disc)
>
> Bzzt. One I/O for the whole mess. Even if seeks were free, rotational
> delays are not and I/O transactions are not. I want one I/O for the
> whole directory.
Yes, agreed, but I wanted to establish a point, that the number of
seeks would be halved. I just wanted to check if you agreed with that
before going on to the next point:
> : - combining this with massive read-ahead (MBytes and more) would yield
> : very good performance for ext2fs as well as tarfs and reiserfs
>
> No, it wouldn't. It would only do so if all the files in a single
> directory were adjacent to each other. Which they typically aren't.
> Many file systems will make files contiguous, but very few (waffle
> is the only one I know of) will make files contguous to each other
> because they are in the same directory and written by the same user.
Is your core point, then, that all data blocks (and secondarily, all
inode blocks) for files in a directory are contiguous on the disc?
If so, then that's good/frightening, because that's also what I
advocate. I don't recall if reiserfs does this, but I hope it does.
> Remember, this has to work when you are untarring your crud and I am
> untarring my crud and we are both allocating from the bitmap/free
> list/ whatever at the same time. Waffle actually gets this right,
> or so I was told by Hitz when I asked.
You're mentioning untarring, but I presume you just mean that multiple
users are somehow writing files into a FS at the same time?
> : - discussions about the advantages of packing data can be equally well
> : addressed with sufficient read-ahead.
>
> No they bloody can't. Not unless the allocation policy is smart
> enough to sort things out while writing the data.
By that you mean all blocks for files in a directory are in one
contiguous chunk?
> : I have a suspicion that we may actually agree (at least on some
> : points), but we've been talking around each other.
>
> That's simply not possible, I don't agree with anyone :-)
I hope you at least agree with your SO ;-)
Regards,
Richard....
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/