Re: ext2fs enhacement/fix [re: shrinking directories]

Ted Deppner (ted@jasmine.psyber.com)
Fri, 17 Jul 1998 09:47:12 -0700


On Fri, Jul 17, 1998 at 09:36:13AM +0800, David Luyer wrote:
> It's not the disk space - it's the RAM and CPU. Create a directory with
> 512,000 files with 8-character names in them sometime. Delete all
> except a dozen files in it. 'ls'. wait. wait. wait. See the problem.
> This is a problem on mail and news servers (both in spool directories).
> Things like seasonal or topical newsgroups, spammed mailing lists, etc,
> which might never bloat to the same size again but for some short time get
> to that size.

That makes some sense. For some period of time you could see dramatic
improvements even in cyclic areas, and for non cyclics you'd recover
performance and cpu cycles.

> Other things like 'open()' also get slowed down from memory.
>
> If we were shrinking when the directory size *halved*, a news server with
> decent expires time and frequent enough expires runs would only be shrinking
> seasonal/topical groups - the group size doesn't usually halve in one expire
> run.

I wonder if inodes simply being "packed" would be better (or not?) than a
whole pack/shrink cycle. You could impliment a pack pretty quickly and easily
when looking for your inode.
looking for my inode
found a hole (remember it)
....
found 50 or (n) holes -- I'll shove my inode when I find it in the
first hole.
....
inode found, mv to first hole.

That should be relativly quick and doesn't require an extra program or
thread (unless you want to regain that space at some point). (I don't know
if this would otherwise violate locking or handling and cause more problems.)

I was unable to test if this is feasible, I could detect no noticible speed
diff ACCESSING a directory with 50k entries vs only a dozen -- cached or not.

I did notice an extreeme slowdown creating files beyond the first 3k or so
per instance (the first 3k go quick regardless if there's a dozen in the dir,
or 50k).

I did verify though that creating 10k files in the 0 to 10k area is far
faster than in the 50 to 60k area.... inconclusive, but indiciative.

---
Ted Deppner
ted@psyber.com
http://www.psyber.com

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html