Kernel fs/ext3 dir_index internals

From: Stefan Knecht
Date: Fri Aug 14 2015 - 05:31:50 EST

Hello all

I would like to know if my understanding of the process is correct and
would be glad if someone who knows the internals could confirm or

I'm not a regular reader of the list, so please CC my email address in
any replies.

My question is about the internal mechanics of the directory index on
an ext3 file system. I understand the basics of how it works and what
its purpose is, but I encountered the following event which got me

We have a dir_index on ext3 in use on an EL5 system running
2.6.18-371.4.1, where we saw a warning in syslog of the index running
full on one directory. Listing the directory contents showed over 9.1
millions files, which was apparently enough to fill the index. We
deleted all the files, and checked the directory contents using "ls"
and it showed just 1 file. 15 or so minutes later, another check
showed another 9.1 million files, and another alert in syslog that the
directory index is full again. The files were not newly created, they
were old files (some 3 months old).

That prompted me to investigate. A couple email threads and posts on
sites such as serverfault later, I came to the following theory:

- if present, the index is used whenever the contents of a directory
are accessed
- if the index is full, it may not reflect what's actually in the directory
- once space in the index is freed, the index is refreshed / re-populated

This would mean that:

- "ls" would not show all the files in the directory - only what's
currently in the index
- a loop over the directory in PERL, such as this: perl -e
'for(<*>){((stat)[9]<(unlink))}' would only delete what's in the index

Is my understanding correct?

I'd appreciate some feedback as this has quite a few people puzzled.

Best regards and a good weekend!



Stefan Knecht

Solutions Architect

Pythian - Love your data


Phone: +1-866-798-4426x1331



To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at