Re: Unicode conversion issue

[Linus Torvalds]

On Wed, 11 Dec 2024 at 11:46, Gabriel Krisman Bertazi <krisman@xxxxxxx> wrote:
>
> 1) The first is to revert the patch and fix the original issue in a
> different way.  That would be: We would restore the original database
> and treat Ignorable codepoints as folding to themselves only when doing
> string comparisons, but not when calculating hashes.  This way, the hash
> will be the same, but filenames with Ignorable codepoints will be
> handled as byte sequences.

The problem is that all the filesystems basically do some variation of

        if (IS_CASEFOLDED(dir) ..) {

                len = utf8_casefold(sb->s_encoding, orig_name,
                        new_name, MAXLEN);

and then they use that "new_name" for both hashing and for comparisons.

Which is really really annoying, not just because you hash all these
meaningless conversions (aka "I write buggy crap and call it a
filesystem"), but it also means that now your *comparison* has to deal
with the fact that the name you are comparing against isn't the
original name, so the code in generic_ci_match() tyhat says "let's try
the simple exact match first" will fail too, because the string
comparison isn't comparing the original raw data - that almost
certainly matched.

Because even on a casefolding filesystem, 99.9% of the time you give
the *matching* name (think extremely common things like "readdir ->
stat" that is done by not just "ls -l", but *any* filesystem tree
traveral).

Soi the whole "let's corrupt the filename and then hash and compare
that corrupted version" is doubly wrong.

Christ. I really really hate case-folding filesystems with a passion.
The incompetence just fills me with red-hot rage.

               Linus