REAL reason for 2GB file size limit (was: Ok, making ready...)

Chuck Phillips (cdp@peakpeak.com)
Fri, 24 Dec 1999 01:02:15 -0700

tytso@mit.edu writes:
> You aren't allowed to create a file bigger than 2 Gigs since that would
> sometimes force lseek() to return a negative value, which would cause a
> POSIX violation. Reiserfs ignored this constraint, which is why they
> could offer 4 gig support --- if you try to create file that big, though,
> the results could violate POSIX if you called lseek() with a big file,
> and your application might do something surprising as a result.

NIT: The REAL limitation for POSIX file sizes on 32-bit architectures isn't
*directly* from POSIX, but rather, indirectly from ISO/ANSI C (which is
assumed by POSIX).

o In POSIX, the lseek() file offset is defined as off_t. There's nothing
to prevent a 32-bit POSIX-compliant implementation from typedef'ing off_t
as a 64-bit signed "long long".

o Similarly, ISO/ANSI C's fsetpos()/fgetpos() can be fixed by typedef'ing
fpos_t to be a 64-bit long long.

o The problem lies with ISO/ANSI C's fseek()/ftell(), which use a "long"
for the offset. Why fpos_t wasn't used (consistently!) is beyond me, but
hindsight is 20/20. AFAIK, these are the only two functions that *must*
break if >2GB files are permitted with 32-bit longs. Other functions
*can* break (like the present case), but aren't *required* to break by
the POSIX+C standards.

Thus, it is _ISO/ANSI C compatibility_ that is necessarily broken for file
systems that support >2GB files with 32-bit longs -- something that applies
even to non-POSIX file systems that claim compatibility with ISO/ANSI C.
I'm sure you all can think of at least two, but that's a discussion for the
advocacy lists. ;^)

This comes up often enough (and is misunderstood enough) it belongs in an
FAQ, but I don't know which one. Suggestions?

Regards,
Chuck

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/