Re: How to increat [sic.] max open files - some answers
H. Peter Anvin (hpa@transmeta.com)
6 Jan 1997 11:21:03 GMT
Followup to: <Pine.LNX.3.95.970103181041.308A-100000@nextd.demon.co.uk>
By author: markhe@nextd.demon.co.uk (Mark Hemment)
In newsgroup: linux.dev.kernel
>
> This is a little long winded, but bear with me and I'll try to reveal
> all :)
>
> Lets clear up one BIG misunderstanding;
> FD_SETSIZE <= sysconf(_SC_OPEN_MAX)
> Note the '<' part! (for those who write non-portable code, sysconf(...) =
> NR_OPEN).
>
> OK, under Linux FD_SETSIZE does equal the max num of open files. But try
> it out on UnixWare/Solaries (and other SVR4 derived systems). XPG/4
> states select() only supports fds upto FD_SETSIZE.
>
> So how to you select on an fd greater than FD_SETSIZE?
> With a simple programming trick. 'Reserve' some low-numbered fds when the
> app starts (open() /dev/zero and dup() it a few times). Implement a
> wrapper for select() which does a bit magic to 'map' high-numbered fds to
> the reserved low ones, and then make the system call.
> This works fine when the number of descriptors to be select()d is not
> greater than FD_SETSIZE. If it is greater, then (on most UNIX OSes) the
> poll() system call can be used (which has no limits).
> There is no need for re-compiling libc when Linux supports a v. large
> number of file descriptors - just re-coding of apps. (sorry, but you
> would need to make the same changes if you wish your code to be portable).
>
The problem is not with select()! select() can handle these things
just fine, since the first parameter is can be treated by the kernel
as the maximum size to support. The BSD people did that part
correctly.
The problem is, instead, with the auxilliary support functions, in
particular with the definition of fd_set. Unlike, for example, FILE,
which is only user-visible as a pointer, so that tons of programs
won't have a preconcieved notion of the size of the object, in the
stardard API the user program handles allocation of fd_set's, and has
to know the size. This is where BSD blew it, and blew it big time.
Since it is standard practice for programs to do things like:
fd_set fooset;
int fd;
fd = open(...);
FD_SET(fd, &fooset);
... if open() can return a descriptor larger than the size of the set,
you're dead.
If you can require recoding of programs, the problem becomes easy:
just make FD_ZERO() allocate dynamic data and require a new function
FD_FREE() to free it up. Unfortunately, acceptance is not likely to
be that great.
In C++, of course, this problem can be handled with constructors and
destructors.
-hpa
--
This space intentionally has nothing but text explaining why this
space has nothing but text explaining why this space would otherwise
have been left blank, and would otherwise have been left blank.