Re: InfoWorld web server shootout

Alex Belits (
Wed, 9 Jul 1997 17:17:44 -0700 (PDT)

On 9 Jul 1997, Matthias Urlichs wrote:

> Alex Belits <> writes:
> > On Wed, 9 Jul 1997, Alan Cox wrote:
> >
> > > > IMHO in Linux threads have no advantage over plain operations on
> > > > select()'ed descriptors, given that the time spent in request parsing and
> > > > preparing the response isn't huge compared to actual data transfer --
> > >
> > > At very high speed they do because your single threaded task blocks briefly
> > > every page fault it takes
> >
> > How threads help with performance on page faults, on mmaped files or
> > plain memory (given that use of threads doesn't decrease the number of
> > them)? I'm probably really missing something.
> >
> Easy, when one thread hangs on the missing page another thread, presumably
> doing something else, can continue.

This assumes that other threads do something with different region of
memory, and this is what is mostly avoided in one-loop applications.
Multiple processes handle that kind of situation better because they don't
have any syncronization that may cause overhead in threads.

> All of this OF COURSE assumes that you have kernel threads. Your basic
> userland mit-pthreads-or-whatever package is no help at all here.

Of course.

> > > and whenever it writes a block to the log files.
> >
> > That can be a problem, and I think that for a lot of other reasons logs
> > should not be written directly in the main select loop but should
> > use separate process to do it, connected through pipe (context switches
> > between loop and that process don't look like a significant danger for
> > performance).
> >
> In fact, log files should be written to an entirely different machine...

In some environments - yes, but when the main concern is speed, it's
better not to put even more network-related work on kernel.

> > > Both on Linux are extremely cheap, but a clone() based thread is a fair
> > > bit faster than a process switch if you are using CLONE_VM as it doesnt
> > > cause a TLB flush on a task switch. You also want one thread per cpu or
> > > more of course 8)
> >
> > What are advantages of having >=1 thread per cpu to having >=1 process
> > per cpu, if any?
> He just explained that, didn't he? Context switching among (kernel) threads
> under Linux is faster than switching between processes.

On different procesors? I understand that threads context switch takes
less time, but that also requires additional processing for
syncronization. With SMP there will be less context switches, so advantage
will decrease while overhead remains the same.

> > heavy-processing chunks of code, select() on everything that is waiting
> > shouldn't cause delays compared to threads that basically will do the same
> > thing implicitly in this model, but will introduce additional "light"
> > context switches between themselves.
> >
> select() is not that cheap (the kernel has to copy and scan three lists of
> however-many file descriptors), plus you'll have to do a process switch
> afterwards.

What process switch?

> A thread switch is cheap: A thread hangs in accept() => as soon
> as it returns, make another thread runnable by raising a semaphore (which
> then will do another accept() on the socket), service the request
> synchronously, then wait to be called again (by waiting on the semaphore).
> In the trivial case, this will even work with just one thread per port
> you're listening to.

Umm... my HTTP server is listening for clients on one port (using
INADDR_ANY), and it's possible to do use multiple accept()'ing threads on
one port if they are syncronized to avoid simultaneous accept()'s. But
that model increases the amount of syncronization/blocking code while
giving an advantage only for the case of data, scattered in memory.