Re: Asynchrony (was Re: Locking a process or thread onto a specific CPU)

David Wragg (dpw@doc.ic.ac.uk)
20 Feb 1999 16:30:24 +0000

Alexander Kjeldaas <astor@fast.no> writes:
> There is something else we might need as well: Yielding to a specific
> process.
>
> The problem manifests itself in large threaded programs. Say you have
> a couple of hundred threads in a program. In these programs, there
> will always be some bottlenecks. Locks surrounding malloc() is a
> typical example. Lots of processes will block trying to allocate
> memory. After the process that has aquired the lock releases it, all
> the other processes will be waked up, leading to a "thundering
> herd"-problem exactly like the one happening for networking
> applications. Now if we could yield explicitly to the first process
> waiting for the lock, we would get acceptable performance again.

But this is exactly what LinuxThreads does! When a mutex is unlocked,
a single thread gets taken off the mutex's queue and sent a signal,
causing it to wake up.

There is a serialization problem if you have threads constantly
hammering on malloc/free, so that heap contention causes context
switches on a significant proportion of malloc/free calls (when that
starts happening, you have problems even for user-space context
switches). The only way around that is to have multiple heaps
(*). Does any other system do this? (For malloc; I know of other
language implementations that do it). For programs with reasonable
heap contention, the LinuxThreads/glibc malloc performance should be
pretty good.

(*) Given that few malloc calls take >1000 cycles, but to block a
thread in mutex_lock costs >1000 cycles, it *might* help for a thread
to backoff then retry once or twice when locking the malloc locks, but
it would only be a win on SMP, and it would need a real program that
can be demonstrated to suffer from heap contention to show benefit --
does anyone have one?

> LinuxThreads seems to loose against FreeBSD user-land threads in this
> regard.

Perhaps it is because user-space thread switches are so very much
cheaper.

I'd like to see a nice implementation of user-on-kernel-threads on
Linux, but I'm really not sure it would be a win for typical pthreads
C programs. When you say LinuxThreads loses against FreeBSD threads,
is that for micro-benchmarks or real programs?

Dave Wragg

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/