Re: sem.c multithreading

Manfred Spraul (manfreds@colorfullife.com)
Tue, 02 Nov 1999 16:37:45 +0100


Christoph Rohland wrote:
>
> As I said I am working on making SEMMNI, SEMMSL and SEMMNS
> sysctlable. This leads to allocating the array via kmalloc and also
> possibly having a really big array.
>
I found a document on IBM's web site, and AFAICS they recommend:
* MSGMNI: memory dependant, for 512MB+: 256.
* MSGMAX 65535
* SEMMNI: memory dependant, for 512MB+: 1024.
* SEMMSL: nothing is said, but it seems only 2? (since SEMMNS usually =
SEMMSL*SEMMNI)
* SEMOPL: nothing is said, but <=SEMMSL.

Unfortunately, I found no infos about > 1 GB memory. Linux 2.4 will
support up to 32 GB memory.
IIRC Oracle only needs a few semaphore arrays, but they are large: they
limit the PROCESSES parameter to SEMMSL, and 512 is far to low.

Could you ask an administrator what the typical requirements of a
high-end DB2/Oracle/Sybase/etc. database are? Could you perhaps post the
output from 'ipcs'?

Which changes are necessary?
a) MSGMAX: add a linked list to 'struct msg_msg'. No locking changes are
required. I'll do that.

b) SEMMSL: there are 3 implementation limits:
* 512: stack usage during GETALL and SETALL (semctl_down() and
semctl_locked_unlock())
* ~700: kmalloc for the semaphore array exceeds PAGE_SIZE
* ~ 2000: kmalloc for the undo structure exceeds PAGE_SIZE.

possible solution:
* use the stack are for small 'sma->nsems', and a dynamic allocation for
large areas. GETALL and SETALL should be rare. eg. do_readv_writev() in
fs/read_write.c does that.
* allocate the memory with vmalloc(), eg alloc_fd_array() in fs/file.c
does that.

c) SEMOPL:
* limit 160 due to stack usage, same solution as for GETALL/SETALL.

d) SEMMNI, MSGMNI:
The current implementation has a hard limit of 32767, but it's easy to
increase that to MAX_INT. The problem is that you cannot change it at
run time (locking problem!).
I think there are only 3 options:
1) the values can be set at boot-time with a command-line parameter, but
it cannot be changed at runtime.
2) We remove the per-id spinlocks and use one global spinlock.
3) We add an additional rw-lock, everyone must grab that lock shared,
the sysctl grabs it exclusive.

I would prefer 1), what do you think?

I think you have already written a patch, could you post it?

--
	Manfred

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/