Re: [PATCH 1/3] epoll: increase default max_user_instances to 1024

[Bron Gondwana]

On Tue, Jan 27, 2009 at 08:00:30PM -0800, Davide Libenzi wrote:
> On Wed, 28 Jan 2009, Bron Gondwana wrote:
> 
> > Both Postfix and Apache use an epoll instance per child, which
> > leads to significant scalability issues with max_user_instances
> > set so low.  Bump the default to 1024 so medium sized sites are
> > not impacted.
> 
> NACK. Epoll allocates globally about 100 to 160 bytes (32/64 bit) for each 
> file added to the interface:
> 
> 	for i 1..1024
> 		for j 1..1024
> 			if i!=j
> 				add j -> i
> 
> That's (N^2 * {100, 160}) = 100MB to 160MB of pinned kernel memory, 

Woah - that's serious.

This:

instances_uid     0  (root)
num_instances	142
max_instances  4096
watches_uid     107  (postfix)
num_watches    1097
max_watches  266555

isn't serious.  It's pretty sane.  142 processes with an epoll watcher,
and fewer than 10 fds per epoll.  Unfortunately, it wouldn't work on an
unpatched and un-specially-configured stock kernel.  That's steady-state
too, not a peak.  I just grabbed it off a running MX:

[brong@mx1 ~]$ free
             total       used       free     shared    buffers
cached
Mem:       4151652    3113128    1038524          0     130808
2014152
-/+ buffers/cache:     968168    3183484
Swap:      2047992      50364    1997628
[brong@mx1 ~]$ uptime
 00:31:05 up 2 days, 18:03,  2 users,  load average: 0.86, 1.23, 1.08

Hardly looking stressed right now.

If I'm reading it right, your concern is the massively recursive case,
where every single epoll gets added to every other epoll as a chained
file descriptor?

That's clearly not happening here - so it seems that maybe our "happy
medium" is actually in closer inspection of what's going on rather than
a blanket low N to keep N^2 down.

Bron.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/