Re: [PATCH 1/3] epoll: increase default max_user_instances to 1024

[Bron Gondwana]

On Wed, 28 Jan 2009 15:51 -0800, "Davide Libenzi" <davidel@xxxxxxxxxxxxxxx> wrote:
> On Thu, 29 Jan 2009, Bron Gondwana wrote:
> 
> > On Wed, Jan 28, 2009 at 08:52:51AM -0800, Davide Libenzi wrote:
> > > On Wed, 28 Jan 2009, Vegard Nossum wrote:
> > > 
> > > > On Wed, Jan 28, 2009 at 6:32 AM, Bron Gondwana <brong@xxxxxxxxxxx> wrote:
> > > > > That's clearly not happening here - so it seems that maybe our "happy
> > > > > medium" is actually in closer inspection of what's going on rather than
> > > > > a blanket low N to keep N^2 down.
> > > > 
> > > > Mh, could another solution to this all be to limit the number times
> > > > you can add a single epoll descriptor to another descriptor's set?
> > > 
> > > In the example that was posted, a single fd was added a single time inside 
> > > the other 1000+ fds. Epoll already has detection for too long chains and 
> > > closed loops, but you can't put those in the fast path. And epoll_ctl() is 
> > > one of those.
> > 
> > Not even if you're adding an epoll watcher inside another epoll watcher?
> 
> Adding an epoll fd inside another epoll fd is perfectly legal. It would
> kinda suck if epoll itself wouldn't expose a pollable interface too.

Yeah, I'm not suggesting killing it completely, just putting the limit
at that level rather than limiting watches completely.

The other thing I've looked at is just limiting watches per process to
rlim[RLIMIT_NFILE].  Any reason why that wouldn't be enough?  It means
you get limited to less than RLIMIT_NFILE if you add the same file
descriptor to multiple epolls... but that's not too scary.

> > The problem I have here is that "a single fd was added a single time
> > inside the other 1000+ fds" is different behaviour to the daemons out
> > there.  They're pretty much all using flat layouts:
> 
> Yes, that is not what programs normally do. Most of the times you have 
> nesting level equal zero, although we've seen recently that the 
> epoll-being-pollable feature (hence nesting) is used too. Say you have
> two 
> (or more) libraries, each own monitoring different things, and each own 
> with its own wait+dispatch loop. If these libraries didn't have a chance 
> to expose a pollable fd, you'd have to run their wait+dispatch loop in 
> seaprate threads. Whereas epoll being itself pollable allows you to:
> 
> 	epoll_wait(lib1_fd, lib2_fd)
> 	if (ready(lib1_fd))
> 		lib1_dispatch()
> 	if (ready(lib2_fd))
> 		lib2_dispatch()
> 
> This is pretty powerful, although needs care for wakeups and poll nested 
> calls.

That would only add a couple of "epoll watching epoll" instances though - you
could still limit that to a pretty low number and not impact normal use.

Bron.
-- 
  Bron Gondwana
  brong@xxxxxxxxxxx

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/