epoll and listener sockets

[Hassink, Brian]

$ uname -r
2.6.32-279.5.2.el6prerel6.0.0_80.23.0.x86_64
$ cat /etc/issue
CentOS release 6.3 (Final)

I sincerely hope this is the correct forum in which to ask about this, and apologize profusely if it is not.

I have a listener socket in an epoll set, and it will occasionally fail to receive an EPOLLIN event for a connection.  I have looked at a few example programs, which typically have the following sequence...

  1. call socket()
  2. call bind()
  3. call fcntl() to make fd non-blocking
  4. call epoll_ctl() to add the fd with (EPOLLET | EPOLLONESHOT | EPOLLIN)
  5. call listen()
  6. enter epoll_wait() loop

...where the listener socket is added to the epoll set before the epoll_wait() loop.

In my application, concurrent threads are running in an epoll_wait() loop and a listener socket may be created at any time.  I had initially tried this sequence...

  1. call socket()
  2. call bind()
  3. call fcntl() to make fd non-blocking
  4. call epoll_ctl() to add the fd with (EPOLLET | EPOLLONESHOT | EPOLLIN)
  5. call listen()

...but often received an EPOLLHUP event because of a concurrent epoll_wait() call between step 4 and 5.  So I switched the sequence to...

  1. call socket()
  2. call bind()
  3. call fcntl() to make fd non-blocking
  4. call listen()
  5. call epoll_ctl() to add the fd with (EPOLLET | EPOLLONESHOT | EPOLLIN)

In my testing there is only one connection attempt to the listener port, so EPOLLONESHOT should not be a factor.  I have also tried level-triggered with the same result.

I should also note that the connection attempt is made exactly one second after the listener is created.  So there isn't a race where the connection attempt is already queued before the listener is added to the epoll set.

I saw that there was a recent patch for EPOLL_CTL_MOD and EPOLLONESHOT, but I don't think that is relevant here.  Any thoughts on what the problem might be?

Thanks in advance,
Brian
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/