Re: disappearing listen()ed SO_REUSEPORT sockets across fork() whenusing epoll

From: Shawn Landden
Date: Mon Nov 25 2013 - 16:03:52 EST


On Mon, Nov 25, 2013 at 12:05 PM, Mateusz Guzik <mguzik@xxxxxxxxxx> wrote:
> On Mon, Nov 25, 2013 at 11:53:24AM -0800, Shawn Landden wrote:
>> On Mon, Nov 25, 2013 at 10:05 AM, Jason Baron <jbaron@xxxxxxxxxx> wrote:
>> > On 11/22/2013 12:53 PM, Shawn Landden wrote:
>> >> Hello, when running the attached program on 3.12 child processes
>> >> are missing a socket fd opened, set with SO_REUSEPORT, listen()ed to,
>> >> and added to epoll_ctl().
>> >>
>> >> This is the output I get when pointing "wget http://localhost:5555/";
>> >> at the attached program:
>> >>
>> >> main PID 31591
>> >> PID 31634 started
>> >> PID 31634 accept()ed connection
>> >> PID 31635 started
>> >> PID 31636 started
>> >> PID 31635 accept() failed: Bad file descriptor
>> >> PID 31636 accept() failed: Bad file descriptor
>> >> PID 31634 accept()ed connection
>> >> PID 31634 accept()ed connection
>> >> PID 31634 accept()ed connection
>> >> PID 31634 accept()ed connection
>> >>
>> >>
>> >> While I would expect something like:
>> >>
>> >> main PID 31591
>> >> PID 31634 started
>> >> PID 31634 accept()ed connection
>> >> PID 31635 started
>> >> PID 31636 started
>> >> PID 31635 accept()ed connection
>> >> PID 31636 accept()ed connection
>> >>
>> >> -more new processes, but inversely proportional to number of listening processes
>> >> -accept() always returns successfully
>> >>
>> >>
>> >
>> > The 'close(sockfd);' looks to be racing with the accept() calls. Removing seems
>> > to get the result you are looking for.
>> Interesting. That works, but it shouldn't. The close() is operating in
>> the parent, so it shouldn't affect the child,
>> there is a leak here of process separation.
>>
>
> You fork, then close sockfd in the parent. Thus, the very first child
> can accept connectins just fine.
>
> Subsequent forks give you children without sockfd, thus accept fails.
> The first child continues to work just fine.
Now I feel like an idiot.



--

---
Shawn Landden
+1 360 389 3001 (SMS preferred)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/