Networking Question

[Ryan P. Nicholl]

I've been unable to find any Linux API for asynchronously waiting for the TCP send buffer to drain.

The problem I have, in a nutshell, is noted in this part of the documentation:

       If fildes refers to a socket, close() shall cause the socket to
       be destroyed. If the socket is in connection-mode, and the
       SO_LINGER option is set for the socket with non-zero linger time,
       and the socket has untransmitted data, then close() shall block
       for up to the current linger interval until all data is
       transmitted.

Ok, so not good for asynchronous programming, so I could disable the SO_LINGER option, but that leaves me with another problem, namely that I *want* the socket to linger.

The behavior I want is something like, calling "close", getting EAGAIN instead of triggering TCP RST, and something like EPOLLWRITEFLUSHED to wait for the TCP send buffer to be drained. I know neither of these are possible.

Right now the only solution I can think of is to enable SO_LINGER and spawn a thread to run close in, but this might spawn a lot of threads, and doesn't support cancellation well.

Alternatively, I could call getsockopt with TCP_INFO in a loop, but this triggers a lot of wake-ups and might result in sockets hanging around a lot longer than they need to.

I want to allow linger indefinitely on close until some event happens like running out of ram or other resources, basically to intelligently do something like: "OK, we're running low on RAM/resources, time to send RST and drop the send buffer for the 5k worst behaving connections". So unfortunately even with the timeout provided by SO_LINGER, even assuming close would somehow complete in the background, this could be an issue.

Is there any way to do this properly on Linux? If not, any possibility that something like adding EPOLLWRITEFLUSHED would be a welcome addition?

Please CC me on responses.

--
Ryan P. Nicholl