Re: [RFC PATCH net-next] net/udp: Add 4-tuple hash for connected socket

[Eric Dumazet]

On Fri, Sep 13, 2024 at 5:07 PM Dust Li <dust.li@xxxxxxxxxxxxxxxxx> wrote:
>
> On 2024-09-13 16:39:33, Eric Dumazet wrote:
> >On Fri, Sep 13, 2024 at 4:22 PM Dust Li <dust.li@xxxxxxxxxxxxxxxxx> wrote:
> >>
> >> On 2024-09-13 13:49:03, Eric Dumazet wrote:
> >> >On Fri, Sep 13, 2024 at 12:09 PM Philo Lu <lulie@xxxxxxxxxxxxxxxxx> wrote:
> >> >>
> >> >> This RFC patch introduces 4-tuple hash for connected udp sockets, to
> >> >> make udp lookup faster. It is a tentative proposal and any comment is
> >> >> welcome.
> >> >>
> >> >> Currently, the udp_table has two hash table, the port hash and portaddr
> >> >> hash. But for UDP server, all sockets have the same local port and addr,
> >> >> so they are all on the same hash slot within a reuseport group. And the
> >> >> target sock is selected by scoring.
> >> >>
> >> >> In some applications, the UDP server uses connect() for each incoming
> >> >> client, and then the socket (fd) is used exclusively by the client. In
> >> >> such scenarios, current scoring method can be ineffcient with a large
> >> >> number of connections, resulting in high softirq overhead.
> >> >>
> >> >> To solve the problem, a 4-tuple hash list is added to udp_table, and is
> >> >> updated when calling connect(). Then __udp4_lib_lookup() firstly
> >> >> searches the 4-tuple hash list, and return directly if success. A new
> >> >> sockopt UDP_HASH4 is added to enable it. So the usage is:
> >> >> 1. socket()
> >> >> 2. bind()
> >> >> 3. setsockopt(UDP_HASH4)
> >> >> 4. connect()
> >> >>
> >> >> AFAICT the patch (if useful) can be further improved by:
> >> >> (a) Support disable with sockopt UDP_HASH4. Now it cannot be disabled
> >> >> once turned on until the socket closed.
> >> >> (b) Better interact with hash2/reuseport. Now hash4 hardly affects other
> >> >> mechanisms, but maintaining sockets in both hash4 and hash2 lists seems
> >> >> unnecessary.
> >> >> (c) Support early demux and ipv6.
> >> >>
> >> >> Signed-off-by: Philo Lu <lulie@xxxxxxxxxxxxxxxxx>
> >> >
> >> >Adding a 4-tuple hash for UDP has been discussed in the past.
> >>
> >> Thanks for the information! we don't know the history.
> >>
> >> >
> >> >Main issue is that this is adding one cache line miss per incoming packet.
> >>
> >> What about adding something like refcnt in 'struct udp_hslot' ?
> >> if someone enabled uhash4 on the port, we increase the refcnt.
> >> Then we can check if that port have uhash4 enabled. If it's zero,
> >> we can just bypass the uhash4 lookup process and goto the current
> >> udp4_lib_lookup2().
> >>
> >
> >Reading anything (thus a refcnt) in 'struct udp_hslot' will need the
> >same cache line miss.
>
> hslot2->head in 'struct udp_hslot' will be read right away in
> udp4_lib_lookup2() in any case, it's just a few instructions
> later(about 20). So I think cache miss should not be a problem
> in this case.

I guess this could work.