Re: [PATCH net-next v9] net: reduce RFS/ARFS flow updates by checking LLC affinity

[Eric Dumazet]

On Sat, Sep 12, 2026 at 8:50 PM chuang <nashuiliang@xxxxxxxxx> wrote:
>
> Hi, let me restructure the issues.
>
> > 1) Also not sure why you're trying to touch RFS, the change only helps
> aRFS and you could avoid bulk of the refactoring issues.
>
> The scenario is similar to the one described in "[RFC] problems with
> RFS on bRPC applications"[1].
>
> I attempted to enable ARFS on a Mellanox CX-6 NIC. While it performs
> well for simple workloads, performance degrades significantly when
> running a bRPC[2]  workload on a 2-node NUMA machine. After tracing, I
> identified patterns that ARFS/RFS fails to handle efficiently:
>
> - Multiple threads use epoll to read from the same socket, causing
> frequent flow updates in sock_flow_table.
> - Threads reading from the socket migrate frequently between CPUs.
>
> I tested a PoC version using a bRPC service, utilizing funccount [3]
> to monitor execution frequency and perf top to observe hotspots:
>
> Before Patch
>
> The mlx5e_rx_flow_steer frequency is over 380k/s, and queued_spin_lock
> is a major hotspot (6.30% in perf top). The application also suffers
> from a noticeable drop.
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer              387594
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer              390142
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer              386694
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer              389094
>
> # perf top hotspot:
> queued_spin_lock                         6.30%
>
> After Patch
>
> The ARFS update frequency is significantly reduced. queued_spin_lock
> is no longer a hotspot in perf top, and the application's overall
> performance has improved.
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer                      43
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer                        9
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer                    207
>
> FUNC                                     COUNT
> mlx5e_rx_flow_steer                      26
>
> > 2) You put a very fast path function out-of-line, why ?
>
> The reason is that tun uses sock_rps_record_flow_hash(). When I moved
> all rps_record_sock_flow and rps_record_cond modifications into
> include/net/rps.h, it triggered the following compilation errors due
> to symbol visibility:
>
> ERROR: modpost: "cpus_share_cache" [drivers/net/tun.ko] undefined!
> ERROR: modpost: "cpus_share_cache" [net/sctp/sctp.ko] undefined!
> make[2]: *** [scripts/Makefile.modpost:147: Module.symvers] Error 1
>
> This arises because the patch uses cpus_share_cache() to limit the
> RFS/ARFS update frequency at the LLC level. To keep this in the fast
> path, I could move cpus_share_cache() to
> include/linux/sched/topology.h.

Hmmm

If the LLC affinity check is performed in set_rps_cpu() inside
net/core/dev.c before triggering ndo_rx_flow_steer:

net/core/dev.c is built-in, so it can call cpus_share_cache() directly
without any module export issues.

sock_rps_record_flow() and sock_rps_record_flow_hash() in
include/net/rps.h remain 100% inline, retaining the zero-cost static
key NOP in the TCP/socket fast path.

No exports, no refactoring of tun.ko or sctp.ko, and no fast-path degradation.