Re: [PATCH net-next v9] net: reduce RFS/ARFS flow updates by checking LLC affinity

[chuang]

Hi, let me restructure the issues.

> 1) Also not sure why you're trying to touch RFS, the change only helps
aRFS and you could avoid bulk of the refactoring issues.

The scenario is similar to the one described in "[RFC] problems with
RFS on bRPC applications"[1].

I attempted to enable ARFS on a Mellanox CX-6 NIC. While it performs
well for simple workloads, performance degrades significantly when
running a bRPC[2]  workload on a 2-node NUMA machine. After tracing, I
identified patterns that ARFS/RFS fails to handle efficiently:

- Multiple threads use epoll to read from the same socket, causing
frequent flow updates in sock_flow_table.
- Threads reading from the socket migrate frequently between CPUs.

I tested a PoC version using a bRPC service, utilizing funccount [3]
to monitor execution frequency and perf top to observe hotspots:

Before Patch

The mlx5e_rx_flow_steer frequency is over 380k/s, and queued_spin_lock
is a major hotspot (6.30% in perf top). The application also suffers
from a noticeable drop.

FUNC                                     COUNT
mlx5e_rx_flow_steer              387594

FUNC                                     COUNT
mlx5e_rx_flow_steer              390142

FUNC                                     COUNT
mlx5e_rx_flow_steer              386694

FUNC                                     COUNT
mlx5e_rx_flow_steer              389094

# perf top hotspot:
queued_spin_lock                         6.30%

After Patch

The ARFS update frequency is significantly reduced. queued_spin_lock
is no longer a hotspot in perf top, and the application's overall
performance has improved.

FUNC                                     COUNT
mlx5e_rx_flow_steer                      43

FUNC                                     COUNT
mlx5e_rx_flow_steer                        9

FUNC                                     COUNT
mlx5e_rx_flow_steer                    207

FUNC                                     COUNT
mlx5e_rx_flow_steer                      26

> 2) You put a very fast path function out-of-line, why ?

The reason is that tun uses sock_rps_record_flow_hash(). When I moved
all rps_record_sock_flow and rps_record_cond modifications into
include/net/rps.h, it triggered the following compilation errors due
to symbol visibility:

ERROR: modpost: "cpus_share_cache" [drivers/net/tun.ko] undefined!
ERROR: modpost: "cpus_share_cache" [net/sctp/sctp.ko] undefined!
make[2]: *** [scripts/Makefile.modpost:147: Module.symvers] Error 1

This arises because the patch uses cpus_share_cache() to limit the
RFS/ARFS update frequency at the LLC level. To keep this in the fast
path, I could move cpus_share_cache() to
include/linux/sched/topology.h.

1: https://lore.kernel.org/netdev/CAHCEFEwToeQe_Ey8e=sf8fOmoobvrDCPsxw+hfUSoRawPX03+Q@xxxxxxxxxxxxxx/t/#u
2: https://github.com/apache/brpc
3: https://github.com/iovisor/bcc/blob/master/tools/funccount.py

On Wed, Sep 9, 2026 at 4:36 AM Jakub Kicinski <kuba@xxxxxxxxxx> wrote:
>
> On Tue,  8 Sep 2026 14:04:00 +0800 Chuang Wang wrote:
> > v8 -> v9:
> > - fix errors in AI submissions by Simon Horman
>
> I commented on v8 6 hours before you sent this.
> Please pay more attention to what you're doing.
> --
> pw-bot: cr