Re: [RFC net v2 2/2] af_packet: retire TPACKET_V3 blocks without waiting for copiers

[Willem de Bruijn]

Jingxuan Xie wrote:
> From: jingxuanxie <jingxuanxie@xxxxxxxxxxxx>
> 
> TPACKET_V3 waits for a ring-wide read lock while holding the receive
> queue lock before retiring a block. On PREEMPT_RT a preempted copier
> can delay another packet path or the retirement timer until it runs
> again. This was investigated after an approximately 355 ms retirement
> wait in a PREEMPT_RT deployment.
> 
> Keep copy accounting in each block's existing private pgv descriptor.
> Seal a full or expired block and advance to the next block without
> waiting. Retain the original descriptor across copying and release its
> copy reference after all packet data and headers have been written.
> Publish sealed blocks in order when an acquire read observes no copies.
> 
> Publication runs under the existing receive queue lock in packet
> allocation and timer paths. Completion does not reacquire that lock.
> No worker or publication budget is needed. When traffic stops, completed
> blocks may remain unpublished until the next timer callback; this does
> not promise a hard capture delivery deadline. Ring exhaustion can still
> drop capture copies.
> 
> Use each retired block's saved last packet and its own cache-flush range,
> and exclude sealed blocks from allocation and fanout space checks. Poll
> uses the publication cursor instead of the producer cursor. Widen the
> producer index to cover the same block count as the publication index.
> Allow an exactly fitting record so a maximum-size first record cannot
> cause an empty block to be reopened instead of advancing publication.
> 
> Reproduction uses a QEMU ARM64 SMP PREEMPT_RT kernel with validation-only
> instrumentation delaying an outgoing copier after frame reservation.
> The instrumentation is excluded from these patches.
> 
> Fixes: 632ca50f2cbd ("af_packet: TPACKET_V3: replace busy-wait loop")
> Assisted-by: LLM
> Signed-off-by: Jingxuan Xie <jingxuanxie@xxxxxxxxxxxx>
> ---
>  net/packet/af_packet.c | 140 ++++++++++++++++++++---------------------
>  net/packet/internal.h  |   9 ++-
>  2 files changed, 76 insertions(+), 73 deletions(-)

This is still a very large change across many chunks, for a fairly
small well defined problem.

Let's try to a find a smaller solution to this problem.

For instance, switch to a non-blocking write_trylock for block
retirement. If it fails, it will retry in 8ms.

Or, if spillover to the next block is really needed (explain why),
at least simplify. Do not allow running N blocks ahead.

There are also bugs introduced in this patch, which is to be expected
given its complexity. Instead of focusing on those, let's try to move
to a much simpler fix.