Re: [PATCH net-next v1 02/11] net: Add memory provider capabilities
From: Björn Töpel
Date: Tue Nov 17 2026 - 12:07:44 EST
Mina Almasry <almasrymina@xxxxxxxxxx> writes:
> On Sun, Nov 15, 2026 at 10:47 AM Björn Töpel <bjorn@xxxxxxxxxx> wrote:
>>
>> Mina Almasry <almasrymina@xxxxxxxxxx> writes:
>>
>> > On Tue, Nov 10, 2026 at 2:21 AM Björn Töpel <bjorn@xxxxxxxxxx> wrote:
>> >>
>> >> The core treats every memory provider as unreadable: installing one
>> >> needs header split and no XDP program, and any page pool that allows
>> >> unreadable netmem takes it. An AF_XDP UMEM provider is readable, needs
>> >> XDP, and hands out whole buffers.
>> >>
>> >> Describe providers by MP_CAP_READABLE and MP_CAP_FRAG; devmem and
>> >> io_uring keep their behavior with MP_CAP_FRAG. A page pool takes a
>> >> readable provider only with the new PP_FLAG_MEM_PROVIDER:
>> >
>> > Why do we need this? I thought whatever creates the pool actually
>> > shouldn't get to pick whether the pool picks up the memory provider. I
>> > thought if core has configured the rxq for a memory provider then the
>> > pool is forced to take it.
>>
>> Thanks for looking at series, Mina. ...this needed some thinking.
>>
>> It isn't forced today either: only pools with
>> PP_FLAG_ALLOW_UNREADABLE_NETMEM take the queue's provider. Drivers set
>> it on the pool that holds payloads, and keep headers in host memory: in
>> a separate pool (fbnic, bnxt), or outside the page pool (gve, mlx5).
>> That works because every provider so far backs payloads only.
>>
>> UMEM doesn't. The socket gets whole packets, so UMEM has to back the
>> header buffers, and with multi-buffer the payloads too. The flag was my
>> way to get it there, but you're right that it's the wrong knob: it lets
>> the driver opt in, where it should only say what each pool holds.
>>
>> How about this instead. Both sides use the rx_roles bits from patch 5,
>> HEAD and PAYLOAD:
>>
>> - The provider says which buffers it backs, next to rx_page_size in
>> pp_memory_provider_params: devmem and io_uring zcrx PAYLOAD, AF_XDP
>> HEAD, or HEAD|PAYLOAD with multi-buffer.
>> - The driver says what each pool holds, in rx_roles. A pool without
>> roles but with PP_FLAG_ALLOW_UNREADABLE_NETMEM holds PAYLOAD, so
>> existing drivers don't change.
>> - page_pool_create() gives the provider to a pool whose roles are all
>> backed; the others get host pages. No opt-in.
>> - After ndo_queue_mem_alloc(), the core checks that the new pools that
>> took the provider cover everything it backs, or fails the restart.
>>
>> With fbnic's header and payload pools:
>>
>> devmem: header pool host, payload pool devmem
>> AF_XDP single-buffer: header pool UMEM, payload pool host
>> AF_XDP multi-buffer: one shared pool, UMEM
>>
>> A shared pool on a devmem queue doesn't take devmem, since devmem
>> doesn't back HEAD, so the check fails and the driver has to split.
>> That's the "headers in host memory" rule, without the driver asking
>> which provider it is.
>>
>> Wdyt?
>>
>
> Hmm, I think maybe the exsiting knobs PP_FLAG_ALLOW_UNREADABLE +
> CAP_READABLE covers your needs.
>
> The pp says whether the memory it provides is readable or not via
> CAP_READABLE (or !CAP_UNREADABLE).
>
> The driver says whether it will allow unreadable memory for this pool.
> For header pools, it doesn't specfiy PP_FLAG_ALLOW_UNREADABLE and for
> paylaod pools it specifies ALLOW_UNREADABLE.
>
> The change I think you need to make is: today we disallow any
> providers if the caller did not specify ALLOW_UNREADABLE (because we
> assume mp's are unreadable). After your series we only need to
> disallow memory providers if the mp is unreadable and the caller said
> it's not fine with unreadable.
I'm all in on making v1 simpler -- I'll try this path! So, an unreadable
provider still goes only to pools that set ALLOW_UNREADABLE. A readable
provider goes to all RX pools of the queue. The roles and
PP_FLAG_MEM_PROVIDER are gone.
One addition: a driver must opt in with a readable_mp flag in its
queue ops. Without it, its pools don't look for a provider. The reason
is that today only ALLOW_UNREADABLE pools must set queue_idx and be
created under the instance lock. Other pools may not, so they could
pick up the wrong queue's provider. Maybe there's a better way?
Thanks!
Björn