Re: [PATCH net-next v1 04/11] xsk: Allocate UMEM chunks through page_pool

From: Björn Töpel

Date: Tue Nov 17 2026 - 12:15:21 EST


Mina Almasry <almasrymina@xxxxxxxxxx> writes:

> On Sun, Nov 15, 2026 at 10:55 AM Björn Töpel <bjorn@xxxxxxxxxx> wrote:
>>
>> Mina Almasry <almasrymina@xxxxxxxxxx> writes:
>>
>> > On Tue, Nov 10, 2026 at 2:21 AM Björn Töpel <bjorn@xxxxxxxxxx> wrote:
>>
>> >> diff --git a/include/net/xdp.h b/include/net/xdp.h
>> >> index 1105bb9e3292..5ab7802ac4e1 100644
>> >> --- a/include/net/xdp.h
>> >> +++ b/include/net/xdp.h
>> >> @@ -368,7 +368,7 @@ static inline bool xdp_buff_add_frag(struct xdp_buff *xdp, netmem_ref netmem,
>> >>
>> >> if (unlikely(netmem_is_pfmemalloc(netmem)))
>> >> xdp_buff_set_frag_pfmemalloc(xdp);
>> >> - if (unlikely(netmem_is_net_iov(netmem)))
>> >> + if (unlikely(netmem_is_net_iov(netmem) && !netmem_address(netmem)))
>> >> xdp_buff_set_frag_unreadable(xdp);
>> >>
>> >
>> >
>> > netmem_is_net_iov() check became redundant.
>>
>> Hmm, not quite... netmem_address() is also NULL for a highmem page
>> without a mapping, and the stack reads those through kmap, so they
>> aren't unreadable. The net_iov test keeps them out. I'll add a comment.
>>
>
>
> Oh. I missed that. There are a bunch of docs actually that explicitly
> wrongly say to use netmem_address to tell if the memory si readable.
>
> Can you, in a separate patch, add a netmem_readable() helper and
> update the docs (and if possible code places) that checked net-iov
> when they were really checks for readability? Now that you're adding
> readable net-iovs, all that assumption break right?

Uhm, yes. I'll cook a separate patch that adds netmem_readable().

>> >> diff --git a/net/xdp/xsk_pp.c b/net/xdp/xsk_pp.c
>> >> new file mode 100644
>> >> index 000000000000..fdd55265a6cd
>> >> --- /dev/null
>> >> +++ b/net/xdp/xsk_pp.c
>> >> @@ -0,0 +1,246 @@
>> >> +// SPDX-License-Identifier: GPL-2.0
>> >> +/* AF_XDP UMEM as a page pool memory provider
>> >> + *
>> >> + * Each aligned UMEM chunk is a readable net_iov with one owner at a time:
>> >> + *
>> >> + * userspace --FILL--> page pool --release--> reuse --alloc--> page pool
>> >> + * ^ |
>> >> + * +-----RX ring-------+
>> >> + */
>> >> +
>> >> +#include <net/page_pool/helpers.h>
>> >> +#include <net/page_pool/memory_provider.h>
>> >> +#include <net/xdp_sock_drv.h>
>> >> +
>> >> +#include "xsk.h"
>> >> +#include "xsk_queue.h"
>> >> +
>> >> +enum xsk_pp_chunk {
>> >> + XSK_PP_USER, /* owned by userspace */
>> >> + XSK_PP_POOL, /* owned by a page pool */
>> >> + XSK_PP_REUSE, /* on the reuse stack */
>> >> +};
>> >
>> > maybe should be xsk_mp_chunk.
>> >
>> > But also, why is this 3? Usually the mp's track allocated and free. I
>> > assume PP_REUSE is actually free, but I'm curious why it matters to
>> > your mp whether the pool or the userspace holds the memory (or whether
>> > the net stack holds it. Can't it hold it?)
>>
>> REUSE is free, yes. USER and POOL are both allocated, but userspace
>> gives chunks back by writing addresses to FILL, and it can write any
>> address, also one of a chunk the kernel still owns. Taking that entry
>> would hand the device the same chunk twice. So the provider only takes
>> FILL entries for chunks in USER, and drops the rest. io_uring zcrx
>> checks the same with user_refs, and devmem with the socket's token
>> xarray.
>>
>
> Sounds like devmem choose to do extra-pp-refs on netmems given to
> userspace, io-uring choose to give extra user_refs for netmems given
> to userspace, and you have choosen to do something new? I need to read
> the patches more thoroughly to understand.

It is close to io_uring's scheme, but simpler. io_uring counts user refs
per buffer, because one buffer can reach userspace in several pieces,
each returned on its own, and it keeps a page pool ref while userspace
holds it. A UMEM chunk is never split, so userspace holds it at most
once, and one state per chunk replaces the counter: USER means userspace
has it, REUSE is the free list. The chunk also leaves its page pool
before userspace gets it, so no page pool ref is held meanwhile. When
userspace returns a chunk on the FILL ring, we take it only if it is in
USER state, so userspace cannot return a chunk the kernel still uses.


Björn