Re: [PATCH net-next v2 05/12] iavf: always use a full order-0 page

[Alexander Lobakin]

From: David Laight <David.Laight@xxxxxxxxxx>
Date: Fri, 26 May 2023 08:57:31 +0000

> From: Alexander Lobakin
>> Sent: 25 May 2023 13:58

[...]

>> 4096 page
>> 64 head, 320 tail
>> 3712 HW buffer size
>> 3686 max MTU w/o frags
> 
> I'd have thought it was important to pack multiple buffers for
> MTU 1500 into a single page.
> 512 bytes split between head and tail room really ought to
> be enough for most cases.
> 
> Is much tailroom ever used for received packets?

You don't have any tailroom at all when you split 4k page into two
halves on x86_64. From those 512 bytes, 320+ are used for
skb_shared_info. And then you're left with 192 bytes or even less (with
increased MAX_SKB_FRAGS, which becomes a new trend thanks to Eric and
Big TCP). XDP requires 256 bytes of headroom -- and you already don't
have them even with the default number of frags.

> It is used to append data to packets being sent - but that isn't
> really relevant here.
> 
> While the unused memory is moderate for 4k pages, it is horrid
> for anything with large pages - think 64k and above.

But hey, there's always unused space and it's arbitrary whether to treat
it "horrid". For example, imagine a machine mostly handling 64-byte
traffic. From that point of view, even splitting pages into 2 halves
still is "horrid" -- 2048 bytes of truesize only to receive 64 bytes :>

> IIRC large pages are common on big PPC and maybe some arm cpus.

Even in that case, the percentage of 4k-page machines running those NICs
are much higher than > 4k. I thought we usually optimize things for the
most common usecases. Adding a couple hundred lines, branches on
hotpaths, limitations etc. only to serve a particular architecture... Dunno.
I have 16k pages on my private development machines and I have no issues
with using 1 page per frame in their NIC drivers :s

> 
> 	David
> 
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
> 

Thanks,
Olek