RE: [PATCH net-next 2/4] atl1c: improve performance by avoiding unnecessary pcie writes on xmit

[David Laight]

From: Chris Snook <chris.snook@xxxxxxxxx>
> Sent: 12 May 2021 03:40
> 
> On Tue, May 11, 2021 at 12:05 PM Gatis Peisenieks <gatis@xxxxxxxxxxxx> wrote:
> >
> > The kernel has xmit_more facility that hints the networking driver xmit
> > path about whether more packets are coming soon. This information can be
> > used to avoid unnecessary expensive PCIe transaction per tx packet at a
> > slight increase in latency.
> 
> Increases in latency tend to hurt more on single-queue devices. Has
> this been tested on the original gigabit atl1c?

It probably depends a lot on how expensive it is to 'kick' the mac unit.

A simple (posted) PCIe write when the PCIe host interface is idle (as is likely
when you've just been updating descriptors) is probably noise compared
to the rest of the cost of sending the packet.
(Eric will probably say they measured gains.)

OTOH if you have (as I have on one system) the e1000e driver and some
completely broken 'management interface' hardware which means it can
take a lot of microseconds to write to any MAC register you really
do need to look at netdev_xmit_more() [1].

Unfortunately it doesn't help that much.
netdev_xmit_more() reports the state of the tx queue when the current
skb transmit was passed to the mac driver.
It doesn't report the state of the queue at the time netdev_xmit_more()
is called - so any packets queued while the transmit setup is in
progress don't cause netdev_xmit_more() to return true.
I've traced this happening repeatedly...

[1] If the MI is active MAC writes are broken (may write to the
wrong register), so there is horrid code before each access that
(IIRC) effectively does:
	while (mi_active())
		mdelay(10);
This is just so broken (interrupts are even enabled).

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)