Re: [PATCH net v2] net: dsa: tag_rtl4_a: Bump min packet size

[Vladimir Oltean]

On Mon, Oct 30, 2023 at 10:26:50AM +0100, Linus Walleij wrote:
> It was reported that the "LuCI" web UI was not working properly
> with a device using the RTL8366RB switch. Disabling the egress
> port tagging code made the switch work again, but this is not
> a good solution as we want to be able to direct traffic to a
> certain port.
> 
> It turns out that packets between 1496 and 1500 bytes are
> dropped unless padded out to 1518 bytes.
> 
> If we pad the ethernet frames to a minimum of ETH_FRAME_LEN + FCS
> (1518 bytes) everything starts working fine.
> 
> 1496 is the size of a normal data frame minus the internal DSA
> tag. The number is intuitive, the explanation evades me.
> 
> As we completely lack datasheet or any insight into how this
> switch works, this trial-and-error solution is the best we
> can do. FWIW this bug may very well be the reason why Realteks
> code drops are not using CPU tagging. The vendor routers using
> this switch are all hardwired to disable CPU tagging and all
> packets are pushed to all ports on TX. This is also the case
> in the old OpenWrt driver, derived from the vendor code drops.
> 
> I have tested smaller sizes, only 1518 or bigger padding works.
> 
> Modifying the MTU on the switch (one setting affecting all
> ports) has no effect.
> 
> Before this patch:
> 
> PING 192.168.1.1 (192.168.1.1) 1470(1498) bytes of data.
> ^C
> --- 192.168.1.1 ping statistics ---
> 2 packets transmitted, 0 received, 100% packet loss, time 1048ms
> 
> PING 192.168.1.1 (192.168.1.1) 1472(1500) bytes of data.
> ^C
> --- 192.168.1.1 ping statistics ---
> 12 packets transmitted, 0 received, 100% packet loss, time 11267ms
> 
> After this patch:
> 
> PING 192.168.1.1 (192.168.1.1) 1470(1498) bytes of data.
> 1478 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.533 ms
> 1478 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.630 ms
> 
> PING 192.168.1.1 (192.168.1.1) 1472(1500) bytes of data.
> 1480 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.527 ms
> 1480 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.562 ms
> 
> Also LuCI starts working with the 1500 bytes frames it produces
> from the HTTP server.
> 
> Fixes: 9eb8bc593a5e ("net: dsa: tag_rtl4_a: fix egress tags")
> Signed-off-by: Linus Walleij <linus.walleij@xxxxxxxxxx>
> ---
> Changes in v2:
> - Pad packages >= 1496 bytes after some further tests
> - Use more to-the-point description
> - Provide ping results in the commit message
> - Link to v1: https://lore.kernel.org/r/20231027-fix-rtl8366rb-v1-1-d565d905535a@xxxxxxxxxx
> ---
>  net/dsa/tag_rtl4_a.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/net/dsa/tag_rtl4_a.c b/net/dsa/tag_rtl4_a.c
> index c327314b95e3..3292bc49b158 100644
> --- a/net/dsa/tag_rtl4_a.c
> +++ b/net/dsa/tag_rtl4_a.c
> @@ -45,6 +45,16 @@ static struct sk_buff *rtl4a_tag_xmit(struct sk_buff *skb,
>  	if (unlikely(__skb_put_padto(skb, ETH_ZLEN, false)))
>  		return NULL;
>  
> +	/* Packets over 1496 bytes get dropped unless they get padded
> +	 * out to 1518 bytes. 1496 is ETH_DATA_LEN - tag which is hardly
> +	 * a coinicidence, and 1518 is ETH_FRAME_LEN + FCS so we define
> +	 * the threshold size and padding like this.
> +	 */
> +	if (skb->len >= (ETH_DATA_LEN - RTL4_A_HDR_LEN)) {
> +		if (unlikely(__skb_put_padto(skb, ETH_FRAME_LEN + ETH_FCS_LEN, false)))
> +			return NULL;
> +	}

De-obfuscating the macros:

	if (skb->len >= 1496)
		__skb_put_padto(skb, 1518, false);

	(...)

	skb_push(skb, 4);

which means that here, skb->len will be 1522, if it was originally 1496.
So the code adds 26 extra octets, and only 4 of those are legitimate (a tag).
The rest is absolutely unexplained, which means that until there is a
valid explanation for them:

pw-bot: cr

(sorry, but if it works and we don't know why it works, then at some
point it will break and we won't know why it stopped working)

In the discussion on the previous thread:
https://lore.kernel.org/netdev/CACRpkdbq03ZXcB-TaBp5Udo3M47rb-o+LfkEkC-gA1+=x1Zd-g@xxxxxxxxxxxxxx/

you said that what increments is Dot1dTpPortInDiscards. 802.1Q-2018 says
about it: "Count of received valid frames that were discarded (i.e.,
filtered) by the Forwarding Process." which is odd enough to me, since
packets sent by rtl4a_tag_xmit() should *not* be processed by the forwarding
layer of the switch, but rather, force-delivered to the specified egress
port.

Could you please try to revert the effect of commit 339133f6c318 ("net:
dsa: tag_rtl4_a: Drop bit 9 from egress frames") by setting that bit in
the egress tag again? Who knows, maybe it is the bit which tells the
switch to bypass the forwarding process. Or maybe there is a bit in a
different position which does this. You could try to fill in all bits in
unknown positions, check that there are no regressions with packet sizes
< 1496, and then see if that made any changes to packet sizes >= 1496.
If it did, try to see which bit made the difference.

> +
>  	netdev_dbg(dev, "add realtek tag to package to port %d\n",
>  		   dp->index);
>  	skb_push(skb, RTL4_A_HDR_LEN);
> 
> ---
> base-commit: d9e164e4199bc465b3540d5fe3ffc8a9a4fc6157
> change-id: 20231027-fix-rtl8366rb-e752bd5901ca
> 
> Best regards,
> -- 
> Linus Walleij <linus.walleij@xxxxxxxxxx>
>