RE: [EXT] [PATCH net-next 07/15] net: mvpp2: fix the computation of the RXQs

[Yan Markman]

OK-OK. The " It does fix PPv2.1 support, which was broken" is great reason!

-----Original Message-----
From: Antoine Tenart <antoine.tenart@xxxxxxxxxxx> 
Sent: Thursday, February 28, 2019 5:51 PM
To: Yan Markman <ymarkman@xxxxxxxxxxx>
Cc: Antoine Tenart <antoine.tenart@xxxxxxxxxxx>; davem@xxxxxxxxxxxxx; linux@xxxxxxxxxxxxxxx; netdev@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; thomas.petazzoni@xxxxxxxxxxx; maxime.chevallier@xxxxxxxxxxx; gregory.clement@xxxxxxxxxxx; miquel.raynal@xxxxxxxxxxx; Nadav Haklai <nadavh@xxxxxxxxxxx>; Stefan Chulski <stefanc@xxxxxxxxxxx>; mw@xxxxxxxxxxxx
Subject: Re: [EXT] [PATCH net-next 07/15] net: mvpp2: fix the computation of the RXQs

Yan,

On Thu, Feb 28, 2019 at 03:40:50PM +0000, Yan Markman wrote:
> 
> Regarding the MVPP2_DEFAULT_RXQ
> Seems the current variant is flexible, permitting easy customize the 
> configuration according to customer's needs.
> 
> Regarding the Queue in probe():
> Looking into old code there where no2 queue-modes but 3:
> enum mv_pp2_queue_distribution_mode {
> 	MVPP2_QDIST_SINGLE_MODE,
> 	MVPP2_QDIST_MULTI_MODE,
> 	MVPP2_SINGLE_RESOURCE_MODE
> };
> The current   if(MVPP2_QDIST_MULTI_MODE)else is correct also for the
> MVPP2_SINGLE_RESOURCE_MODE, but new/patched isn't.

There are only 2 modes supported in the upstream kernel:
MVPP2_QDIST_SINGLE_MODE and MVPP2_QDIST_MULTI_MODE. The third one you mentioned is only supported in out-of-tree kernels.

Therefore patches sent to the upstream kernel do not take in into account, as it is not supported.

> Since this patch doesn't change any functionality (right now) but 
> reduces the flexibility I do not see real reason to apply it.

This patch do not break the upstream support of PPv2 and does improve two thing:

- It limits the total number of RXQs being allocated, to ensure the
  number of RXQs being used do not exceed the number of available RXQ
  (which would make the driver to fail).
- It does fix PPv2.1 support, which was broken.

I do think the patch will benefit the upstream PPv2 support.

Thanks,
Antoine

> The patch fixes the computation of RXQs being used by the PPv2 driver, 
> which is set depending on the PPv2 engine version and the queue mode 
> used. There are three cases:
> 
> - PPv2.1: 1 RXQ per CPU.
> - PPV2.2 with MVPP2_QDIST_MULTI_MODE: 1 RXQ per CPU.
> - PPv2.2 with MVPP2_QDIST_SINGLE_MODE: 1 RXQ is shared between the CPUs.
> 
> The PPv2 engine supports a maximum of 32 queues per port. This patch 
> adds a check so that we do not overstep this maximum.
> 
> It appeared the calculation was broken for PPv2.1 engines since 
> f8c6ba8424b0, as PPv2.1 ports ended up with a single RXQ while they 
> needed 4. This patch fixes it.
> 
> Fixes: f8c6ba8424b0 ("net: mvpp2: use only one rx queue per port per 
> CPU")
> Signed-off-by: Antoine Tenart <antoine.tenart@xxxxxxxxxxx>
> ---
>  drivers/net/ethernet/marvell/mvpp2/mvpp2.h    |  4 ++--
>  .../net/ethernet/marvell/mvpp2/mvpp2_main.c   | 23 ++++++++++++-------
>  2 files changed, 17 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h 
> b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
> index 17ff330cce5f..687e011de5ef 100644
> --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
> +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2.h
> @@ -549,8 +549,8 @@
>  #define MVPP2_MAX_TSO_SEGS		300
>  #define MVPP2_MAX_SKB_DESCS		(MVPP2_MAX_TSO_SEGS * 2 + MAX_SKB_FRAGS)
>  
> -/* Default number of RXQs in use */
> -#define MVPP2_DEFAULT_RXQ		1
> +/* Max number of RXQs per port */
> +#define MVPP2_PORT_MAX_RXQ		32
>  
>  /* Max number of Rx descriptors */
>  #define MVPP2_MAX_RXD_MAX		1024
> diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c 
> b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> index 24cee6cbe309..9c6200a59910 100644
> --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> @@ -4062,8 +4062,8 @@ static int mvpp2_multi_queue_vectors_init(struct mvpp2_port *port,
>  			snprintf(irqname, sizeof(irqname), "hif%d", i);
>  
>  		if (queue_mode == MVPP2_QDIST_MULTI_MODE) {
> -			v->first_rxq = i * MVPP2_DEFAULT_RXQ;
> -			v->nrxqs = MVPP2_DEFAULT_RXQ;
> +			v->first_rxq = i;
> +			v->nrxqs = 1;
>  		} else if (queue_mode == MVPP2_QDIST_SINGLE_MODE &&
>  			   i == (port->nqvecs - 1)) {
>  			v->first_rxq = 0;
> @@ -4156,8 +4156,7 @@ static int mvpp2_port_init(struct mvpp2_port *port)
>  	    MVPP2_MAX_PORTS * priv->max_port_rxqs)
>  		return -EINVAL;
>  
> -	if (port->nrxqs % MVPP2_DEFAULT_RXQ ||
> -	    port->nrxqs > priv->max_port_rxqs || port->ntxqs > MVPP2_MAX_TXQ)
> +	if (port->nrxqs > priv->max_port_rxqs || port->ntxqs > 
> +MVPP2_MAX_TXQ)
>  		return -EINVAL;
>  
>  	/* Disable port */
> @@ -4778,10 +4777,18 @@ static int mvpp2_port_probe(struct platform_device *pdev,
>  	}
>  
>  	ntxqs = MVPP2_MAX_TXQ;
> -	if (priv->hw_version == MVPP22 && queue_mode == MVPP2_QDIST_MULTI_MODE)
> -		nrxqs = MVPP2_DEFAULT_RXQ * num_possible_cpus();
> -	else
> -		nrxqs = MVPP2_DEFAULT_RXQ;
> +	if (priv->hw_version == MVPP22 && queue_mode == MVPP2_QDIST_SINGLE_MODE) {
> +		nrxqs = 1;
> +	} else {
> +		/* According to the PPv2.2 datasheet and our experiments on
> +		 * PPv2.1, RX queues have an allocation granularity of 4 (when
> +		 * more than a single one on PPv2.2).
> +		 * Round up to nearest multiple of 4.
> +		 */
> +		nrxqs = (num_possible_cpus() + 3) & ~0x3;
> +		if (nrxqs > MVPP2_PORT_MAX_RXQ)
> +			nrxqs = MVPP2_PORT_MAX_RXQ;
> +	}
>  
>  	dev = alloc_etherdev_mqs(sizeof(*port), ntxqs, nrxqs);
>  	if (!dev)
> --
> 2.20.1
> 

--
Antoine Ténart, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com