Re: [PATCH iwl-next 4/5] i40e: Fix broken support for floating VEBs

From: Wojciech Drewek
Date: Tue Nov 14 2023 - 08:21:16 EST

Next message: Linus Walleij: "Re: [PATCH] pinctrl: avoid reload of p state in interation"
Previous message: Herve Codina: "Re: [PATCH 1/1] lib/vsprintf: Fix %pfwf when current node refcount == 0"
In reply to: Ivan Vecera: "[PATCH iwl-next 4/5] i40e: Fix broken support for floating VEBs"
Next in thread: Ivan Vecera: "[PATCH iwl-next 3/5] i40e: Add helpers to find VSI and VEB by SEID and use them"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On 13.11.2023 13:58, Ivan Vecera wrote:
> Although the i40e supports so-called floating VEB (VEB without
> an uplink connection to external network), this support is
> broken. This functionality is currently unused (except debugfs)
> but it will be used by subsequent series for switchdev mode
> slow-path. Fix this by following:
>
> 1) Handle correctly floating VEB (VEB with uplink_seid == 0)
> in i40e_reconstitute_veb() and look for owner VSI and
> create it only for non-floating VEBs and also set bridge
> mode also only for such VEBs as the floating ones are using

nit: Too many "also"

> always VEB mode.
> 2) Handle correctly floating VEB in i40e_veb_release() and
> disallow its release when there are some VSIs. This is
> different from regular VEB that have owner VSI that is
> connected to VEB's uplink after VEB deletion by FW.
> 3) Fix i40e_add_veb() to handle 'vsi' that is NULL for floating
> VEBs. For floating VEB use 0 for downlink SEID and 'true'
> for 'default_port' parameters as per datasheet.
> 4) Fix 'add relay' command in i40e_dbg_command_write() to allow
> to create floating VEB by 'add relay 0 0' or 'add relay'
>
> Tested using debugfs:
> 1) Initial state
> [root@host net-next]# echo dump switch > $CMD
> [root@host net-next]# dmesg -c
> [ 173.701286] i40e 0000:02:00.0: header: 3 reported 3 total
> [ 173.706701] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 173.713241] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 173.719507] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 2) Add floating VEB
> [root@host net-next]# CMD="/sys/kernel/debug/i40e/0000:02:00.0/command"
> [root@host net-next]# echo add relay > $CMD
> [root@host net-next]# dmesg -c
> [ 245.551720] i40e 0000:02:00.0: added relay 162
> [root@host net-next]# echo dump switch > $CMD
> [root@host net-next]# dmesg -c
> [ 276.984371] i40e 0000:02:00.0: header: 4 reported 4 total
> [ 276.989779] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 276.996302] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 277.002569] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
> [ 277.009091] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
>
> 3) Add VMDQ2 VSI to this new VEB
> [root@host net-next]# echo add vsi 162 > $CMD
> [root@host net-next]# dmesg -c
> [ 332.314030] i40e 0000:02:00.0: added VSI 394 to relay 162
> [ 332.337486] enp2s0f0np0v0: NIC Link is Up, 40 Gbps Full Duplex, Flow Control: None
> [root@host net-next]# echo dump switch > $CMD
> [root@host net-next]# dmesg -c
> [ 387.284490] i40e 0000:02:00.0: header: 5 reported 5 total
> [ 387.289904] i40e 0000:02:00.0: type=19 seid=394 uplink=162 downlink=16
> [ 387.296446] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
> [ 387.302708] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 387.309234] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 387.315500] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 4) Try to delete the VEB
> [root@host net-next]# echo del relay 162 > $CMD
> [root@host net-next]# dmesg -c
> [ 428.749297] i40e 0000:02:00.0: deleting relay 162
> [ 428.754011] i40e 0000:02:00.0: can't remove VEB 162 with 1 VSIs left
>
> 5) Do PF reset and check switch status after rebuild
> [root@host net-next]# echo pfr > $CMD
> [root@host net-next]# echo dump switch > $CMD
> [root@host net-next]# dmesg -c
> [ 738.056172] i40e 0000:02:00.0: header: 5 reported 5 total
> [ 738.061577] i40e 0000:02:00.0: type=19 seid=394 uplink=162 downlink=16
> [ 738.068104] i40e 0000:02:00.0: type=17 seid=162 uplink=0 downlink=0
> [ 738.074367] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 738.080892] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 738.087160] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> 6) Delete VSI and delete VEB
> [root@host net-next]# echo del vsi 394 > $CMD
> [root@host net-next]# echo del relay 162 > $CMD
> [root@host net-next]# echo dump switch > $CMD
> [root@host net-next]# dmesg -c
> [ 1233.081126] i40e 0000:02:00.0: deleting VSI 394
> [ 1239.345139] i40e 0000:02:00.0: deleting relay 162
> [ 1244.886920] i40e 0000:02:00.0: header: 3 reported 3 total
> [ 1244.892328] i40e 0000:02:00.0: type=19 seid=392 uplink=160 downlink=16
> [ 1244.898853] i40e 0000:02:00.0: type=17 seid=160 uplink=2 downlink=0
> [ 1244.905119] i40e 0000:02:00.0: type=19 seid=390 uplink=160 downlink=16
>
> Signed-off-by: Ivan Vecera <ivecera@xxxxxxxxxx>

Reviewed-by: Wojciech Drewek <wojciech.drewek@xxxxxxxxx>

> ---
> .../net/ethernet/intel/i40e/i40e_debugfs.c | 29 +++---
> drivers/net/ethernet/intel/i40e/i40e_main.c | 89 ++++++++++---------
> 2 files changed, 68 insertions(+), 50 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_debugfs.c b/drivers/net/ethernet/intel/i40e/i40e_debugfs.c
> index d26348db125f..b2e3bde8ae1d 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_debugfs.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_debugfs.c
> @@ -829,10 +829,14 @@ static ssize_t i40e_dbg_command_write(struct file *filp,
>
> } else if (strncmp(cmd_buf, "add relay", 9) == 0) {
> struct i40e_veb *veb;
> + u8 enabled_tc = 0x1;
> int uplink_seid;
>
> cnt = sscanf(&cmd_buf[9], "%i %i", &uplink_seid, &vsi_seid);
> - if (cnt != 2) {
> + if (cnt == 0) {
> + uplink_seid = 0;
> + vsi_seid = 0;
> + } else if (cnt != 2) {
> dev_info(&pf->pdev->dev,
> "add relay: bad command string, cnt=%d\n",
> cnt);
> @@ -844,23 +848,28 @@ static ssize_t i40e_dbg_command_write(struct file *filp,
> goto command_write_done;
> }
>
> - vsi = i40e_dbg_find_vsi(pf, vsi_seid);
> - if (!vsi) {
> - dev_info(&pf->pdev->dev,
> - "add relay: VSI %d not found\n", vsi_seid);
> - goto command_write_done;
> - }
> -
> veb = i40e_veb_get_by_seid(pf, uplink_seid);
> if (!veb && uplink_seid != 0 && uplink_seid != pf->mac_seid) {
> dev_info(&pf->pdev->dev,
> "add relay: relay uplink %d not found\n",
> uplink_seid);
> goto command_write_done;
> + } else if (uplink_seid) {
> + vsi = i40e_vsi_get_by_seid(pf, vsi_seid);
> + if (!vsi) {
> + dev_info(&pf->pdev->dev,
> + "add relay: VSI %d not found\n",
> + vsi_seid);
> + goto command_write_done;
> + }
> + enabled_tc = vsi->tc_config.enabled_tc;
> + } else if (vsi_seid) {
> + dev_info(&pf->pdev->dev,
> + "add relay: VSI must be 0 for floating relay\n");
> + goto command_write_done;
> }
>
> - veb = i40e_veb_setup(pf, 0, uplink_seid, vsi_seid,
> - vsi->tc_config.enabled_tc);
> + veb = i40e_veb_setup(pf, 0, uplink_seid, vsi_seid, enabled_tc);
> if (veb)
> dev_info(&pf->pdev->dev, "added relay %d\n", veb->seid);
> else
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index e7be145f298a..6ae1206e1c1c 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -10379,41 +10379,48 @@ static int i40e_reconstitute_veb(struct i40e_veb *veb)
> struct i40e_vsi *vsi;
> int v, ret;
>
> - /* build VSI that owns this VEB, temporarily attached to base VEB */
> - i40e_pf_for_each_vsi(pf, v, vsi)
> - if (vsi->veb_idx == veb->idx &&
> - vsi->flags & I40E_VSI_FLAG_VEB_OWNER) {
> - ctl_vsi = vsi;
> - break;
> + if (veb->uplink_seid) {
> + /* Look for VSI that owns this VEB, temporarily attached to base VEB */
> + i40e_pf_for_each_vsi(pf, v, vsi)
> + if (vsi->veb_idx == veb->idx &&
> + vsi->flags & I40E_VSI_FLAG_VEB_OWNER) {
> + ctl_vsi = vsi;
> + break;
> + }
> +
> + if (!ctl_vsi) {
> + dev_info(&pf->pdev->dev,
> + "missing owner VSI for veb_idx %d\n",
> + veb->idx);
> + ret = -ENOENT;
> + goto end_reconstitute;
> }
> + if (ctl_vsi != pf->vsi[pf->lan_vsi])
> + ctl_vsi->uplink_seid =
> + pf->vsi[pf->lan_vsi]->uplink_seid;
>
> - if (!ctl_vsi) {
> - dev_info(&pf->pdev->dev,
> - "missing owner VSI for veb_idx %d\n", veb->idx);
> - ret = -ENOENT;
> - goto end_reconstitute;
> - }
> - if (ctl_vsi != pf->vsi[pf->lan_vsi])
> - ctl_vsi->uplink_seid = pf->vsi[pf->lan_vsi]->uplink_seid;
> - ret = i40e_add_vsi(ctl_vsi);
> - if (ret) {
> - dev_info(&pf->pdev->dev,
> - "rebuild of veb_idx %d owner VSI failed: %d\n",
> - veb->idx, ret);
> - goto end_reconstitute;
> + ret = i40e_add_vsi(ctl_vsi);
> + if (ret) {
> + dev_info(&pf->pdev->dev,
> + "rebuild of veb_idx %d owner VSI failed: %d\n",
> + veb->idx, ret);
> + goto end_reconstitute;
> + }
> + i40e_vsi_reset_stats(ctl_vsi);
> }
> - i40e_vsi_reset_stats(ctl_vsi);
>
> /* create the VEB in the switch and move the VSI onto the VEB */
> ret = i40e_add_veb(veb, ctl_vsi);
> if (ret)
> goto end_reconstitute;
>
> - if (test_bit(I40E_FLAG_VEB_MODE_ENA, pf->flags))
> - veb->bridge_mode = BRIDGE_MODE_VEB;
> - else
> - veb->bridge_mode = BRIDGE_MODE_VEPA;
> - i40e_config_bridge_mode(veb);
> + if (veb->uplink_seid) {
> + if (test_bit(I40E_FLAG_VEB_MODE_ENA, pf->flags))
> + veb->bridge_mode = BRIDGE_MODE_VEB;
> + else
> + veb->bridge_mode = BRIDGE_MODE_VEPA;
> + i40e_config_bridge_mode(veb);
> + }
>
> /* create the remaining VSIs attached to this VEB */
> i40e_pf_for_each_vsi(pf, v, vsi) {
> @@ -14716,29 +14723,29 @@ void i40e_veb_release(struct i40e_veb *veb)
> /* find the remaining VSI and check for extras */
> i40e_pf_for_each_vsi(pf, i, vsi_it)
> if (vsi_it->uplink_seid == veb->seid) {
> - vsi = vsi_it;
> + if (vsi_it->flags & I40E_VSI_FLAG_VEB_OWNER)
> + vsi = vsi_it;
> n++;
> }
>
> - if (n != 1) {
> + /* Floating VEB has to be empty and regular one must have
> + * single owner VSI.
> + */
> + if ((veb->uplink_seid && n != 1) || (!veb->uplink_seid && n != 0)) {
> dev_info(&pf->pdev->dev,
> "can't remove VEB %d with %d VSIs left\n",
> veb->seid, n);
> return;
> }
>
> - /* move the remaining VSI to uplink veb */
> - vsi->flags &= ~I40E_VSI_FLAG_VEB_OWNER;
> + /* For regular VEB move the owner VSI to uplink VEB */
> if (veb->uplink_seid) {
> + vsi->flags &= ~I40E_VSI_FLAG_VEB_OWNER;
> vsi->uplink_seid = veb->uplink_seid;
> if (veb->uplink_seid == pf->mac_seid)
> vsi->veb_idx = I40E_NO_VEB;
> else
> vsi->veb_idx = veb->veb_idx;
> - } else {
> - /* floating VEB */
> - vsi->uplink_seid = pf->vsi[pf->lan_vsi]->uplink_seid;
> - vsi->veb_idx = pf->vsi[pf->lan_vsi]->veb_idx;
> }
>
> i40e_aq_delete_element(&pf->hw, veb->seid, NULL);
> @@ -14756,8 +14763,8 @@ static int i40e_add_veb(struct i40e_veb *veb, struct i40e_vsi *vsi)
> bool enable_stats = !!test_bit(I40E_FLAG_VEB_STATS_ENA, pf->flags);
> int ret;
>
> - ret = i40e_aq_add_veb(&pf->hw, veb->uplink_seid, vsi->seid,
> - veb->enabled_tc, false,
> + ret = i40e_aq_add_veb(&pf->hw, veb->uplink_seid, vsi ? vsi->seid : 0,
> + veb->enabled_tc, vsi ? false : true,
> &veb->seid, enable_stats, NULL);
>
> /* get a VEB from the hardware */
> @@ -14789,9 +14796,11 @@ static int i40e_add_veb(struct i40e_veb *veb, struct i40e_vsi *vsi)
> return -ENOENT;
> }
>
> - vsi->uplink_seid = veb->seid;
> - vsi->veb_idx = veb->idx;
> - vsi->flags |= I40E_VSI_FLAG_VEB_OWNER;
> + if (vsi) {
> + vsi->uplink_seid = veb->seid;
> + vsi->veb_idx = veb->idx;
> + vsi->flags |= I40E_VSI_FLAG_VEB_OWNER;
> + }
>
> return 0;
> }
> @@ -14817,7 +14826,7 @@ struct i40e_veb *i40e_veb_setup(struct i40e_pf *pf, u16 flags,
> u8 enabled_tc)
> {
> struct i40e_veb *veb, *uplink_veb = NULL;
> - struct i40e_vsi *vsi;
> + struct i40e_vsi *vsi = NULL;
> int veb_idx;
> int ret;
>

Next message: Linus Walleij: "Re: [PATCH] pinctrl: avoid reload of p state in interation"
Previous message: Herve Codina: "Re: [PATCH 1/1] lib/vsprintf: Fix %pfwf when current node refcount == 0"
In reply to: Ivan Vecera: "[PATCH iwl-next 4/5] i40e: Fix broken support for floating VEBs"
Next in thread: Ivan Vecera: "[PATCH iwl-next 3/5] i40e: Add helpers to find VSI and VEB by SEID and use them"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]