Re: [Intel-wired-lan] [PATCH iwl v3] ice: acquire NVM lock around each flash read

[Robert Malz]

Hey Marcin,
Thanks for the review. Changes applied in v4.
I removed ice_read_sr_word_aq instead of ice_read_sr_word as callers
already are using ice_read_sr_word.

v4 ref: https://lists.osuosl.org/pipermail/intel-wired-lan/Week-of-Mon-20260803/056281.html

Thanks,
Robert

On Mon, Aug 3, 2026 at 12:05 PM Marcin Szycik
<marcin.szycik@xxxxxxxxxxxxxxx> wrote:
>
>
>
> On 31/07/2026 13:10, Robert Malz via Intel-wired-lan wrote:
> > FW caps the NVM read lock at a maximum of 3000ms regardless of the timeout
> > requested via ice_acquire_nvm(). ice_read_flat_nvm() splits a read into
> > multiple ice_aq_read_nvm() commands, one per 4KB sector, all issued under a
> > single lock taken by the caller. Reading a large region can exceed 3000ms,
> > so FW reclaims the lock mid-read and the remaining commands might fail.
> >
> > Move the lock acquire/release into ice_read_flat_nvm() so it brackets each
> > individual ice_aq_read_nvm() command, ensuring the lock is never held
> > across more than one FW read.
> >
> > ice_release_nvm() issues its own AQ command and overwrites
> > hw->adminq.sq_last_status, which some callers inspect after a failed read.
> > Add an optional read_aq_err output parameter to ice_read_flat_nvm() to
> > capture the failing read's AQ error before the release; callers that need
> > it (ice_discover_flash_size() and the ethtool/devlink log paths) use it
> > instead of sq_last_status, others pass NULL.
> >
> > Callers that previously took the lock around ice_read_flat_nvm(),
> > ice_read_sr_word() or ice_read_flash_module() now call them without it.
> > The now-redundant per-block locking in ice_devlink_nvm_snapshot() is
> > dropped.
> >
> > Fixes: e94509906d6b ("ice: create function to read a section of the NVM and Shadow RAM")
> > Signed-off-by: Robert Malz <robert.malz@xxxxxxxxxxxxx>
> > ---
> > v3:
> > - Log the failure via ice_debug() when ice_acquire_nvm() fails inside
> >   ice_read_flat_nvm(), rather than silently aborting the read.
> > v2:
> > - Replace the save/restore of sq_last_status across ice_release_nvm(),
> >   which could race with a concurrent AdminQ command, with a new optional
> >   read_aq_err output parameter.
> > - Add missing "Return:" kdoc to ice_read_sr_word().
> > ---
> >  .../net/ethernet/intel/ice/devlink/devlink.c  | 32 ++------
> >  drivers/net/ethernet/intel/ice/ice_ethtool.c  | 16 +---
> >  drivers/net/ethernet/intel/ice/ice_nvm.c      | 79 +++++++++++--------
> >  drivers/net/ethernet/intel/ice/ice_nvm.h      |  2 +-
> >  4 files changed, 59 insertions(+), 70 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/intel/ice/devlink/devlink.c b/drivers/net/ethernet/intel/ice/devlink/devlink.c
> > index 22b7d8e6bd9e..5a1ab9654fb8 100644
> > --- a/drivers/net/ethernet/intel/ice/devlink/devlink.c
> > +++ b/drivers/net/ethernet/intel/ice/devlink/devlink.c
> > @@ -1856,6 +1856,7 @@ static int ice_devlink_nvm_snapshot(struct devlink *devlink,
> >  {
> >       struct ice_pf *pf = devlink_priv(devlink);
> >       struct device *dev = ice_pf_to_dev(pf);
> > +     enum libie_aq_err read_aq_err = LIBIE_AQ_RC_OK;
>
> RCT
> Also scope can be reduced.
>
> >       struct ice_hw *hw = &pf->hw;
> >       bool read_shadow_ram;
> >       u8 *nvm_data, *tmp, i;
> > @@ -1891,26 +1892,16 @@ static int ice_devlink_nvm_snapshot(struct devlink *devlink,
> >       for (i = 0; i < num_blks; i++) {
> >               u32 read_sz = min_t(u32, ICE_DEVLINK_READ_BLK_SIZE, left);
> >
> > -             status = ice_acquire_nvm(hw, ICE_RES_READ);
> > -             if (status) {
> > -                     dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n",
> > -                             status, hw->adminq.sq_last_status);
> > -                     NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore");
> > -                     vfree(nvm_data);
> > -                     return -EIO;
> > -             }
> > -
> >               status = ice_read_flat_nvm(hw, i * ICE_DEVLINK_READ_BLK_SIZE,
> > -                                        &read_sz, tmp, read_shadow_ram);
> > +                                        &read_sz, tmp, read_shadow_ram,
> > +                                        &read_aq_err);
> >               if (status) {
> >                       dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
> > -                             read_sz, status, hw->adminq.sq_last_status);
> > +                             read_sz, status, read_aq_err);
> >                       NL_SET_ERR_MSG_MOD(extack, "Failed to read NVM contents");
> > -                     ice_release_nvm(hw);
> >                       vfree(nvm_data);
> >                       return -EIO;
> >               }
> > -             ice_release_nvm(hw);
> >
> >               tmp += read_sz;
> >               left -= read_sz;
> > @@ -1945,6 +1936,7 @@ static int ice_devlink_nvm_read(struct devlink *devlink,
> >  {
> >       struct ice_pf *pf = devlink_priv(devlink);
> >       struct device *dev = ice_pf_to_dev(pf);
> > +     enum libie_aq_err read_aq_err = LIBIE_AQ_RC_OK;
>
> RCT
>
> >       struct ice_hw *hw = &pf->hw;
> >       bool read_shadow_ram;
> >       u64 nvm_size;
>
> ...
>
> >  /**
> > - * ice_read_sr_word - Reads Shadow RAM word and acquire NVM if necessary
> > + * ice_read_sr_word - Reads Shadow RAM word
> >   * @hw: pointer to the HW structure
> >   * @offset: offset of the Shadow RAM word to read (0x000000 - 0x001FFF)
> >   * @data: word read from the Shadow RAM
> >   *
> > - * Reads one 16 bit word from the Shadow RAM using the ice_read_sr_word_aq.
> > + * Reads one 16 bit word from the Shadow RAM using ice_read_sr_word_aq.
> > + *
> > + * The NVM lock is acquired and released internally by ice_read_flat_nvm()
> > + * around the FW read, so this function must be called without the lock held.
> > + *
> > + * Return: zero on success, or a negative error code on failure.
> >   */
> >  int ice_read_sr_word(struct ice_hw *hw, u16 offset, u16 *data)
> >  {
> > -     int status;
> > -
> > -     status = ice_acquire_nvm(hw, ICE_RES_READ);
> > -     if (!status) {
> > -             status = ice_read_sr_word_aq(hw, offset, data);
> > -             ice_release_nvm(hw);
> > -     }
> > -
> > -     return status;
> > +     return ice_read_sr_word_aq(hw, offset, data);
>
> ice_read_sr_word() is now a wrapper - can we remove it and use
> ice_read_sr_word_aq() directly?
>
> >  }
> >
> >  /**
> > @@ -856,20 +876,18 @@ int ice_get_inactive_netlist_ver(struct ice_hw *hw, struct ice_netlist_info *net
> >  static int ice_discover_flash_size(struct ice_hw *hw)
> >  {
> >       u32 min_size = 0, max_size = ICE_AQC_NVM_MAX_OFFSET + 1;
> > -     int status;
> > -
> > -     status = ice_acquire_nvm(hw, ICE_RES_READ);
> > -     if (status)
> > -             return status;
> > +     int status = 0;
> >
> >       while ((max_size - min_size) > 1) {
> >               u32 offset = (max_size + min_size) / 2;
> > +             enum libie_aq_err read_aq_err = LIBIE_AQ_RC_OK;
>
> RCT
>
> >               u32 len = 1;
> >               u8 data;
>
> ---8<---
>
> Thanks,
> Marcin