Re: [PATCH v7 03/17] nvmet: Implement CCR nvme command

From: Dmitry Bogdanov

Date: Tue Nov 17 2026 - 04:38:36 EST


On Mon, Nov 16, 2026 at 04:03:48PM -0800, Mohamed Khalfella wrote:
> On Mon 2026-11-16 11:45:40 +0300, Dmitry Bogdanov wrote:
> > On Fri, Nov 13, 2026 at 07:04:17PM -0800, Mohamed Khalfella wrote:
> > >
> > > Defined by TP8028 Rapid Path Failure Recovery, CCR (Cross-Controller
> > > Reset) command is an nvme command issued to source controller by
> > > initiator to reset impacted controller. Implement CCR command for linux
> > > nvme target.
> > >
> > > Signed-off-by: Mohamed Khalfella <mkhalfella@xxxxxxxxxxxxxxx>
> > > Reviewed-by: Hannes Reinecke <hare@xxxxxxx>
> > > Reviewed-by: Sagi Grimberg <sagi@xxxxxxxxxxx>
> > > ---
> > > drivers/nvme/target/admin-cmd.c | 76 ++++++++++++++++++++++++++++++
> > > drivers/nvme/target/core.c | 83 +++++++++++++++++++++++++++++++++
> > > drivers/nvme/target/nvmet.h | 13 ++++++
> > > include/linux/nvme.h | 23 +++++++++
> > > 4 files changed, 195 insertions(+)
> > >
> > > diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
> > > index 5710ae53eb83..3df9db454db9 100644
> > > --- a/drivers/nvme/target/admin-cmd.c
> > > +++ b/drivers/nvme/target/admin-cmd.c
> > > @@ -378,6 +378,7 @@ static void nvmet_get_cmd_effects_admin(struct nvmet_ctrl *ctrl,
> > > log->acs[nvme_admin_get_features] =
> > > log->acs[nvme_admin_async_event] =
> > > log->acs[nvme_admin_keep_alive] =
> > > + log->acs[nvme_admin_cross_ctrl_reset] =
> > > cpu_to_le32(NVME_CMD_EFFECTS_CSUPP);
> > > }
> > >
> > > @@ -1629,6 +1630,78 @@ void nvmet_execute_keep_alive(struct nvmet_req *req)
> > > nvmet_req_complete(req, status);
> > > }
> > >
> > > +void nvmet_execute_cross_ctrl_reset(struct nvmet_req *req)
> > > +{
> > > + struct nvmet_ctrl *ictrl, *sctrl = req->sq->ctrl;
> > > + struct nvme_command *cmd = req->cmd;
> > > + struct nvmet_ccr *ccr, *new_ccr;
> > > + int ccr_active, ccr_total;
> > > + u16 cntlid, status = NVME_SC_SUCCESS;
> > > +
> > > + if (!nvmet_check_transfer_len(req, 0))
> > > + return;
> > > +
> > > + cntlid = le16_to_cpu(cmd->ccr.icid);
> > > + if (sctrl->cntlid == cntlid) {
> > > + req->error_loc =
> > > + offsetof(struct nvme_cross_ctrl_reset_cmd, icid);
> > > + status = NVME_SC_INVALID_FIELD | NVME_STATUS_DNR;
> > > + goto out;
> > > + }
> >
> > cntlid has to be validated against the configured range of the controller
> > ids (subsys->cntlid_min <= cntlid <= subsys->cntlid_max).
> > That will allow CCR to work correctly on a distributed target.
> >
>
> Thanks for the review. nvmet_ctrl_find_get_ccr() below will verify
> ictrl->dntlid == cntlid before proceeding with the reset. Is this not
> enough?

No, I am talking about something completely different.
On a distributed subsystem the controller id is allocated within each
own range on its node. The controller on one node does not know about
controllers's state on other nodes, the host also does not know where which
controller, it just send CCR on every live controller to get a positive
response. So the controller should fail CCR for the out of ranged impacted
controller id to make the host send other CCR to other controller which
probably know(knew) the impacted controller. Otherwise the host will
complete the outstanding IO too early.


> Adding this check here means if either cntlid_min or cntlid_max change
> after ictrl has been created, CCR will fail saying the controller was not
> found. Why do we want to CCR to fail in this case when ictrl might
> actually be there?

Makes sense to forbid changing cntlid_min and cntlid_max for already discovered
subsystem. It is a static configruation. Or is there a scenario when
they may be changed?

> > > +
> > > + /* Find and get impacted controller */
> > > + ictrl = nvmet_ctrl_find_get_ccr(sctrl->subsys, sctrl->hostnqn,
> > > + cmd->ccr.ciu, cntlid,
> > > + le64_to_cpu(cmd->ccr.cirn));
> > > + if (!ictrl) {
> > > + /* Immediate Reset Successful */
> > > + nvmet_set_result(req, 1);
> > > + status = NVME_SC_SUCCESS;
> > > + goto out;
> > > + }
> > > +
>