Re: [PATCH 0/6] libsas and drivers: NCQ error handling

[John Garry]

On 11/08/2022 19:54, Damien Le Moal wrote:
> On 2022/07/22 4:24, John Garry wrote:
> > As reported in [0], the pm8001 driver NCQ error handling more or less
> > duplicates what libata does in link error handling, as follows:
> > - abort all commands
> > - do autopsy with read log ext 10 command
> > - reset the target to recover
> >
> > Indeed for the hisi_sas driver we want to add similar handling for NCQ
> > errors.
> >
> > This series add a new libsas API - sas_ata_link_abort() - to handle host
> > NCQ errors, and fixes up pm8001 and hisi_sas drivers to use it. As
> > mentioned in the pm8001 changeover patch, I would prefer a better place to
> > locate the SATA ABORT command (rather that nexus reset callback).
> >
> > I would appreciate some testing of the pm8001 change as the read log ext10
> > command mostly hangs on my arm64 machine - these arm64 hangs are a known
> > issue.

Thanks for this!

> I applied this series on top of the current Linus tree and ran some tests: a
> bunch of fio runs and also ran libzbc test suites on a SATA SMR drive as that
> generates many command failures. No problems detected, the tests all pass.
> FYI, messages for failed commands look like this:
>
> pm80xx0:: mpi_sata_event 2685: SATA EVENT 0x23
> sas: Enter sas_scsi_recover_host busy: 1 failed: 1
> sas: sas_scsi_find_task: aborting task 0x00000000ba62a907
> pm80xx0:: mpi_sata_completion 2292: task null, freeing CCB tag 2
> sas: sas_scsi_find_task: task 0x00000000ba62a907 is aborted
> sas: sas_eh_handle_sas_errors: task 0x00000000ba62a907 is aborted
> ata21.00: exception Emask 0x0 SAct 0x20000000 SErr 0x0 action 0x0
> ata21.00: failed command: WRITE FPDMA QUEUED
> ata21.00: cmd 61/02:00:ff:ff:ea/00:00:02:00:00/40 tag 29 ncq dma 8192 out
> res 43/04:02:ff:ff:ea/00:00:02:00:00/00 Emask 0x400 (NCQ error) <F>
> ata21.00: status: { DRDY SENSE ERR }
> ata21.00: error: { ABRT }
> ata21.00: configured for UDMA/133
> ata21: EH complete
> sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 1 tries: 1

For this specific test we don't seem to run a hardreset after the 
autopsy, but we do seem to be getting an NCQ error. That's interesting.

We have noticed this scenario for hisi_sas NCQ error, whereby the 
autopsy decided a reset is not required or useful, such as a medium 
error. Anyway the pm8001 driver relies on the reset being run always for 
the NCQ error. So I am thinking of tweaking sas_ata_link_abort() as follows:

void sas_ata_link_abort(struct domain_device *device)
{
	struct ata_port *ap = device->sata_dev.ap;
	struct ata_link *link = &ap->link;

	link->eh_info.err_mask |= AC_ERR_DEV;
+	link->eh_info.action |= ATA_EH_RESET;
	ata_link_abort(link);
}

This should force a reset.

Thanks,
John

> Seems all good to me.
>
> > Finally with these changes we can make the libsas task alloc/free APIs
> > private, which they should always have been.
> >
> > Based on v5.19-rc6
> >
> > [0] https://lore.kernel.org/linux-scsi/8fb3b093-55f0-1fab-81f4-e8519810a978@xxxxxxxxxx/
> >
> > John Garry (5):
> >    scsi: pm8001: Modify task abort handling for SATA task
> >    scsi: libsas: Add sas_ata_link_abort()
> >    scsi: pm8001: Use sas_ata_link_abort() to handle NCQ errors
> >    scsi: hisi_sas: Don't issue ATA softreset in hisi_sas_abort_task()
> >    scsi: libsas: Make sas_{alloc, alloc_slow, free}_task() private
> >
> > Xingui Yang (1):
> >    scsi: hisi_sas: Add SATA_DISK_ERR bit handling for v3 hw
> >
> >   drivers/scsi/hisi_sas/hisi_sas_main.c  |   5 +-
> >   drivers/scsi/hisi_sas/hisi_sas_v3_hw.c |  22 ++-
> >   drivers/scsi/libsas/sas_ata.c          |  10 ++
> >   drivers/scsi/libsas/sas_init.c         |   3 -
> >   drivers/scsi/libsas/sas_internal.h     |   4 +
> >   drivers/scsi/pm8001/pm8001_hwi.c       | 194 +++++++------------------
> >   drivers/scsi/pm8001/pm8001_sas.c       |  13 ++
> >   drivers/scsi/pm8001/pm8001_sas.h       |   8 +-
> >   drivers/scsi/pm8001/pm80xx_hwi.c       | 177 ++--------------------
> >   include/scsi/libsas.h                  |   4 -
> >   include/scsi/sas_ata.h                 |   5 +
> >   11 files changed, 132 insertions(+), 313 deletions(-)