RE: [EXTERNAL] [PATCH 00/13] nvme-fc: FPIN link integrity handling
From: Nilesh Javali
Date: Thu Aug 13 2026 - 00:54:43 EST
From: Jesse Taube <jtaubepe@xxxxxxxxxx>
Sent: Wednesday, August 12, 2026 11:43 PM
To: linux-scsi@xxxxxxxxxxxxxxx
Cc: Jonathan Corbet <corbet@xxxxxxx>; Shuah Khan <skhan@xxxxxxxxxxxxxxxxxxx>; Keith Busch <kbusch@xxxxxxxxxx>; Jens Axboe <axboe@xxxxxxxxx>; Christoph Hellwig <hch@xxxxxx>; Sagi Grimberg <sagi@xxxxxxxxxxx>; Justin Tee <justin.tee@xxxxxxxxxxxx>; Naresh Gottumukkala <nareshgottumukkala83@xxxxxxxxx>; Paul Ely <paul.ely@xxxxxxxxxxxx>; Chaitanya Kulkarni <kch@xxxxxxxxxx>; James E.J. Bottomley <James.Bottomley@xxxxxxxxxxxxxxxxxxxxx>; Martin K. Petersen <martin.petersen@xxxxxxxxxx>; Nilesh Javali <njavali@xxxxxxxxxxx>; GR-QLogic-Storage-Upstream@xxxxxxxxxxx; Hannes Reinecke <hare@xxxxxxx>; Jesse Taube <jtaubepe@xxxxxxxxxx>; Gustavo A. R. Silva <gustavoars@xxxxxxxxxx>; John Meneghini <jmeneghi@xxxxxxxxxx>; Bryan Gurney <bgurney@xxxxxxxxxx>; Chris Leech <cleech@xxxxxxxxxx>; Ewan D . Milne <emilne@xxxxxxxxxx>; shinichiro.kawasaki@xxxxxxx; linux-doc@xxxxxxxxxxxxxxx; linux-kernel@xxxxxxxxxxxxxxx; linux-nvme@xxxxxxxxxxxxxxxxxxx; linux-block@xxxxxxxxxxxxxxx
Subject: [EXTERNAL] [PATCH 00/13] nvme-fc: FPIN link integrity handling
FPIN LI (link integrity) messages are received when the attached fabric detects hardware errors. In response to these messages I/O should be directed away from the affected ports, and only used as a last resort. This patch adds a new controller
ZjQcmQRYFpfptBannerStart
Prioritize security for external emails:
Confirm sender and content safety before clicking links or opening attachments
https://us-phishalarm-ewt.proofpoint.com/EWT/v1/CRVmXkqW!te3Z1f8UYnTa1E-duf06bvZlnS-1FrR2Jkcx8WQcjCWqN2c_NKqyKXnJoJcD4hWMKkUnua_-glfm3m67hWRah9Kf0UTKp-APm3w$ ;
ZjQcmQRYFpfptBannerEnd
FPIN LI (link integrity) messages are received when the attached fabric
detects hardware errors. In response to these messages I/O should be
directed away from the affected ports, and only used as a last resort.
This patch adds a new controller flag 'NVME_CTRL_MARGINAL' which will be
checked during multipath path selection, causing the path to be skipped
when checking for paths. If no other paths are available the 'marginal'
paths are considered for path selection.
The testing for this patch set was performed by Bryan Gurney, using the
process outlined by John Meneghini's presentation at LSFMM 2024, where
the fibre channel switch sends an FPIN notification on a specific switch
port, and the following is checked on the initiator:
1. The controllers corresponding to the paths on the port that has
received the notification are showing a set NVME_CTRL_MARGINAL flag.
\
+- nvme4 fc traddr=c,host_traddr=e live optimized
+- nvme5 fc traddr=8,host_traddr=e live non-optimized
+- nvme8 fc traddr=e,host_traddr=f marginal optimized
+- nvme9 fc traddr=a,host_traddr=f marginal non-optimized
2. The I/O statistics of the test namespace show no I/O activity on the
controllers with NVME_CTRL_MARGINAL set.
Device tps MB_read/s MB_wrtn/s MB_dscd/s
nvme4c4n1 0.00 0.00 0.00 0.00
nvme4c5n1 25001.00 0.00 97.66 0.00
nvme4c9n1 25000.00 0.00 97.66 0.00
nvme4n1 50011.00 0.00 195.36 0.00
Device tps MB_read/s MB_wrtn/s MB_dscd/s
nvme4c4n1 0.00 0.00 0.00 0.00
nvme4c5n1 48360.00 0.00 188.91 0.00
nvme4c9n1 1642.00 0.00 6.41 0.00
nvme4n1 49981.00 0.00 195.24 0.00
Device tps MB_read/s MB_wrtn/s MB_dscd/s
nvme4c4n1 0.00 0.00 0.00 0.00
nvme4c5n1 50001.00 0.00 195.32 0.00
nvme4c9n1 0.00 0.00 0.00 0.00
nvme4n1 50016.00 0.00 195.38 0.00
Link: https://urldefense.proofpoint.com/v2/url?u=https-3A__people.redhat.com_jmeneghi_LSFMM-5F2024_LSFMM-5F2024-5FNVMe-5FCancel-5Fand-5FFPIN.pdf&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=7My57sfPbDDcMFJyBD6G2keJbhsQZV7WIf8SjB-450c&m=ViZhyiiF6nkaVXOE1PpaKWe2JBe94gTGHA24wKzPP99uuwXoNoSYqcE55NQRqt6_&s=0jhQeJW2swbeXGSnNtXSHqXkNX1eVjXmU7QSm6m2IpU&e=
Testing has been performed by sending all FPIN LI ELS messages from the
switch to the Host and verifying the proper nvme multi-pathing behavior
is effected with each of the eight different FPIN link integrity events.
Results were verified with iostat and with the nvme list-subsys command.
These tests were run with all scenarios including where there were only
non-optimized paths available, and where all paths were
marginal/degraded. All multi-path io-policies were tested including:
numa, round-robin and queue-depth. When all paths on the host are
marginal/degraded, I/O continues on the optimized path that was most
recently non-marginal. If both of the optimized paths are down, I/O
properly continues on one of the marginal/degraded non-optimized paths.
Testing has been complete with both Broadcom (lpfc) and Marvell (qla2xx)
32GB HBAs. Both HBAs successfully complete all tests.
For a complete description of the tests that were run, please see
bugzilla 220329.
To test the path handeling use:
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_linux-2Dblktests_blktests_pull_264&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=7My57sfPbDDcMFJyBD6G2keJbhsQZV7WIf8SjB-450c&m=ViZhyiiF6nkaVXOE1PpaKWe2JBe94gTGHA24wKzPP99uuwXoNoSYqcE55NQRqt6_&s=XKJyyaOO9_UGG6uH5fv88U7bAke7X9H5Xz0qa39qubs&e=
or
https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_linux-2Dnvme_20260812174503.3705830-2D1-2Djtaubepe-40redhat.com_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=7My57sfPbDDcMFJyBD6G2keJbhsQZV7WIf8SjB-450c&m=ViZhyiiF6nkaVXOE1PpaKWe2JBe94gTGHA24wKzPP99uuwXoNoSYqcE55NQRqt6_&s=qPfemamvpsdH9yAG-9CRAILRyvMrzEaa_xjIaJTo1Ds&e=
To test the full set including FPIN decoding use:
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_johnmeneghini_fpin-5Fli-5Ftests&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=7My57sfPbDDcMFJyBD6G2keJbhsQZV7WIf8SjB-450c&m=ViZhyiiF6nkaVXOE1PpaKWe2JBe94gTGHA24wKzPP99uuwXoNoSYqcE55NQRqt6_&s=SzJUICtyPkmr7xYyCu6_gZb5itPn8MZRQm4z0Zoewak&e=
Closes: https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.kernel.org_show-5Fbug.cgi-3Fid-3D220329&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=7My57sfPbDDcMFJyBD6G2keJbhsQZV7WIf8SjB-450c&m=ViZhyiiF6nkaVXOE1PpaKWe2JBe94gTGHA24wKzPP99uuwXoNoSYqcE55NQRqt6_&s=Ybu1spaO4qBWAVWu7hq8KHIP73xHRmGPfJCrZVoqacY&e=
New refactored implementation enables administrators to manually control
port marginal states via sysfs. For example:
# Set remote port to marginal state
echo "Marginal" > /sys/class/fc_remote_ports/rport-4:0-1/port_state
# Clear marginal state (set to online)
echo "Online" > /sys/class/fc_remote_ports/rport-4:0-1/port_state
Changes to the original submission:
- Changed flag name to 'marginal'
- Do not block marginal path; influence path selection instead
to de-prioritize marginal paths
Changes to v2:
- Split off driver-specific modifications
- Introduce 'union fc_tlv_desc' to avoid casts
Changes to v3:
- Include reviews from Justin Tee
- Split marginal path handling patch
Changes to v4:
- Change 'u8' to '__u8' on fc_tlv_desc to fix a failure to build
- Print 'marginal' instead of 'live' in the state of controllers
when they are marginal
Changes to v5:
- Minor spelling corrections to patch descriptions
Changes to v6:
- No code changes; added note about additional testing
Changes to v7:
- Split nvme core marginal flag addition into its own patch
- Add patch for queue_depth marginal path support
Changes to v8:
- Rebased patch series to nvme-6.17.
- Added patch from Gustavo Silva, "scsi: qla2xxx: Fix memcpy field-spanning
write issue", which resolves the field-spanning write issue
- We decided to leave the "marginal" state as is, because the transport
driver uses the term "marginal".
Changes to v9:
- Rebased patch series to nvme-6.18.
- Refactor and fix a patch from Gustavo Silva, "scsi: qla2xxx: Fix 2 memcpy
field-spanning write issue", which resolves the field-spanning write
issue.
This new version of Gustavo's patch fixes a bug found in testing.
[NJ:] I could not find this qla2xxx fix in this series. Am I missing something.
This fix is certainly required to avoid few call traces reported, as discussed here,
https://lore.kernel.org/linux-nvme/97526d45-ec7d-48a0-bdc6-659f75839f53@xxxxxxxxxxxxxx/#t
Thanks,
Nilesh