[PATCH] nvmet-tcp: report a bounded MDTS instead of "no limit"

[Alfonso Kuen]

nvmet-tcp does not implement .get_mdts, so nvmet_ctrl_mdts() falls back
to 0 and identify-controller advertises "no maximum data transfer size".
The initiator takes that literally: max_hw_sectors becomes UINT_MAX, and
the block layer then merges requests up to its generic ceiling -- 32 MiB
on the hosts we measured.

The target cannot actually serve those. nvmet_tcp_map_data() allocates
the command scatterlist with

	cmd->req.sg = sgl_alloc(len, GFP_KERNEL | __GFP_NOWARN,
				&cmd->req.sg_cnt);
	if (!cmd->req.sg)
		return NVME_SC_INTERNAL;

For a 32 MiB command that is 8192 scatterlist entries, so
sgl_alloc_order() asks kmalloc for an order-5/6 block. Under memory
fragmentation that fails, and because of __GFP_NOWARN it fails silently
-- nothing is logged on the target side. The initiator gets
NVME_SC_INTERNAL, which is a generic status, so NVMe multipath does not
fail the command over to another path. A 1 MiB command needs 256
entries, an order-0 allocation, which does not fail. This is why the
failure is intermittent and load-dependent rather than deterministic.

nvmet-rdma has advertised a bounded MDTS since the series "nvmet: Add
mdts setting op for controllers" (Max Gurtovoy, Mar 2020), which
deliberately left other transports untouched. Do the same for TCP, using
the same 1 MiB value, so initiators size their requests to something the
target can allocate.

Measured against a Linux nvmet TCP target with three initiators: of
~1040 failed I/O commands, 980 were exactly 65536 blocks (32 MiB).
Capping the initiator side with max_sectors_kb=1024 removed them
entirely. With the failures reaching a page-cached writer, a 2 TiB image
copy lost roughly 98 GiB while the copy tool exited 0.

Signed-off-by: Alfonso Kuen <gerencia@xxxxxxxxxxxxxx>
---
 drivers/nvme/target/tcp.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index e4f603b2a..414ba896e 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -23,6 +23,9 @@
 #include "nvmet.h"
 
 #define NVMET_TCP_DEF_INLINE_DATA_SIZE	(4 * PAGE_SIZE)
+
+/* Assume mpsmin == device_page_size == 4KB */
+#define NVMET_TCP_MAX_MDTS		8
 #define NVMET_TCP_MAXH2CDATA		0x400000 /* 16M arbitrary limit */
 #define NVMET_TCP_BACKLOG 128
 
@@ -2243,6 +2246,11 @@ static ssize_t nvmet_tcp_host_port_addr(struct nvmet_ctrl *ctrl,
 			(struct sockaddr *)&queue->sockaddr_peer);
 }
 
+static u8 nvmet_tcp_get_mdts(const struct nvmet_ctrl *ctrl)
+{
+	return NVMET_TCP_MAX_MDTS;
+}
+
 static const struct nvmet_fabrics_ops nvmet_tcp_ops = {
 	.owner			= THIS_MODULE,
 	.type			= NVMF_TRTYPE_TCP,
@@ -2254,6 +2262,7 @@ static const struct nvmet_fabrics_ops nvmet_tcp_ops = {
 	.install_queue		= nvmet_tcp_install_queue,
 	.disc_traddr		= nvmet_tcp_disc_port_addr,
 	.host_traddr		= nvmet_tcp_host_port_addr,
+	.get_mdts		= nvmet_tcp_get_mdts,
 };
 
 static int __init nvmet_tcp_init(void)
-- 
2.47.3