[PATCH net] tls: check the right open record in TLS_TX_MAX_PAYLOAD_LEN

From: Rishikesh Jethwani

Date: Mon Oct 19 2026 - 18:28:04 EST


do_tls_setsockopt_tx_payload_len() refuses to change the record size
limit while a record is open, because both TX paths size the record they
are filling against tx_max_payload_len. It looks for the open record
through tls_sw_ctx_tx(), a plain cast of priv_ctx_tx to
struct tls_sw_context_tx.

On a socket whose TX side is HW offloaded (tx_conf == TLS_HW),
priv_ctx_tx is a struct tls_offload_context_tx, so the load lands in an
unrelated field. In the tested x86_64 configuration, open_rec sits at
offset 144, which aliases sg_tx_data[2].dma_address. TLS never DMA-maps
that table, so the check reads zero even while a record is open. This
alias depends on the structure layouts: other configurations can read a
populated field instead and reject a legitimate call with EBUSY.

The missed check matters:

setsockopt(SOL_TLS, TLS_TX, ...) TX offloaded to the NIC
send(fd, buf, N, MSG_MORE) leaves an open_record of N
setsockopt(SOL_TLS, TLS_TX_MAX_PAYLOAD_LEN, M), M < N, accepted
send(fd, buf, len, 0)

tls_push_data() recomputes max_open_record_len from the new limit, so
max_open_record_len - record->len underflows. The next chunk is appended
to the open record regardless of the limit. The record on the wire
exceeds the record_size_limit the peer advertised, which the peer must
answer with a fatal record_overflow alert, exactly what the option exists
to prevent.

Dispatch on tx_conf and look at tls_offload_context_tx::open_record for
TLS_HW. TLS_BASE has no TX context and nothing to check.

Found during review of the TLS core.

Tested on two nodes using mlx5 TLS TX offload. Sending 256 bytes with
MSG_MORE and then lowering the limit returns EBUSY with this patch and is
silently accepted without it. The same sequence on a loopback TLS_SW
socket returns EBUSY either way. With the limit set to 512 before
sending, records carry at most 512 bytes of payload. Without this patch,
lowering the limit after a MSG_MORE send produces a record exceeding the
requested payload limit.

Fixes: 82cb5be6ad64 ("net/tls: support setting the maximum payload size")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Rishikesh Jethwani <rjethwani@xxxxxxxxxxxxxxx>
---
net/tls/tls_main.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c
index fbb274287aa5..8f11b8629492 100644
--- a/net/tls/tls_main.c
+++ b/net/tls/tls_main.c
@@ -837,12 +837,23 @@ static int do_tls_setsockopt_tx_payload_len(struct sock *sk, sockptr_t optval,
unsigned int optlen)
{
struct tls_context *ctx = tls_get_ctx(sk);
- struct tls_sw_context_tx *sw_ctx = tls_sw_ctx_tx(ctx);
u16 value;
bool tls_13 = ctx->prot_info.version == TLS_1_3_VERSION;

- if (sw_ctx && sw_ctx->open_rec)
- return -EBUSY;
+ /* Both TX paths size the record they are filling against
+ * tx_max_payload_len, so refuse to change it under an open record.
+ * priv_ctx_tx is a different struct for each TX configuration.
+ */
+ switch (ctx->tx_conf) {
+ case TLS_SW:
+ if (tls_sw_ctx_tx(ctx)->open_rec)
+ return -EBUSY;
+ break;
+ case TLS_HW:
+ if (tls_offload_ctx_tx(ctx)->open_record)
+ return -EBUSY;
+ break;
+ }

if (sockptr_is_null(optval) || optlen != sizeof(value))
return -EINVAL;
--
2.50.1