[PATCH] serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

[Nicolas Thibert]

__stop_tx() (8250_port.c) only calls the RS485 rs485_stop_tx() hook
(which de-asserts the direction GPIO/RTS line) once it has observed
both UART_LSR_THRE and UART_LSR_TEMT for the last byte. If TEMT is
never seen and the driver hasn't set UART_CAP_NOTEMT, the function
returns without scheduling any retry -- the direction line is left
asserted (driver enabled) forever, with nothing to un-stick it short
of another kernel-visible LSR event.

of_platform_serial_setup() unconditionally wires up the generic em485
GPIO-RTS RS485 support (rs485_config/rs485_start_tx/rs485_stop_tx) for
every port it registers, but never sets UART_CAP_NOTEMT, so any board
using this driver whose 16550-compatible core doesn't reliably surface
TEMT for its shift register hits the stuck-direction-GPIO case above.

Confirmed live on an ath79 QCA9531 board (SoC-internal ns16550a-
compatible UART, RS485 transceiver DE/RE tied together on a GPIO via
rts-gpios, linux,rs485-enabled-at-boot-time): the direction GPIO
correctly asserts for the duration of a transmit, but never
de-asserts afterwards -- confirmed by sampling the GPIO's debugfs
state through and after a multi-hundred-byte write, on both the first
transmit and repeated back-to-back transmits. Setting
UART_CAP_NOTEMT, which makes __stop_tx() fall back to a frame-time-
based timer instead of waiting indefinitely on TEMT, makes the
direction GPIO reliably return low right after each transmit
completes.

Scope the fix to ports that declare a GPIO-controlled direction line
(rts-gpios), rather than setting it unconditionally for every port
this driver registers: this is the class of hardware actually
affected (RTS state has to be explicitly un-stuck by software, unlike
a UART's native RTS pin), and it avoids adding the extra frame-time
margin to ports relying on the native RTS pin, which has not been
observed to need it.

Signed-off-by: Nicolas Thibert <nithibert@xxxxxxxxx>
Assisted-by: LLM (Claude Sonnet 5, Anthropic)
---
 drivers/tty/serial/8250/8250_of.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

--- a/drivers/tty/serial/8250/8250_of.c
+++ b/drivers/tty/serial/8250/8250_of.c
@@ -156,6 +156,22 @@ static int of_platform_serial_setup(struct platform_device *ofdev,
 	up->rs485_start_tx = serial8250_em485_start_tx;
 	up->rs485_stop_tx = serial8250_em485_stop_tx;

+	/*
+	 * This generic driver never enables a dedicated line-status
+	 * interrupt on TEMT, so for ports whose RS485 direction is
+	 * controlled via a GPIO (rts-gpios) rather than the native RTS
+	 * pin, __stop_tx() (8250_port.c) can see THRE without TEMT on the
+	 * last byte and bail out without ever retrying -- leaving the
+	 * direction GPIO stuck asserted after the last byte sent, on
+	 * hardware whose shift register doesn't reliably surface TEMT.
+	 * UART_CAP_NOTEMT makes it fall back to a frame-time-based timer
+	 * instead of waiting on that interrupt. Scoped to rts-gpios users
+	 * only, to avoid changing timing for ports relying on the native
+	 * RTS pin, which this has not been observed to affect.
+	 */
+	if (of_property_present(np, "rts-gpios"))
+		up->capabilities |= UART_CAP_NOTEMT;
+
 	switch (type) {
 	case PORT_RT2880:
 		ret = rt288x_setup(port);