[PATCH v8 1/2] serial: core: fix baud rate fallback in uart_get_baud_rate()

[Hui Peng]

When uart_get_baud_rate() evaluates a requested baud rate against a port's
supported limits [min, max], the retry loop operates as follows:

  try == 0: Evaluates the requested baud rate. If out of range and an old
            termios is available, it switches termios to old.
  try == 1: If old is also out of range (or unavailable), the fallback rule
            at the end of the loop clips baud to [min, max - 1] and encodes
            it into termios via tty_termios_encode_baud_rate(termios, baud,
            baud).
  try == 2: Evaluates baud = tty_termios_baud_rate(termios) for the newly
            encoded clipped rate, which now satisfies min <= baud && baud
            <= max and returns baud from within the loop body.

However, because the current loop bound is for (try = 0; try < 2; try++),
the loop terminates immediately after try == 1 without executing try == 2 to
re-evaluate the clipped rate. Upon loop exit, the function hits WARN_ON(1)
and returns 0, which triggers a fatal divide-by-zero (Oops: divide error)
in uart_get_divisor():

  WARNING: drivers/tty/serial/serial_core.c:548 at uart_get_baud_rate+0x136/0x260
  [ ... ]
  divide error: 0000 [#1] PREEMPT SMP KASAN

Increase the loop retry limit in uart_get_baud_rate() from 2 to 3 iterations
(try < 3) so that clipped fallback baud rates are re-evaluated in try == 2.

Tested in QEMU against Linux 7.3.0-rc3 by setting B4000000 on /dev/ttyS0 via
tcsetattr(): on the unfixed kernel it triggers WARN_ON(1) and divide-by-zero
Oops, whereas with this fix applied uart_get_baud_rate() smoothly falls back
to 115200 without error.

Fixes: 091ea8e5d34e ("serial: core: prevent division by zero by always returning non-zero baud rate")
Cc: stable@xxxxxxxxxxxxxxx
Assisted-by: LLM
Signed-off-by: Hui Peng <benquike@xxxxxxxxx>
---
Changes in v8:
- Rewrote commit description to detail the step-by-step loop iteration
  progression (try == 0, try == 1, try == 2) as requested by Ilpo Järvinen
  and Greg Kroah-Hartman.

 drivers/tty/serial/serial_core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/tty/serial/serial_core.c b/drivers/tty/serial/serial_core.c
index f91bcfa30113..6ed0195e6912 100644
--- a/drivers/tty/serial/serial_core.c
+++ b/drivers/tty/serial/serial_core.c
@@ -470,7 +470,7 @@ unsigned int uart_get_baud_rate(struct uart_port *port, struct ktermios *termi
 	 * Ask the low level driver to verify the baud rate if it can't
 	 * then it will have encode_baud_rate set the Closet else .
 	 */
-	for (try = 0; try < 2; try++) {
+	for (try = 0; try < 3; try++) {
 		baud = tty_termios_baud_rate(termios);

 		/*
-- 
2.47.3