Re: [PATCH v2] serial: 8250_dw: Fix common clocks usage race condition

[Maxime Ripard]

On Mon, Mar 23, 2020 at 04:50:17PM +0300, Sergey Semin wrote:
> On Mon, Mar 23, 2020 at 11:01:09AM +0100, Maxime Ripard wrote:
> > Hi,
> >
> > On Mon, Mar 23, 2020 at 05:46:09AM +0300, Sergey.Semin@xxxxxxxxxxxxxxxxxxxx wrote:
> > > From: Serge Semin <Sergey.Semin@xxxxxxxxxxxxxxxxxxxx>
> > >
> > > There are races possible in the dw8250_set_termios() callback method
> > > and while the device is in PM suspend state. A race condition may
> > > happen if the baudrate clock source device is shared with some other
> > > device (in our machine it's another DW UART port). In this case if that
> > > device changes the clock rate while serial console is using it the
> > > DW 8250 UART port might not only end up with an invalid uartclk value
> > > saved, but may also experience a distorted output data since baud-clock
> > > could have been changed. In order to fix this lets enable an exclusive
> > > reference clock rate access in case if "baudclk" device is specified.
> > >
> > > So if some other device also acquires the rate exclusivity during the
> > > time of a DW UART 8250 port being opened, then DW UART 8250 driver
> > > won't be able to alter the baud-clock. It shall just use the available
> > > clock rate. Similarly another device also won't manage to change the
> > > rate at that time. If nothing else have the exclusive rate access
> > > acquired except DW UART 8250 driver, then the driver will be able to
> > > alter the rate as much as it needs to in accordance with the currently
> > > implemented logic.
> > >
> > > Signed-off-by: Serge Semin <Sergey.Semin@xxxxxxxxxxxxxxxxxxxx>
> > > Cc: Alexey Malahov <Alexey.Malahov@xxxxxxxxxxxxxxxxxxxx>
> > > Cc: Maxim Kaurkin <Maxim.Kaurkin@xxxxxxxxxxxxxxxxxxxx>
> > > Cc: Pavel Parkhomenko <Pavel.Parkhomenko@xxxxxxxxxxxxxxxxxxxx>
> > > Cc: Ramil Zaripov <Ramil.Zaripov@xxxxxxxxxxxxxxxxxxxx>
> > > Cc: Ekaterina Skachko <Ekaterina.Skachko@xxxxxxxxxxxxxxxxxxxx>
> > > Cc: Vadim Vlasov <V.Vlasov@xxxxxxxxxxxxxxxxxxxx>
> > > Cc: Thomas Bogendoerfer <tsbogend@xxxxxxxxxxxxxxxx>
> > > Cc: Paul Burton <paulburton@xxxxxxxxxx>
> > > Cc: Ralf Baechle <ralf@xxxxxxxxxxxxxx>
> > > Cc: Maxime Ripard <mripard@xxxxxxxxxx>
> > > Cc: Chen-Yu Tsai <wens@xxxxxxxx>
> > > CC: Ray Jui <rjui@xxxxxxxxxxxx>
> > > Cc: Scott Branden <sbranden@xxxxxxxxxxxx>
> > > Cc: Florian Fainelli <f.fainelli@xxxxxxxxx>
> > > Cc: Wei Xu <xuwei5@xxxxxxxxxxxxx>
> > > Cc: Jason Cooper <jason@xxxxxxxxxxxxxx>
> > > Cc: Andrew Lunn <andrew@xxxxxxx>
> > > Cc: Gregory Clement <gregory.clement@xxxxxxxxxxx>
> > > Cc: Sebastian Hesselbarth <sebastian.hesselbarth@xxxxxxxxx>
> > > Cc: Jisheng Zhang <Jisheng.Zhang@xxxxxxxxxxxxx>
> > > Cc: Heiko Stuebner <heiko@xxxxxxxxx>
> > > Cc: Catalin Marinas <catalin.marinas@xxxxxxx>
> > > Cc: Will Deacon <will@xxxxxxxxxx>
> > > Cc: Russell King <linux@xxxxxxxxxxxxxxx>
> > > Cc: linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
> > > Cc: Michael Turquette <mturquette@xxxxxxxxxxxx>
> > > Cc: Stephen Boyd <sboyd@xxxxxxxxxx>
> > > Cc: linux-clk@xxxxxxxxxxxxxxx
> > >
> > > ---
> > >
> > > Changelog v2:
> > > - Move exclusive ref clock lock/unlock precudures to the 8250 port
> > >   startup/shutdown methods.
> > > - The changelog message has also been slightly modified due to the
> > >   alteration.
> > > - Remove Alexey' SoB tag.
> > > - Cc someone from ARM who might be concerned regarding this change.
> > > - Cc someone from Clocks Framework to get their comments on this patch.
> > > ---
> > >  drivers/tty/serial/8250/8250_dw.c | 36 +++++++++++++++++++++++++++++++
> > >  1 file changed, 36 insertions(+)
> > >
> > > diff --git a/drivers/tty/serial/8250/8250_dw.c b/drivers/tty/serial/8250/8250_dw.c
> > > index aab3cccc6789..08f3f745ed54 100644
> > > --- a/drivers/tty/serial/8250/8250_dw.c
> > > +++ b/drivers/tty/serial/8250/8250_dw.c
> > > @@ -319,6 +319,40 @@ static void dw8250_set_ldisc(struct uart_port *p, struct ktermios *termios)
> > >  	serial8250_do_set_ldisc(p, termios);
> > >  }
> > >
> > > +static int dw8250_startup(struct uart_port *p)
> > > +{
> > > +	struct dw8250_data *d = to_dw8250_data(p->private_data);
> > > +
> > > +	/*
> > > +	 * Some platforms may provide a reference clock shared between several
> > > +	 * devices. In this case before using the serial port first we have to
> > > +	 * make sure nothing will change the rate behind our back and second
> > > +	 * the tty/serial subsystem knows the actual reference clock rate of
> > > +	 * the port.
> > > +	 */
> > > +	if (clk_rate_exclusive_get(d->clk)) {
> > > +		dev_warn(p->dev, "Couldn't lock the clock rate\n");
> > > +	} else if (d->clk) {
> > > +		p->uartclk = clk_get_rate(d->clk);
> > > +		if (!p->uartclk) {
> > > +			clk_rate_exclusive_put(d->clk);
> > > +			dev_err(p->dev, "Clock rate not defined\n");
> > > +			return -EINVAL;
> > > +		}
> > > +	}
> > > +
> > > +	return serial8250_do_startup(p);
> > > +}
> >
> > I've been facing that issue, so it would be great to get it fixed, but
> > I'm not sure this is the right solution.
> >
> > clk_rate_exclusive_get is pretty intrusive, and due to the usual
> > topology of clock trees, this will lock down 3-4 parent clocks to
> > their current rate as well. In the Allwinner SoCs case for example,
> > this will lock down the same PLL than the one used by the CPU,
> > preventing cpufreq from running.
> >
>
> Speaking about weak design of a SoC' clock tree. Our problems are nothing
> with respect to the Allwinner SoC, in which case of changing the
> CPU-frequency may cause the UART glitches subsequently causing data
> transfer artefacts.)

I just remembered the RPi3/4 are in this situation too.

> Moreover as I can see the same issue may raise for I2C, QSPI, PWM
> devices there.

We also have the I2C on the same parent clock in the Allwinner SoCs
indeed :)

> Anyway your concern does make sense.
>
> > However, the 8250 has a pretty wide range of dividers and can adapt to
> > any reasonable parent clock rate, so we don't really need to lock the
> > rate either, we can simply react to a parent clock rate change using
> > the clock notifiers, just like the SiFive UART is doing.
> >
> > I tried to do that, but given that I don't really have an extensive
> > knowledge of the 8250, I couldn't find a way to stop the TX of chars
> > while we change the clock rate. I'm not sure if this is a big deal or
> > not, the SiFive UART doesn't seem to care.
>
> Yes, your solution is also possible, but even in case of stopping Tx/Rx it
> doesn't lack drawbacks. First of all AFAIK there is no easy way to just
> pause the transfers. We'd have to first wait for the current transfers
> to be completed, then somehow lock the port usage (both Tx and Rx
> traffic), permit the reference clock rate change, accordingly adjust the
> UART clock divider, and finally unlock the port. While if we don't mind
> to occasionally have UART data glitches, we can just adjust the UART ref
> divider synchronously with ref clock rate change as you and SiFive UART
> driver suggest.
>
> So we are now at a zugzwang - a fork to three not that good solutions:
> 1) lock the whole clock branch and provide a glitchless interfaces. But
> by doing so we may (in case of Allwinner SoCs we will) lockup some very
> important functionality like CPU-frequency change while the UART port is
> started up. In this case we won't have the data glitches.

This solution, at least without a whitelist of platforms where it
makes sense, would be NAK'd for me. And I'm strongly suspecting that
that whitelist would be very short, so the overall usefulness is
pretty limited.

Maxime

Attachment: signature.asc
Description: PGP signature