Re: DM3730 Bluetooth Performance differences between SERIAL_8250_OMAP vs SERIAL_OMAP

[Adam Ford]

On Fri, Oct 4, 2019 at 11:51 AM Adam Ford <aford173@xxxxxxxxx> wrote:
>
> On Fri, Oct 4, 2019 at 9:08 AM Adam Ford <aford173@xxxxxxxxx> wrote:
> >
> > On Fri, Oct 4, 2019 at 7:27 AM Yegor Yefremov
> > <yegorslists@xxxxxxxxxxxxxx> wrote:
> > >
> > > Hi Adam,
> > >
> > > On Fri, Oct 4, 2019 at 12:39 PM Adam Ford <aford173@xxxxxxxxx> wrote:
> > > >
> > > > On Fri, Oct 4, 2019 at 5:02 AM Adam Ford <aford173@xxxxxxxxx> wrote:
> > > > >
> > > > > I am running Kernel 5.3.2 trying to troubleshoot some intermittent
> > > > > Bluetooth issues, and I think I have narrowed it down to the serial
> > > > > driver in use.
> > > >
> > > > I should have also noted that it's using UART2 with CTS and RTS on the
> > > > DM3730 (omap3630) and its configured with a baud rate of 3M.
> > > > I tried slowing it to 115200, but that didn't help.  I tried disabling
> > > > the DMA hooks from the device tree, and that didn't help.
> > > >
> > > > > By default, omap2plus_defconfig enables both SERIAL_8250_OMAP and
> > > > > SERIAL_OMAP.  I have my console device configured as  ttyS0, and all
> > > > > appears fine.  When I enable Bluetooth, however, I get intermittent
> > > > > errors on an DM3730 / OMAP3630.
> > > > >
> > > > > Using the 8250 driver for Blueotooth I get intermittent frame errors
> > > > > and data loss.
> > > > >
> > > > > Scanning ...
> > > > > [   28.482452] Bluetooth: hci0: Frame reassembly failed (-84)
> > > > > [   36.162170] Bluetooth: hci0: Frame reassembly failed (-84)
> > > > >         F4:4E:FC:C9:2F:57       BluJax
> > > > > # l2ping F4:4E:FC:C9:2F:57
> > > > > Ping: F4:4E:FC:C9:2F:57 from 00:18:30:49:7D:63 (data size 44) ...
> > > > > 44 bytes from F4:4E:FC:C9:2F:57 id 0 time 8.27ms
> > > > > no response from F4:4E:FC:C9:2F:57: id 1
> > > > > ^C2 sent, 1 received, 50% loss
> > > > >
> > > > > (after a fairly long hang, I hit control-c)
> > > > >
> > > > > However, disabling the 8250 driver and using the only SERIAL_OMAP and
> > > > > the console routed to ttyO0, the Bluetooth works well, so I believe it
> > > > > to be a serial driver issue and not a Bluetooth error.
> > > > >
> > > > > # hcitool scan
> > > > > Scanning ...
> > > > >         F4:4E:FC:C9:2F:57       BluJax
> > > > > ^C
> > > > > # l2ping F4:4E:FC:C9:2F:57
> > > > > Ping: F4:4E:FC:C9:2F:57 from 00:18:30:49:7D:63 (data size 44) ...
> > > > > 44 bytes from F4:4E:FC:C9:2F:57 id 0 time 6.90ms
> > > > > ...
> > > > > 44 bytes from F4:4E:FC:C9:2F:57 id 14 time 28.29ms
> > > > > ^C15 sent, 15 received, 0% loss
> > > > > #
> > > > >
> > > > > 0% loss and regular, repeatable communication without any Frame
> > > > > reassembly errors.
> > > > >
> > > >
> > > > I tried disabling SERIAL_OMAP and using only SERIAL_8250_OMAP, but
> > > > that didn't help.  Because the issue goes away when I disable
> > > > SERIAL_8250_OMAP, I am wondering if something is either being
> > > > misconfigured or some IRQ or DMA integration is missing that may be
> > > > present with the older SERIAL_OMAP driver.
> > > >
> > > > > Any suggestions on how to troubleshoot or what might cause the
> > > > > difference between the two drivers?
> > >
> > > Can it be related to this issue [1]? Can you confirm that 5.2 is
> > > working as expected with the 8250 driver?
> > >
> > > [1] https://marc.info/?l=linux-serial&m=156965039008649&w=2
> >
> > I reverted the whole 8250 directory to d99482673f95 ("serial:
> > mctrl_gpio: Check if GPIO property exisits before requesting it") and
> > it is somewhat better, but it's not as good as the stock OMAP serial
> > driver.  I get some frame errors and eventually, I get some timeouts,
> > but it's not as bad.  I'll try to implement the RTS and CTS as gpio
> > pins and change the device tree accordingly.  It might shed some light
> > on the situation.
>
> I tried to manually setup RTS and CTS pins as GPIO, but that didn't
> work, so I changed it back.
>
> It looks like the clocking is correct.  I don't know enough about the
> DMA or the IRQ to know if it's working correctly.
>
> I was wondering if the problem is in the handshaking or not.
> I added " uart-has-rtscts;" to by uart node thinking it might help,
> but it did not.
>
> >
> 8250_omap.c has some checks to see if we can enable autoRTS:
>
> if (termios->c_cflag & CRTSCTS && up->port.flags & UPF_HARD_FLOW &&
>     !up->gpios) {
>      /* Enable AUTOCTS (autoRTS is enabled when RTS is raised) */
>      ...
> }
>
> Based on this, I would expect up->gpios to always be zero if we want
> auto RTS CTS.
>
> I threw some debug code into the serial driver to look at the status
> of the various flags that go into setting up auto RTS/CTS.
>
> [   13.837005] termios->c_cflag & CRTSCTS = 80000000
> [   13.841888] up->port.flags & UPF_HARD_FLOW = 300000
> [   13.846801] up->gpios = ce3f3cc0
> [   17.166595] termios->c_cflag & CRTSCTS = 0
> [   17.170745] up->port.flags & UPF_HARD_FLOW = 300000
> [   17.175781] up->gpios = ce3f3d40
>
> I don't know which port is which, but I only have two active uarts
> (console and Bluetooth)
> The Bluetooth port should use handshaking and the console should not.
>
> From the look of the dump, up->gpios is never 0, so the last
> evaluation would always be false unless I am misinterpreting
> something.
>
> I tried to modify the check to remove !up->gpios check, but that
> didn't help either, but it made it a little better.

I have it working now on the 5.3.y branch, but I had to do 4 things.

1)   Disable the older OMAP serial driver
2)   Apply [1] patch
3)   Disable 8250 DMA support
4)  Remove checks for !up->gpios in the 8250-omap.c code.

With all those changes, I am able to successfully and repeatedly scan
and l2ping Bluetooth devices.  Without any one of those, I get frame
errors or lost packets.

I wonder if we can dump the legacy omap serial driver once  the 8250
driver is working better.  That would eliminate at least one of the 4.
I am not sure what's up with the DMA, and I've already stated some
thoughts on item 4 in a previous e-mail.

I think the [1] patch is probably going to help fix some other issues
and should probably be backported, but by itself it isn't enough to
fix the Bluetooth problem on the OMAP3630.

[1] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/tty/serial/8250/8250_omap.c?h=v5.4-rc1&id=627a545c6bb0c7de09208e6546f5111290477261

adam
>
>
> > adam
> > >
> > > Cheers,
> > > Yegor