[RFC PATCH 0/2] Bluetooth: detect broken extended scan instead of guessing by chip id

[Kamil Serwus]

Hi,

commit 5ead2063611ae5 ("Bluetooth: btrtl: fix RTL8761B/BU broken LE
extended scan"), which landed in v7.2, regresses a different RTL8761BU
dongle than the one it was written for. This series reports that and
proposes a way out that should not need a new patch per dongle.

The regression
--------------

Hardware here is USB 0bda:8771, firmware 0xdfc6d922, ROM lmp_subver
0x8761 - an RTL8761BU, so CHIP_ID_8761B, so it gets the new quirk, which
makes use_ext_scan() false and moves scanning from LE Set Extended Scan
Enable (0x2042) to the legacy LE Set Scan Enable (0x200c).

This dongle is the opposite of the one the quirk was written for:
extended scan works flawlessly, but the firmware stops answering the
legacy scan-disable command about ten seconds into any scan:

  Bluetooth: hci0: command 0x200c tx timeout
  Bluetooth: hci0: Opcode 0x200c failed: -110
  Bluetooth: hci0: Unable to disable scanning: -110
  Bluetooth: hci0: disable scanning failed: -110
  Bluetooth: hci0: Resetting usb device.
  Bluetooth: hci0: start background scanning failed: -110
  usb 3-1: reset full-speed USB device number 2 using xhci_hcd

btusb resets the device, it re-enumerates (hci0 -> hci1 -> hci2 -> ...)
and every connected BLE device is dropped. With a Bluetooth keyboard and
trackball that leaves the machine with no input at all.

v7.1.7 drives the same dongle, with byte-identical linux-firmware and
identical CONFIG_BT_HCIBTUSB_*, for eleven hours and nine BLE HID
connects without a single scan error.

Why not just narrow the quirk
-----------------------------

Nothing available at setup time separates the two dongles:

  - same project_id (CHIP_ID_8761B)
  - same ic_id_table entry, IC_INFO(RTL_ROM_LMP_8761A, 0xb, 0xa, HCI_USB)
  - same firmware file, rtl_bt/rtl8761bu_fw.bin
  - both advertise the extended scan commands in commands[37]

They differ only by USB id, which btrtl_set_quirks() cannot see.
driver_info cannot carry it either: 0bda:a728 has no entry of its own, so
it falls to the generic Realtek entry in btusb's quirks_table -
USB_VENDOR_AND_INTERFACE_INFO(0x0bda, 0xe0, 0x01, 0x01) - which precedes
the per-device entries. usb_match_id() takes the first match and
0bda:8771 also presents e0/01/01, so the specific entry for it at
quirks_table:840 never applies.

What this series does instead
-----------------------------

The quirk is documented as being for controllers which "erroneously
claim to support extended scanning" - and one that claims support in
commands[37] and then answers Command Disallowed is saying so itself.
Patch 1 latches the quirk right where that command is issued; every
later scan uses the legacy commands. Patch 2 then drops the static
per-chip-id guess, which 0bda:a728 no longer needs. In this order the
series is bisectable: patch 1 alone changes nothing for anyone whose
quirk is already set at setup.

An affected controller pays one rejected scan round per power cycle
instead of a permanent stream of failures, and dongles nobody has owned
yet are handled without waiting for a patch.

The latch is sticky, as quirk_flags is never cleared, so it matters that
false positives are unlikely: hci_passive_scan_sync() always disables
scanning before starting it, so the controller is never asked to enable
a scan that is already running - the obvious legitimate source of
Command Disallowed.

Testing
-------

Built and running on v7.2 on the 0bda:8771 dongle. Extended scan is used,
the quirk is never set, no 0x200c and no controller resets, and a
keyboard plus a trackball stay connected across repeated LE scans - the
exact sequence that wedged the adapter within ten seconds before.

What I cannot test is the other half: I do not have a 0bda:a728, so the
err == -EBUSY path has never executed here. Alexej, does your dongle
still work with this series, and do you see the "extended scan rejected,
using legacy scan" warning once? Also, 0x2042 carries both the enable and
the disable, so I cannot tell from your report which call site returned
-EBUSY - knowing that would tell us whether a single rejected round is
really all it costs you.

Sending as RFC mainly because patch 1 sets the quirk at runtime, while
the existing comment says it is set before hci_register_dev or in
hdev->setup. I updated that comment, but if you would rather keep the
quirk strictly setup-time I am happy to respin as a USB-id match in
btusb.c instead - it just needs the quirks_table ordering sorted out
first.

#regzbot introduced: 5ead2063611ae56809b1b113ac44cef9547c81d7

Kamil Serwus (2):
  Bluetooth: hci_sync: latch broken ext scan on Command Disallowed
  Bluetooth: btrtl: drop the blanket RTL8761B extended scan quirk

 drivers/bluetooth/btrtl.c   | 13 -------------
 include/net/bluetooth/hci.h |  4 ++--
 net/bluetooth/hci_sync.c    | 15 +++++++++++++--
 3 files changed, 15 insertions(+), 17 deletions(-)


base-commit: 8d3ae59288f1e7d58d76558a6ee96d533bc5019f
-- 
2.55.0