[BUG] ar5523: possible deadlocks involving locking and waiting operations

[Jia-Ju Bai]

Hello,

My static analysis tool reports two possible deadlocks in the ar5523 
driver in Linux 5.16:

#BUG 1
ar5523_hwconfig()
  mutex_lock(&ar->mutex); --> Line 1135 (Lock A)
  ar5523_flush_tx()
    wait_event_timeout(ar->tx_flush_waitq, ...) --> Line 926 (Wait X)

ar5523_tx_work()
  mutex_lock(&ar->mutex); --> Line 888 (Lock A)
  ar5523_tx_work_locked()
    ar5523_data_tx_pkt_put()
      wake_up(&ar->tx_flush_waitq); --> Line 727 (Wake X)

#BUG 2
ar5523_configure_filter()
  mutex_lock(&ar->mutex); --> Line 1331 (Lock A)
  ar5523_flush_tx()
    wait_event_timeout(ar->tx_flush_waitq, ...) --> Line 926 (Wait X)

ar5523_tx_work()
  mutex_lock(&ar->mutex); --> Line 888 (Lock A)
  ar5523_tx_work_locked()
    ar5523_data_tx_pkt_put()
      wake_up(&ar->tx_flush_waitq); --> Line 727 (Wake X)

When ar5523_hwconfig()/ar5523_configure_filter() is executed, "Wait X" 
is performed by holding "Lock A". If ar5523_tx_work() is executed at 
this time, "Wake X" cannot be performed to wake up "Wait X", because 
"Lock A" has been already held, causing possible deadlock.
I find that "Wait X" is performed with a timeout, to relieve the 
possible deadlocks; but I think this timeout can cause inefficient 
execution.

I am not quite sure whether these possible problems are real.
Any feedback would be appreciated, thanks :)


Best wishes,
Jia-Ju Bai