Re: lockdep: incorrect deadlock warning with two GPIO expanders

[Peter Zijlstra]

On Mon, Sep 12, 2016 at 01:51:55PM +0200, Bartosz Golaszewski wrote:
> I'm trying to figure out a way of getting rid of an incorrect lockdep
> deadlock warning, but the issue is not trivial.
> 
> In our hardware an I2C multiplexer is controlled by a GPIO provided by
> an expander. There's a second expander using the same device driver
> (pca953x) on one of the I2C bus segments. The diagram below presents
> the setup:
> 
>                                                - - - - -
>  -------             ---------  Bus segment 1 |         |
> |       |           |         |---------------  Devices
> |       | SCL/SDA   |         |               |         |
> | Linux |-----------| I2C MUX |                - - - - -
> |       |    |      |         | Bus segment 2
> |       |    |      |         |-------------------
>  -------     |       ---------                    |
>              |           |                    - - - - -
>         ------------     | MUX GPIO          |         |
>        |            |    |                     Devices
>        |    GPIO    |    |                   |         |
>        | Expander 1 |----                     - - - - -
>        |            |                             |
>         ------------                              | SCL/SDA
>                                                   |
>                                              ------------
>                                             |            |
>                                             |    GPIO    |
>                                             | Expander 2 |
>                                             |            |
>                                              ------------

> Using mutex_lock_nested(&chip->i2c_lock, SINGLE_DEPTH_NESTING) in
> pca953x_gpio_get_value() and pca953x_gpio_direction_input/output()
> helps for reading the values or setting the direction, but doesn't do
> anything if used in pca953x_gpio_set_value() since we still end up
> taking the lock of the same subclass again.
> 
> It would require some nasty hacks to figure out that a GPIO is being
> used by an I2C mux if we wanted to explicitly provide a different
> sublass in this case, but that would not fix the culprit, since the
> same problem would occur in other gpio drivers under similar
> circumstances.
> 
> It seems the problem is with the way lockdep works, but I lack the
> knowledge to propose any solution.
> 
> Any help & ideas are appreciated.

So I'm entirely clueless on how the device model works let alone i2c
and/or gpio. So I'm going to need some help as well. What's an SCL/SDA
for instance?

So the 'problem' is that pca953x_probe()'s mutex_init() will collapse
all mutexes it initializes into a single class. It assumes that the
locking rules for all instances will be the same.

This happens to not be true in this case.

The tricky part, and here I have absolutely no clue what so ever, is
being able to tell at pca953x_probe() time that this is so.

Once we can tell, at probe time, there are two different annotations we
could use, depending on need.

I suppose that theoretically you can keep nesting like that ad
infinitum, but I also expect that its uncommon enough, and maybe not
practical, to really nest like this -- seeing this is the first instance
of such issues.

In any case, can you tell at probe time? And how deep a nesting should
we worry about?

Seeing how this lock is specific to the driver, and there is no generic
infrastructure, I don't see how we could solve it other than on a
per-driver basis.