Re: MSI, fun for the whole family

[Eric W. Biederman]

Jeff Garzik <jeff@xxxxxxxxxx> writes:

> Roland Dreier wrote:
>> I think you've fundamentally misunderstood what the PCI spec for MSI
>> multi message means.  It is true that if the whole system agrees, then
>
> You seem to be ignoring a key usage...
>
>
>> an MSI-capable device that supports multiple messages might be allocated
>> a range of vectors (MSI is kind of stupid because it only allows
>> multiple messages to be generated by varying the low order bits -- MSI-X
>> fixes this limitation).  However, the way that these different messages
>> are handled is that they are all independent interrupt vectors.
>>
>> Now, it is true that the kernel could do something crazy and collapse
>> all these interrupt vectors into a single "IRQ" and then tell the
>> interrupt handler which vector it was by passing some "metadata" in, but
>> why not just give each MSI message it's own IRQ?
>
> The answer is: the driver might prefer to see the message as it arrived, rather
> than dividing it up into independent vectors.  The message itself is a unit of
> data consistency, and there is value in letting the driver see the bounds of
> that unit.

On x86 the driver very much sees the message as it arrived at the cpu.
If you want the message as it was sent we would need to deduce it from
which cpu we are on and which interrupt descriptors function was called.

As for the bounds of data consistency currently no information is lost.

> As it stands now, we only a spray of $N function calls for each message, with no
> notion of "we started processing this set of messages" and "we ended
> processing[...]"

Hopefully my other messages explains it better but there is no
spray of function calls.  Nor can there be.  There is not a bitmask in
your MSI message there is instead a base 2 number encoding which port
had traffic.  You don't have 32 bits but 5.

> Don't assume that the way Linux supports this stuff today is the best, or the
> only way to do things.  It's not "collapsing all these interrupt vectors" --
> remember that an expansion occurred, and /avoiding expansion/ into multiple
> vectors for multiple messages may be an optimal path for a specific driver
> application.

If we were receiving a bitmask I would find this more persuasive.

To implement what you suggest currently feels like a lot of work digging
through each architectures implementation of MSI and seeing if it possible
and maintainable on each architecture, and refactoring a lot of the interrupt
handling infrastructure to get there.  All to support hardware vendors
that were too lazy to either implement MSI-X or implement a DMAing a
status value to a known spot before sending the MSI.

I don't see it being worth it at this time.

Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/