Re: PCIe MSI address is not written at pci_enable_msi_range call

[Marc Zyngier]

On 13/07/16 10:10, Bharat Kumar Gogada wrote:
>> Subject: Re: PCIe MSI address is not written at pci_enable_msi_range call
>>
>> On 13/07/16 09:33, Bharat Kumar Gogada wrote:
>>>> Subject: Re: PCIe MSI address is not written at pci_enable_msi_range call
>>>>
>>>> On 13/07/16 07:22, Bharat Kumar Gogada wrote:
>>>>>> Subject: Re: PCIe MSI address is not written at pci_enable_msi_range
>>>>>> call
>>>>>>
>>>>>> On 11/07/16 10:33, Bharat Kumar Gogada wrote:
>>>>>>> Hi Marc,
>>>>>>>
>>>>>>> Thanks for the reply.
>>>>>>>
>>>>>>> From PCIe Spec:
>>>>>>> MSI Enable Bit:
>>>>>>> If 1 and the MSI-X Enable bit in the MSI-X Message Control register
>>>>>>> (see Section 6.8.2.3) is 0, the function is permitted to use MSI to
>>>>>>> request service and is prohibited from using its INTx# pin.
>>>>>>>
>>>>>>> From Endpoint perspective, MSI Enable = 1 indicates MSI can be used
>>>>>> which means MSI address and data fields are available/programmed.
>>>>>>>
>>>>>>> In our SoC whenever MSI Enable goes from 0 --> 1 the hardware
>>>>>>> latches
>>>>>> onto MSI address and MSI data values.
>>>>>>>
>>>>>>> With current MSI implementation in kernel, our SoC is latching on to
>>>>>>> incorrect address and data values, as address/data are updated much
>>>>>>> later
>>>>>> than MSI Enable bit.
>>>>>>
>>>>>> As a side question, how does setting the affinity work on this
>>>>>> end-point if this involves changing the address programmed in the MSI
>>>> registers?
>>>>>> Do you expect the enabled bit to be toggled to around the write?
>>>>>>
>>>>>
>>>>> Yes,
>>>>
>>>> Well, that's pretty annoying, as this will not work either. But maybe your
>> MSI
>>>> controller has a single doorbell? You haven't mentioned which HW that
>> is...
>>>>
>>> The MSI address/data is located in config space, in our SoC for the logic
>> behind PCIe
>>> to become aware of new address/data  MSI enable transition is used (0 to
>> 1).
>>> The logic cannot keep polling these registers in configuration space as it
>> would consume power.
>>>
>>> So the logic uses the transition in MSI enable to latch on to address/data.
>>
>> I understand the "why". I'm just wondering if your SoC needs to have
>> the MSI address changed when changing the affinity of the MSI? What MSI
>> controller are you using? Is it in mainline?
>>
> Can you please give more information on MSI affinity ?
> For cpu affinity for interrupts we would use MSI-X.
> 
> We are using GIC 400 v2.

None of that is relevant. GIC400 doesn't have the faintest notion of
what an MSI is, and MSI-X vs MSI is an end-point property.

Please answer these questions: does your MSI controller have a unique
doorbell, or multiple doorbells? Does it use wired interrupts (SPIs)
connected to the GIC? Is the support code for this MSI controller in
mainline or not?

I'm trying to work out what I can do to help you.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...