Re: Memory synchronization vs. interrupt handlers

[Catalin Marinas]

On 26 August 2013 16:49, Alan Stern <stern@xxxxxxxxxxxxxxxxxxx> wrote:
> Here's a question that doesn't seem to be answered in
> Documentation/memory-barriers.txt.  Are memory accesses within an
> interrupt handler synchronized with respect to interrupts?
>
> In more detail, suppose we have an interrupt handler that uses a memory
> variable A.  The device attached to the IRQ line sends two interrupt
> requests, and we get:
>
>         CPU 0                           CPU 1
>         -----                           -----
>         Receive IRQ
>         Call the interrupt handler
>                 Write A
>         Finish IRQ processing
>
>                                         Receive IRQ
>                                         Call the interrupt handler
>                                                 Read A
>                                         Finish IRQ processing
>
> Is CPU 0's write to A guaranteed to be visible on CPU 1?  Given that
> interrupts on an IRQ line are serialized, and that IRQ processing must
> involve some amount of memory barriers, I would expect the answer to be
> Yes.

On arm (or arm64), this is not guaranteed. Finishing the IRQ
processing usually involves a device write but there is no ordering
required with other write accesses. It could easily be fixed in the
IRQ controller code (drivers/irqchip/irq-gic.c for newer ARM
processors). We have a barrier for SMP cross-calls for the same
ordering reason, so I guess we need one for EOI as well.

In practice I would think it's nearly impossible to hit this issue,
given the time to save the state when taking the interrupt plus some
spinlocks, the write from CPU0 would become visible.

Also, if the data is accessed by the same driver with the same IRQ,
most likely the interrupt goes to the same CPU (unless you had some
rebalancing, but this being rarer it makes it less likely). If the
data is being accessed between two separate IRQ handlers, a spinlock
must be used anyway.

-- 
Catalin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/