Re: [PATCH] ARM: Mark the FDT_FIXED sections as shareable

[Ard Biesheuvel]

On Mon, 6 Jun 2022 at 17:52, Ard Biesheuvel <ardb@xxxxxxxxxx> wrote:
>
> Hello Zhen Lei,
>
> On Mon, 6 Jun 2022 at 14:49, Zhen Lei <thunder.leizhen@xxxxxxxxxx> wrote:
> >
> > commit 7a1be318f579 ("ARM: 9012/1: move device tree mapping out of linear
> > region") use FDT_FIXED_BASE to map the whole FDT_FIXED_SIZE memory area
> > which contains fdt. But it only reserves the exact physical memory that
> > fdt occupied. Unfortunately, this mapping is non-shareable. An illegal or
> > speculative read access can bring the RAM content from non-fdt zone into
> > cache, PIPT makes it to be hit by subsequently read access through
> > shareable mapping(such as linear mapping), and the cache consistency
> > between cores is lost due to non-shareable property.
> >
> > |<---------FDT_FIXED_SIZE------>|
> > |                               |
> >  -------------------------------
> > | <non-fdt> | <fdt> | <non-fdt> |
> >  -------------------------------
> >
> > 1. CoreA read <non-fdt> through MT_ROM mapping, the old data is loaded
> >    into the cache.
> > 2. CoreB write <non-fdt> to update data through linear mapping. CoreA
> >    received the notification to invalid the corresponding cachelines, but
> >    the property non-shareable makes it to be ignored.
> > 3. CoreA read <non-fdt> through linear mapping, cache hit, the old data
> >    is read.
> >
>
> Thanks for the excellent write-up, and for what must have been a lot
> of work to narrow down and diagnose!
>
> > To eliminate this risk, mark the MT_ROM sections as shareable.
> >
> > The other user of MT_ROM is XIP_KERNEL. XIP allows the kernel to run from
> > flash to save RAM space. Not sure if anyone is still using XIP in order to
> > save a little memory and not care about performance degradation. Add a new
> > memory type MT_ROM_XIP to be compatible with it.
> >
> > BTW: Another solution is to memblock_reserve() all the sections that fdt
> > spans, but this will waste 2-4MiB memory.
> >
>
> I agree that we should not add shareable attributes to the memory type
> used by XIP kernels for code regions: NOR flash is not usually
> integrated in a way that allows it to participate in the coherency
> protocol, so that will likely break things.
>
> I think, though, that it would be better to leave MT_ROM alone, and
> introduce a new type MT_MEMORY_RO instead, which is wired up in the
> right way (see below), so that we get NX attributes, and can use it to
> create non-section mappings as well.
>
> Then, as a followup which does not need to go into -stable, we can
> reduce the size of the mapping: there is really no need for the
> permanent mapping to be section granular - this is only for the early
> asm code that is not able to create 2 levels of page tables.
>

Actually, on second thought, I think reducing the size of the FDT
mapping is also needed for correctness, as the non-fdt regions could
potentially be covered by a no-map memory reservation, or get mapped
non-cacheable for things like non-coherent DMA.