Re: [PATCH v2 1/5] arm/xen: define xen_remap as ioremap_cached

From: Stefano Stabellini
Date: Tue Jun 04 2013 - 07:35:32 EST


On Tue, 4 Jun 2013, Catalin Marinas wrote:
> On Tue, Jun 04, 2013 at 10:20:50AM +0100, Ian Campbell wrote:
> > On Mon, 2013-06-03 at 16:33 +0100, Stefano Stabellini wrote:
> > > Define xen_remap as ioremap_cache (MT_MEMORY and MT_DEVICE_CACHED end up
> > > having the same AttrIndx encoding).
> >
> > The entries in static struct mem_type mem_types[] look entirely
> > different to me. What am I missing?
> > [MT_DEVICE_CACHED] = { /* ioremap_cached */
> > .prot_pte = PROT_PTE_DEVICE | L_PTE_MT_DEV_CACHED,
> > .prot_l1 = PMD_TYPE_TABLE,
> > .prot_sect = PROT_SECT_DEVICE | PMD_SECT_WB,
> > .domain = DOMAIN_IO,
> > },
> > [MT_MEMORY] = {
> > .prot_pte = L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_DIRTY,
> > .prot_l1 = PMD_TYPE_TABLE,
> > .prot_sect = PMD_TYPE_SECT | PMD_SECT_AP_WRITE,
> > .domain = DOMAIN_KERNEL,
> > },
> >
> > I can see in pgtable-3level.h how L_PTE_MT_DEV_CACHED and
> > L_PTE_MT_WRITEBACK are the same but not where the MT_WRITEBACK comes
> > from for MT_MEMORY. Things are less clear in pgtable-2level.h, where one
> > is 0x3 and the other is 0xb. I can see that the entries are the same in
> > armv6_mt_table but how that would apply to a v7 processor?
>
> PROT_PTE_DEVICE and PROT_SECT_DEVICE above don't contain any memory type
> information, just attributes/permission - present, young, dirty and XN:
>
> #define PROT_PTE_DEVICE L_PTE_PRESENT|L_PTE_YOUNG|L_PTE_DIRTY|L_PTE_XN
> #define PROT_SECT_DEVICE PMD_TYPE_SECT|PMD_SECT_AP_WRITE
>
> The memory type is given by the L_PTE_MT_DEV_CACHED and PMD_SECT_WB
> macros. Let's take prot_sect first as it's simpler. For MT_DEVICE_CACHED
> we have:
>
> .prot_sect = PMD_TYPE_SECT | PMD_SECT_AP_WRITE | PMD_SECT_WB
>
> For MT_MEMORY we have:
>
> .prot_sect = PMD_TYPE_SECT | PMD_SECT_AP_WRITE
>
> The cache policy is added later to MT_MEMORY which is either WB or WBWA
> (based on SMP, no particular reason as these are just processor hints;
> for some historical reasons we enabled WBWA for ARM11MPCore but we could
> leave it on all the time).
>
> Similarly for prot_pte, present, young, dirty are the same.
>
> Regarding the type, on ARMv7 (with or without LPAE) we use TEX remapping
> and L_PTE_MT_DEVICE has the same index (3-bit TEX[0], C, B for NMRR/PRRR
> or TEX[2:0] for MAIR0/MAIR1 registers) as Normal Cacheable Writeback
> memory (there is no such thing as Device memory with cacheability
> attributes, only Normal Cacheable memory).
>
> We have XN in addition for MT_DEVICE_CACHED to prevent speculative
> instruction fetches. However, you still get speculative D-cache line
> fills since the memory is Normal Cacheable.
>
> > Anyhow, even if I'm prepared to believe that MT_MEMORY and
> > MT_DEVICE_CACHED end up being the same thing (which TBH I am) it seems
> > that the level of abstraction involved makes us vulnerable to future
> > changes subtly breaking things for us. What about:
> >
> > /* Device shared memory */
> > #define ioremap_shm(cookie,size) __arm_ioremap((cookie), (size), MT_MEMORY)
>
> For my understanding, what is Xen doing with such mapping? I would
> rather make ioremap_cached() use MT_MEMORY on ARMv6/v7 (or make it
> closer to that, I'm not sure about the implications on ARMv5 and earlier
> but for newer architectures I don't see the point in pretending to have
> Cacheable Device memory). That's however for Russell to comment.

Xen guests share these pages with one another and place a lockless ring
buffer on it for bidirectional communication. So the page that is being
ioremapped actually belongs to another guest and it's RAM.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/