Re: [PATCH v1 1/1] ARM: Select DMA_DIRECT_REMAP to fix restricted DMA

[Jim Quinlan]

On Thu, Sep 28, 2023 at 11:47 AM Robin Murphy <robin.murphy@xxxxxxx> wrote:
>
> On 28/09/2023 1:07 pm, Jim Quinlan wrote:
> > On Wed, Sep 27, 2023 at 7:10 PM Linus Walleij <linus.walleij@xxxxxxxxxx> wrote:
> >>
> >> Hi Jim,
> >>
> >> thanks for your patch!
> >>
> >> On Tue, Sep 26, 2023 at 7:52 PM Jim Quinlan <james.quinlan@xxxxxxxxxxxx> wrote:
> >>
> >>> Without this commit, the use of dma_alloc_coherent() while
> >>> using CONFIG_DMA_RESTRICTED_POOL=y breaks devices from working.
> >>> For example, the common Wifi 7260 chip (iwlwifi) works fine
> >>> on arm64 with restricted memory but not on arm, unless this
> >>> commit is applied.
> >>>
> >>> Signed-off-by: Jim Quinlan <james.quinlan@xxxxxxxxxxxx>
> >>
> >> (...)
> >>> +       select DMA_DIRECT_REMAP
> >>
> >> Christoph invented that symbol so he can certainly
> >> explain what is missing to use this on ARM.
> >>
> >> This looks weird to me, because:
> >>> git grep atomic_pool_init
> >> arch/arm/mm/dma-mapping.c:static int __init atomic_pool_init(void)
> >> kernel/dma/pool.c:static int __init dma_atomic_pool_init(void)
> >>
> >> Now you have two atomic DMA pools in the kernel,
> >> and a lot more than that is duplicated. I'm amazed that it
> >> compiles at all.
> >>
> >> Clearly if you want to do this, surely the ARM-specific
> >> arch/arm/mm/dma-mapping.c and arch/arm/mm/dma-mapping-nommu.c
> >> needs to be removed at the same time?
> >>
> >> However I don't think it's that simple, because Christoph would surely
> >> had done this a long time ago if it was that simple.
> >
> > Hello Linus,
> >
> > Yes, this is the reason I used "RFC" as the fix looked too easy to be viable :-)
> > I debugged it enough to see that the host driver's
> > writes to the dma_alloc_coherent() region  were not appearing in
> > memory, and that
> > led me to DMA_DIRECT_REMAP.
>
> Oh, another thing - the restricted-dma-pool is really only for streaming
> DMA - IIRC there can be cases where the emergency fallback of trying to
> allocate out of the bounce buffer won't work properly. Are you also
> using an additional shared-dma-pool carveout to satisfy the coherent
> allocations, per the DT binding?

Hello Robin,
Sorry for the delay.  We use "restricted DMA" as a poor person's IOMMU; we can
restrict the DMA memory of a device to a narrow region, and our memory
bus HW has
"checkers' to enforce said region for a specific memory client, e.g. PCIe.

We can confirm the assignment of restricted DMA in the bootlog when the device
is probed:

        iwlwifi 0001:01:00.0: assigned reserved memory node pcieSR1@4a000000
        iwlwifi 0001:01:00.0: enabling device (0000 -> 0002)

As far as your other question, why don't I  just post our relevant DT [1].

Regards,
Jim Quinlan
Broardcom STB/CM

[1]
memory {
        device_type = "memory";
        reg = <0x0 0x40000000 0x1 0x0>;
};

reserved-memory {
        #address-cells = <0x2>;
        #size-cells = <0x2>;
        ranges;
        /* ... */

        pcieSR1@4a000000 {
                linux,phandle = <0x2a>;
                phandle = <0x2a>;
                compatible = "restricted-dma-pool";
                reserved-names = "pcieSR1";
                reg = <0x0 0x4a000000 0x0 0x2400000>;
        };
};
pcie@8b20000 {
        /* ... */
        pci@0,0 {
                /* ... */
                pci-ep@0,0 {
                        memory-region = <0x2a>;
                        reg = <0x10000 0x0 0x0 0x0 0x0>;
                };
        };
};




>
> Thanks,
> Robin.

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature