Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a separate file

[Palmer Dabbelt]

On Mon, 16 Sep 2019 11:18:00 PDT (-0700), will@xxxxxxxxxx wrote:
> On Sun, Sep 15, 2019 at 05:03:38AM +0000, Anup Patel wrote:
> > > -----Original Message-----
> > > From: linux-kernel-owner@xxxxxxxxxxxxxxx <linux-kernel-
> > > owner@xxxxxxxxxxxxxxx> On Behalf Of Palmer Dabbelt
> > > Sent: Saturday, September 14, 2019 7:31 PM
> > > To: will@xxxxxxxxxx
> > > Cc: guoren@xxxxxxxxxx; Will Deacon <will.deacon@xxxxxxx>;
> > > julien.thierry@xxxxxxx; aou@xxxxxxxxxxxxxxxxx; james.morse@xxxxxxx;
> > > Arnd Bergmann <arnd@xxxxxxxx>; suzuki.poulose@xxxxxxx;
> > > marc.zyngier@xxxxxxx; catalin.marinas@xxxxxxx; Anup Patel
> > > <Anup.Patel@xxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx;
> > > rppt@xxxxxxxxxxxxx; Christoph Hellwig <hch@xxxxxxxxxxxxx>; Atish Patra
> > > <Atish.Patra@xxxxxxx>; julien.grall@xxxxxxx; gary@xxxxxxxxxxx; Paul
> > > Walmsley <paul.walmsley@xxxxxxxxxx>; christoffer.dall@xxxxxxx; linux-
> > > riscv@xxxxxxxxxxxxxxxxxxx; kvmarm@xxxxxxxxxxxxxxxxxxxxx; linux-arm-
> > > kernel@xxxxxxxxxxxxxxxxxxx; iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx
> > > Subject: Re: [PATCH RFC 11/14] arm64: Move the ASID allocator code in a
> > > separate file
> > >
> > > On Thu, 12 Sep 2019 07:02:56 PDT (-0700), will@xxxxxxxxxx wrote:
> > > > On Sun, Sep 08, 2019 at 07:52:55AM +0800, Guo Ren wrote:
> > > >> On Mon, Jun 24, 2019 at 6:40 PM Will Deacon <will@xxxxxxxxxx> wrote:
> > > >> > > I'll keep my system use the same ASID for SMP + IOMMU :P
> > > >> >
> > > >> > You will want a separate allocator for that:
> > > >> >
> > > >> > https://lkml.kernel.org/r/20190610184714.6786-2-jean-philippe.bruck
> > > >> > er@xxxxxxx
> > > >>
> > > >> Yes, it is hard to maintain ASID between IOMMU and CPUMMU or
> > > >> different system, because it's difficult to synchronize the IO_ASID
> > > >> when the CPU ASID is rollover.
> > > >> But we could still use hardware broadcast TLB invalidation
> > > >> instruction to uniformly manage the ASID and IO_ASID, or OTHER_ASID in
> > > our IOMMU.
> > > >
> > > > That's probably a bad idea, because you'll likely stall execution on
> > > > the CPU until the IOTLB has completed invalidation. In the case of
> > > > ATS, I think an endpoint ATC is permitted to take over a minute to
> > > > respond. In reality, I suspect the worst you'll ever see would be in
> > > > the msec range, but that's still an unacceptable period of time to hold a
> > > CPU.
> > > >
> > > >> Welcome to join our disscusion:
> > > >> "Introduce an implementation of IOMMU in linux-riscv"
> > > >> 9 Sep 2019, 10:45 Jade-room-I&II (Corinthia Hotel Lisbon) RISC-V MC
> > > >
> > > > I attended this session, but it unfortunately raised many more
> > > > questions than it answered.
> > >
> > > Ya, we're a long way from figuring this out.
> >
> > For everyone's reference, here is our first attempt at RISC-V ASID allocator:
> > http://archive.lwn.net:8080/linux-kernel/20190329045111.14040-1-anup.patel@xxxxxxx/T/#u
>
> With a reply stating that the patch "absolutely does not work" ;)
>
> What exactly do you want people to do with that? It's an awful lot of effort
> to review this sort of stuff and given that Guo Ren is talking about sharing
> page tables between the CPU and an accelerator, maybe you're better off
> stabilising Linux for the platforms that you can actually test rather than
> getting so far ahead of yourselves that you end up with a bunch of wasted
> work on patches that probably won't get merged any time soon.
>
> Seriously, they say "walk before you can run", but this is more "crawl
> before you can fly". What's the rush?

I agree, and I think I've been pretty clear here: we're not merging this ASID 
stuff until we have a platform we can test on, particularly as the platforms we 
have now already need some wacky hacks around TLB flushing that we haven't 
gotten to the bottom of.

> Will