Re: [PATCH v12 00/25] Linux RISC-V AIA Support

From: Anup Patel
Date: Sat Jan 27 2024 - 11:25:20 EST

Hi Thomas,

On Sat, Jan 27, 2024 at 9:48 PM Anup Patel <apatel@xxxxxxxxxxxxxxxx> wrote:
>
> The RISC-V AIA specification is ratified as-per the RISC-V international
> process. The latest ratified AIA specifcation can be found at:
> https://github.com/riscv/riscv-aia/releases/download/1.0/riscv-interrupts-1.0.pdf
>
> At a high-level, the AIA specification adds three things:
> 1) AIA CSRs
> - Improved local interrupt support
> 2) Incoming Message Signaled Interrupt Controller (IMSIC)
> - Per-HART MSI controller
> - Support MSI virtualization
> - Support IPI along with virtualization
> 3) Advanced Platform-Level Interrupt Controller (APLIC)
> - Wired interrupt controller
> - In MSI-mode, converts wired interrupt into MSIs (i.e. MSI generator)
> - In Direct-mode, injects external interrupts directly into HARTs
>
> For an overview of the AIA specification, refer the AIA virtualization
> talk at KVM Forum 2022:
> https://static.sched.com/hosted_files/kvmforum2022/a1/AIA_Virtualization_in_KVM_RISCV_final.pdf
> https://www.youtube.com/watch?v=r071dL8Z0yo
>
> To test this series, use QEMU v7.2 (or higher) and OpenSBI v1.2 (or higher).
>
> These patches can also be found in the riscv_aia_v12 branch at:
> https://github.com/avpatel/linux.git
>
> Changes since v11:
> - Rebased on Linux-6.8-rc1
> - Included kernel/irq related patches from "genirq, irqchip: Convert ARM
> MSI handling to per device MSI domains" series by Thomas.
> (PATCH7, PATCH8, PATCH9, PATCH14, PATCH16, PATCH17, PATCH18, PATCH19,
> PATCH20, PATCH21, PATCH22, PATCH23, and PATCH32 of
> https://lore.kernel.org/linux-arm-kernel/20221121135653.208611233@xxxxxxxxxxxxx/)
> - Updated APLIC MSI-mode driver to use the new WIRED_TO_MSI mechanism.
> - Updated IMSIC driver to support per-device MSI domains for PCI and
> platform devices.
>
> Changes since v10:
> - Rebased on Linux-6.6-rc7
> - Dropped PATCH3 of v10 series since this has been merged by MarcZ
> for Linux-6.6-rc7
> - Changed the IMSIC ID management strategy from 1-n approach to
> x86-style 1-1 approach
>
> Changes since v9:
> - Rebased on Linux-6.6-rc4
> - Use builtin_platform_driver() in PATCH5, PATCH9, and PATCH12
>
> Changes since v8:
> - Rebased on Linux-6.6-rc3
> - Dropped PATCH2 of v8 series since we won't be requiring
> riscv_get_intc_hartid() based on Marc Z's comments on ACPI AIA support.
> - Addressed Saravana's comments in PATCH3 of v8 series
> - Update PATCH9 and PATCH13 of v8 series based on comments from Sunil
>
> Changes since v7:
> - Rebased on Linux-6.6-rc1
> - Addressed comments on PATCH1 of v7 series and split it into two PATCHes
> - Use DEFINE_SIMPLE_PROP() in PATCH2 of v7 series
>
> Changes since v6:
> - Rebased on Linux-6.5-rc4
> - Updated PATCH2 to use IS_ENABLED(CONFIG_SPARC) instead of
> !IS_ENABLED(CONFIG_OF_IRQ)
> - Added new PATCH4 to fix syscore registration in PLIC driver
> - Update PATCH5 to convert PLIC driver into full-blown platform driver
> with a re-written probe function.
>
> Changes since v5:
> - Rebased on Linux-6.5-rc2
> - Updated the overall series to ensure that only IPI, timer, and
> INTC drivers are probed very early whereas rest of the interrupt
> controllers (such as PLIC, APLIC, and IMISC) are probed as
> regular platform drivers.
> - Renamed riscv_fw_parent_hartid() to riscv_get_intc_hartid()
> - New PATCH1 to add fw_devlink support for msi-parent DT property
> - New PATCH2 to ensure all INTC suppliers are initialized which in-turn
> fixes the probing issue for PLIC, APLIC and IMSIC as platform driver
> - New PATCH3 to use platform driver probing for PLIC
> - Re-structured the IMSIC driver into two separate drivers: early and
> platform. The IMSIC early driver (PATCH7) only initialized IMSIC state
> and provides IPIs whereas the IMSIC platform driver (PATCH8) is probed
> provides MSI domain for platform devices.
> - Re-structure the APLIC platform driver into three separe sources: main,
> direct mode, and MSI mode.
>
> Changes since v4:
> - Rebased on Linux-6.5-rc1
> - Added "Dependencies" in the APLIC bindings (PATCH6 in v4)
> - Dropped the PATCH6 which was changing the IOMMU DMA domain APIs
> - Dropped use of IOMMU DMA APIs in the IMSIC driver (PATCH4)
>
> Changes since v3:
> - Rebased on Linux-6.4-rc6
> - Droped PATCH2 of v3 series instead we now set FWNODE_FLAG_BEST_EFFORT via
> IRQCHIP_DECLARE()
> - Extend riscv_fw_parent_hartid() to support both DT and ACPI in PATCH1
> - Extend iommu_dma_compose_msi_msg() instead of adding iommu_dma_select_msi()
> in PATCH6
> - Addressed Conor's comments in PATCH3
> - Addressed Conor's and Rob's comments in PATCH7
>
> Changes since v2:
> - Rebased on Linux-6.4-rc1
> - Addressed Rob's comments on DT bindings patches 4 and 8.
> - Addessed Marc's comments on IMSIC driver PATCH5
> - Replaced use of OF apis in APLIC and IMSIC drivers with FWNODE apis
> this makes both drivers easily portable for ACPI support. This also
> removes unnecessary indirection from the APLIC and IMSIC drivers.
> - PATCH1 is a new patch for portability with ACPI support
> - PATCH2 is a new patch to fix probing in APLIC drivers for APLIC-only systems.
> - PATCH7 is a new patch which addresses the IOMMU DMA domain issues pointed
> out by SiFive
>
> Changes since v1:
> - Rebased on Linux-6.2-rc2
> - Addressed comments on IMSIC DT bindings for PATCH4
> - Use raw_spin_lock_irqsave() on ids_lock for PATCH5
> - Improved MMIO alignment checks in PATCH5 to allow MMIO regions
> with holes.
> - Addressed comments on APLIC DT bindings for PATCH6
> - Fixed warning splat in aplic_msi_write_msg() caused by
> zeroed MSI message in PATCH7
> - Dropped DT property riscv,slow-ipi instead will have module
> parameter in future.
>
> Anup Patel (11):
> irqchip/sifive-plic: Convert PLIC driver into a platform driver
> irqchip/riscv-intc: Add support for RISC-V AIA
> dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller
> irqchip: Add RISC-V incoming MSI controller early driver
> irqchip/riscv-imsic: Add device MSI domain support for platform
> devices
> irqchip/riscv-imsic: Add device MSI domain support for PCI devices
> dt-bindings: interrupt-controller: Add RISC-V advanced PLIC
> irqchip: Add RISC-V advanced PLIC driver for direct-mode
> irqchip/riscv-aplic: Add support for MSI-mode
> RISC-V: Select APLIC and IMSIC drivers
> MAINTAINERS: Add entry for RISC-V AIA drivers
>
> Björn Töpel (1):
> genirq/matrix: Dynamic bitmap allocation
>
> Thomas Gleixner (13):
> irqchip/gic-v3: Make gic_irq_domain_select() robust for zero parameter
> count
> genirq/irqdomain: Remove the param count restriction from select()
> genirq/msi: Extend msi_parent_ops
> genirq/irqdomain: Add DOMAIN_BUS_DEVICE_IMS
> platform-msi: Prepare for real per device domains
> irqchip: Convert all platform MSI users to the new API
> genirq/msi: Provide optional translation op
> genirq/msi: Split msi_domain_alloc_irq_at()
> genirq/msi: Provide DOMAIN_BUS_WIRED_TO_MSI
> genirq/msi: Optionally use dev->fwnode for device domain
> genirq/msi: Provide allocation/free functions for "wired" MSI
> interrupts
> genirq/irqdomain: Reroute device MSI create_mapping
> genirq/msi: Provide MSI_FLAG_PARENT_PM_DEV

I have rebased and included 13 patches (which add per-device MSI domain
infrastructure) from your series [1]. In this series, the IMSIC driver
implements the msi_parent_ops and APLIC driver implements wired-to-msi
bridge using your new infrastructure.

The remaining 27 patches of your series [1] requires testing on ARM
platforms which I don't have. I suggest these remaining patches to
go as separate series.

I hope you are okay with this approach.

Best Regards,
Anup

[1] https://lore.kernel.org/linux-arm-kernel/20221121135653.208611233@xxxxxxxxxxxxx/


>
> .../interrupt-controller/riscv,aplic.yaml | 172 ++++
> .../interrupt-controller/riscv,imsics.yaml | 172 ++++
> MAINTAINERS | 14 +
> arch/riscv/Kconfig | 2 +
> arch/x86/include/asm/hw_irq.h | 2 -
> drivers/base/platform-msi.c | 97 ++
> drivers/dma/mv_xor_v2.c | 8 +-
> drivers/dma/qcom/hidma.c | 6 +-
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 5 +-
> drivers/irqchip/Kconfig | 25 +
> drivers/irqchip/Makefile | 3 +
> drivers/irqchip/irq-gic-v3.c | 6 +-
> drivers/irqchip/irq-riscv-aplic-direct.c | 343 +++++++
> drivers/irqchip/irq-riscv-aplic-main.c | 232 +++++
> drivers/irqchip/irq-riscv-aplic-main.h | 53 ++
> drivers/irqchip/irq-riscv-aplic-msi.c | 256 +++++
> drivers/irqchip/irq-riscv-imsic-early.c | 241 +++++
> drivers/irqchip/irq-riscv-imsic-platform.c | 403 ++++++++
> drivers/irqchip/irq-riscv-imsic-state.c | 887 ++++++++++++++++++
> drivers/irqchip/irq-riscv-imsic-state.h | 105 +++
> drivers/irqchip/irq-riscv-intc.c | 34 +-
> drivers/irqchip/irq-sifive-plic.c | 239 +++--
> drivers/mailbox/bcm-flexrm-mailbox.c | 8 +-
> drivers/perf/arm_smmuv3_pmu.c | 4 +-
> drivers/ufs/host/ufs-qcom.c | 8 +-
> include/linux/irqchip/riscv-aplic.h | 119 +++
> include/linux/irqchip/riscv-imsic.h | 87 ++
> include/linux/irqdomain.h | 17 +
> include/linux/irqdomain_defs.h | 2 +
> include/linux/msi.h | 21 +
> kernel/irq/irqdomain.c | 28 +-
> kernel/irq/matrix.c | 28 +-
> kernel/irq/msi.c | 184 +++-
> 33 files changed, 3636 insertions(+), 175 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aplic.yaml
> create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,imsics.yaml
> create mode 100644 drivers/irqchip/irq-riscv-aplic-direct.c
> create mode 100644 drivers/irqchip/irq-riscv-aplic-main.c
> create mode 100644 drivers/irqchip/irq-riscv-aplic-main.h
> create mode 100644 drivers/irqchip/irq-riscv-aplic-msi.c
> create mode 100644 drivers/irqchip/irq-riscv-imsic-early.c
> create mode 100644 drivers/irqchip/irq-riscv-imsic-platform.c
> create mode 100644 drivers/irqchip/irq-riscv-imsic-state.c
> create mode 100644 drivers/irqchip/irq-riscv-imsic-state.h
> create mode 100644 include/linux/irqchip/riscv-aplic.h
> create mode 100644 include/linux/irqchip/riscv-imsic.h
>
> --
> 2.34.1
>