[PATCH v2 00/29] Add KVM LoongArch support

From: Tianrui Zhao
Date: Mon Feb 20 2023 - 01:58:20 EST


This series adds KVM LoongArch support. Loongson 3A5000 supports hardware
assisted virtualization. With cpu virtualization, there are separate
hw-supported user mode and kernel mode in guest mode. With memory
virtualization, there are two-level hw mmu table for guest mode and host
mode. Also there is separate hw cpu timer with consant frequency in
guest mode, so that vm can migrate between hosts with different freq.
Currently, we are able to boot LoongArch Linux Guests.

Few key aspects of KVM LoongArch added by this series are:
1. Enable kvm hardware function when kvm module is loaded.
2. Implement VM and vcpu related ioctl interface such as vcpu create,
vcpu run etc. GET_ONE_REG/SET_ONE_REG ioctl commands are use to
get general registers one by one.
3. Hardware access about MMU, timer and csr are emulated in kernel.
4. Hardwares such as mmio and iocsr device are emulated in user space
such as APIC, IPI, pci devices etc.

The running environment of LoongArch virt machine:
1. Cross tools to build kernel and uefi:
$ wget https://github.com/loongson/build-tools/releases/download/2022.09.06/loongarch64-clfs-6.3-cross-tools-gcc-glibc.tar.xz
tar -vxf loongarch64-clfs-6.3-cross-tools-gcc-glibc.tar.xz -C /opt
export PATH=/opt/cross-tools/bin:$PATH
export LD_LIBRARY_PATH=/opt/cross-tools/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/cross-tools/loongarch64-unknown-linux-gnu/lib/:$LD_LIBRARY_PATH
2. This series is based on the linux source code:
https://github.com/loongson/linux-loongarch-kvm
Build command:
git checkout kvm-loongarch
make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu- loongson3_defconfig
make ARCH=loongarch CROSS_COMPILE=loongarch64-unknown-linux-gnu-
3. QEMU hypervisor with LoongArch supported:
https://github.com/loongson/qemu
Build command:
git checkout kvm-loongarch
./configure --target-list="loongarch64-softmmu" --enable-kvm
make
4. Uefi bios of LoongArch virt machine:
Reference: https://github.com/tianocore/edk2-platforms/tree/master/Platform/Loongson/LoongArchQemuPkg#readme
5. you can also access the binary files we have already build:
https://github.com/yangxiaojuan-loongson/qemu-binary

The command to boot loongarch virt machine:
$ qemu-system-loongarch64 -machine virt -m 4G -cpu la464 \
-smp 1 -bios QEMU_EFI.fd -kernel vmlinuz.efi -initrd ramdisk \
-serial stdio -monitor telnet:localhost:4495,server,nowait \
-append "root=/dev/ram rdinit=/sbin/init console=ttyS0,115200" \
--nographic

Changes since v1:
1. Seprate the original patch-01 and patch-03 into small patches, and the
patches mainly contain kvm module init, module exit, vcpu create, vcpu run,
etc.
2. Remove the original KVM_{GET,SET}_CSRS ioctl in the kvm uapi header,
and we use the common KVM_{GET,SET}_ONE_REG to access register.
3. Use BIT(x) to replace the "1 << n_bits" statement.

Tianrui Zhao (29):
LoongArch: KVM: Add kvm related header files
LoongArch: KVM: Implement kvm module related interface
LoongArch: KVM: Implement kvm hardware enable, disable interface
LoongArch: KVM: Implement VM related functions
LoongArch: KVM: Add vcpu related header files
LoongArch: KVM: Implement vcpu create and destroy interface
LoongArch: KVM: Implement vcpu run interface
LoongArch: KVM: Implement vcpu handle exit interface
LoongArch: KVM: Implement vcpu get, vcpu set registers
LoongArch: KVM: Implement vcpu ENABLE_CAP, CHECK_EXTENSION ioctl
interface
LoongArch: KVM: Implement fpu related operations for vcpu
LoongArch: KVM: Implement vcpu interrupt operations
LoongArch: KVM: Implement misc vcpu related interfaces
LoongArch: KVM: Implement vcpu load and vcpu put operations
LoongArch: KVM: Implement vcpu status description
LoongArch: KVM: Implement update VM id function
LoongArch: KVM: Implement virtual machine tlb operations
LoongArch: KVM: Implement vcpu timer operations
LoongArch: KVM: Implement kvm mmu operations
LoongArch: KVM: Implement handle csr excption
LoongArch: KVM: Implement handle iocsr exception
LoongArch: KVM: Implement handle idle exception
LoongArch: KVM: Implement handle gspr exception
LoongArch: KVM: Implement handle mmio exception
LoongArch: KVM: Implement handle fpu exception
LoongArch: KVM: Implement kvm exception vector
LoongArch: KVM: Implement vcpu world switch
LoongArch: KVM: Implement probe virtualization when loongarch cpu init
LoongArch: KVM: Enable kvm config and add the makefile

arch/loongarch/Kbuild | 1 +
arch/loongarch/Kconfig | 2 +
arch/loongarch/configs/loongson3_defconfig | 2 +
arch/loongarch/include/asm/cpu-features.h | 22 +
arch/loongarch/include/asm/cpu-info.h | 13 +
arch/loongarch/include/asm/inst.h | 16 +
arch/loongarch/include/asm/kvm_csr.h | 89 ++
arch/loongarch/include/asm/kvm_host.h | 257 +++++
arch/loongarch/include/asm/kvm_types.h | 11 +
arch/loongarch/include/asm/kvm_vcpu.h | 112 ++
arch/loongarch/include/asm/loongarch.h | 195 +++-
arch/loongarch/include/uapi/asm/kvm.h | 107 ++
arch/loongarch/kernel/asm-offsets.c | 32 +
arch/loongarch/kernel/cpu-probe.c | 53 +
arch/loongarch/kvm/Kconfig | 38 +
arch/loongarch/kvm/Makefile | 21 +
arch/loongarch/kvm/exit.c | 702 ++++++++++++
arch/loongarch/kvm/interrupt.c | 126 +++
arch/loongarch/kvm/main.c | 152 +++
arch/loongarch/kvm/mmu.c | 821 ++++++++++++++
arch/loongarch/kvm/switch.S | 327 ++++++
arch/loongarch/kvm/timer.c | 266 +++++
arch/loongarch/kvm/tlb.c | 31 +
arch/loongarch/kvm/trace.h | 137 +++
arch/loongarch/kvm/vcpu.c | 1118 ++++++++++++++++++++
arch/loongarch/kvm/vm.c | 85 ++
arch/loongarch/kvm/vmid.c | 64 ++
include/uapi/linux/kvm.h | 12 +
28 files changed, 4806 insertions(+), 6 deletions(-)
create mode 100644 arch/loongarch/include/asm/kvm_csr.h
create mode 100644 arch/loongarch/include/asm/kvm_host.h
create mode 100644 arch/loongarch/include/asm/kvm_types.h
create mode 100644 arch/loongarch/include/asm/kvm_vcpu.h
create mode 100644 arch/loongarch/include/uapi/asm/kvm.h
create mode 100644 arch/loongarch/kvm/Kconfig
create mode 100644 arch/loongarch/kvm/Makefile
create mode 100644 arch/loongarch/kvm/exit.c
create mode 100644 arch/loongarch/kvm/interrupt.c
create mode 100644 arch/loongarch/kvm/main.c
create mode 100644 arch/loongarch/kvm/mmu.c
create mode 100644 arch/loongarch/kvm/switch.S
create mode 100644 arch/loongarch/kvm/timer.c
create mode 100644 arch/loongarch/kvm/tlb.c
create mode 100644 arch/loongarch/kvm/trace.h
create mode 100644 arch/loongarch/kvm/vcpu.c
create mode 100644 arch/loongarch/kvm/vm.c
create mode 100644 arch/loongarch/kvm/vmid.c

--
2.31.1