Re: [PATCH v5 0/6] KVM: arm64: nv: Implement nested stage-2 reverse map (new data structure)

[Marc Zyngier]

On Thu, 03 Sep 2026 14:28:16 +0100,
Wei-Lin Chang <weilin.chang@xxxxxxx> wrote:
> 
> On Thu, Sep 03, 2026 at 08:43:35AM +0100, Marc Zyngier wrote:
> > On Wed, 02 Sep 2026 17:35:00 +0100,
> > Wang Han <wanghan@xxxxxxxxxxxxxxxxx> wrote:
> > > 
> > > Hi Wei-Lin,
> > > 
> > > I tested this series on a Yitian 710 system with an ARM Neoverse-N2 CPU
> > > (128 CPUs, 2 NUMA nodes).
> > > 
> > > Test environment
> > > ----------------
> > > 
> > >   L0 kernel: Linux v7.2-rc6
> > >   L1 guest: Ubuntu 26.04 LTS, kernel 7.0.0-27-generic (aarch64)
> > >   QEMU: 10.2.3
> > > 
> > > L0 NUMA balancing was enabled (`/proc/sys/kernel/numa_balancing=1`).
> > > The host was booted with `kvm_arm.mode=nested`.
> > > 
> > > This series fixes a functional hang that is exposed when NUMA balancing is
> > > enabled.  The previous nested stage-2 unmap path is too slow for this
> > > workload, making the performance problem user-visible: NUMA balancing can
> > > leave the L1 guest unable to make progress and eventually hang during boot.
> > >
> > > The L1 was started with 8 vCPUs and 32 GiB of RAM using:
> > > 
> > >   qemu-system-aarch64 -smp 8 -m 32G \
> > >     -machine virt,accel=kvm,gic-version=3,virtualization=on \
> > >     -cpu host -nographic -enable-kvm \
> > >     -drive if=pflash,format=raw,readonly=on,file=pflash0_bak.img \
> > >     -drive if=pflash,format=raw,file=pflash1_bak.img \
> > >     -drive file=./ubuntu-vm.qcow2,format=qcow2,if=virtio,cache=none,aio=native \
> > >     -nic user,model=virtio-net-pci,hostfwd=tcp::11234-:22 \
> > >     -serial mon:stdio
> > > 
> > 
> > Puzzling. If you are only running an L1 in VHE mode, there is no
> > shadow S2, and therefore nothing to unmap. For shadow S2s to be built
> > and affect the MMU notifiers, you need to run an L2.
> 
> I was thinking the same at first, but realized even with L1 in VHE mode
> there is a small period of time where L1 runs in its EL1 during boot, so
> one nested MMU will become valid for each vCPU. That causes
> kvm_nested_s2_unmap() to iterate through the entire IPA space 8 times
> (-smp 8).

It should be one nested MMU for the whole VM, not one per vcpu.
That's assuming they share the same VMID+VTCR.

> What I am curious about is whether one single notifier unmap is enough
> to hang L1, or were there multiple notifier unmaps.
> 
> QEMU with -machine virt uses 40 IPA bits only, unmapping that takes:
> 1024  (4KB pages,  unmapping 1GB per iteration)
> 32768 (16KB pages, unmapping 32MB per iteration)
> 2048  (64KB pages, unmapping 512MB per iteration)
> iterations for each page size. There aren't many mappings in each
> iteration too. Does this really take that long on real hardware (even if
> this must be done 8 times)?

This should be close to being at zero cost, so something else is
amiss.

Could you please have a look?

	M.

-- 
Jazz isn't dead. It just smells funny.