Re: [PATCH v16 00/45] arm64: Support for Arm CCA in KVM
From: Gavin Shan
Date: Tue Aug 11 2026 - 23:08:19 EST
On 8/11/26 9:12 PM, Suzuki K Poulose wrote:
On 11/08/2026 05:44, Gavin Shan wrote:
On 8/7/26 8:05 AM, Suzuki K Poulose wrote:
[...]
Here is a cleaned up version, rebased on to Will's kvmtool master
branch:
https://gitlab.arm.com/linux-arm/kvmtool-cca cca/kvm-v16
(This works with the v15 of the KVM series too)
Build on top of Fuad's Guest memfd support patches.
With the following combination, I'm able boot up the realm guest.
tf-rmm: https://git.trustedfirmware.org/TF-RMM/tf- rmm.git (branch: topics/rmm-v2.0-poc_2)
Please be aware that CCA KVM v15 onwards, the above branch is not compatible. You should be using rmm-v2.0-poc_3. Please could you
confirm if this is still an issue ?
With tf-rmm/topics/rmm-v2.0-poc_2 + cca/host-v16 + kvmtool/cca/v16, there is no issue
and the realm guest can boot up successfully.
When tf-rmm/topics/rmm-v2.0-poc_3 is used, the realm guest boot gets stuck as I reported
earlier. Note the host is emulated by QEMU (TCG mode).
As the following calltrace indicates, -EAGAIN is returned from tf-rmm::update_ripas()
because true is returned from s2tte_drain_pending() for the S2TTE corresponding to
IPA 0x80000000. Linux host received error (RMI_ERROR_RTT, level=3) in ripas_change().
Upon this specific error and the IPA range [0x80000000 0x90000000], find_map_level()
returns level of 2, and realm_create_rtt_levels() returns 0 without populating any
RTTs. After that, rmi_rtt_set_ripas() is re-executed and the above loop starts over
again.
Linux host
==========
kvm_arch_vcpu_ioctl_run // cca/host-v16
check_vcpu_requests
kvm_check_request
kvm_rec_handle_request
kvm_complete_ripas_change
realm_set_ipa_state
ripas_change
rmi_rtt_set_ripas
SMC_RMI_RTT_SET_RIPAS
TF-RMM
======
SMC_RMI_RTT_SET_RIPAS // tf-rmm/topics/rmm-v2.0-poc_3
smc_rtt_set_ripas
s2tt_walk_lock_unlock
rtt_set_ripas_range
update_ripas
s2tte_drain_pending // true, returns -EAGAIN
The problem is the pending-bit for RTE corresponding IPA address 0x80000000 isn't cleared
when SMC_RMI_RTT_SET_RIPAS is invoked. I didn't figure out how this bit is set and why
it's not cleared in time.
Thanks,
Gavin
Cheers
Suzuki
tf-a: https://git.trustedfirmware.org/TF-A/trusted-firmware- a.git (branch: master)
host: https://git.gitlab.arm.com/linux-arm/linux- cca.git (branch: cca-host/v16)
kvmtool: https://gitlab.arm.com/linux-arm/kvmtool- cca (branch: cca/kvm-v16)
However, the guest can't boot up and become stuck in SMC_RSI_IPA_STATE_SET request,
which can't completed by the host.
tf-rmm: https://git.trustedfirmware.org/TF-RMM/tf- rmm.git (branch: topics/rmm-v2.0-poc_3)
(1) Login the emulated host
machine$ ssh -o StrictHostKeyChecking=no root@10.26.1.240
(2) Start realm guest using kvmtool
root@host:~# lkvm run --realm -c 1 -m 256 \
-k /mnt/linux/arch/arm64/boot/Image \
-i /mnt/buildroot/output/images/rootfs.cpio.xz \
-p earlycon=uart,mmio,0x101000000
:
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x000f0510]
[ 0.000000] Linux version 7.2.0-rc5-gavin-gf5098b6bae76 (gshan@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc (GCC) 14.3.1 20251022 (Red Hat 14.3.1-4), GNU ld version 2.41-65.el10) #47 SMP PREEMPT Mon Jul 27 02:29:03 EDT 2026
[ 0.000000] KASLR enabled
[ 0.000000] Machine model: linux,dummy-virt
[ 0.000000] earlycon: uart0 at MMIO 0x0000000101000000 (options '')
[ 0.000000] printk: legacy bootconsole [uart0] enabled
[ 0.000000] efi: UEFI not found.
[ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT
[ 0.000000] NUMA: Faking a node at [mem 0x0000000080000000-0x000000008fffffff]
[ 0.000000] NODE_DATA(0) allocated [mem 0x8ff76dc0-0x8ff7ac7f]
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv1.1 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
[ 0.000000] psci: SMC Calling Convention v1.2
[ 0.000000] RME: Using RSI version 1.0
<... no more output from the guest ...>
(3) The output from host's serial console
SMC_RMI_REALM_CREATE 12a4c7000 12a4c6000 > RMI_INCOMPLETE 0 24 0 0
SMC_RMI_OP_MEM_DONATE 0 1002c7098 9 > RMI_INCOMPLETE 9 0 0 0
SMC_RMI_OP_CONTINUE 0 0 > RMI_SUCCESS 0 0
SMC_RMI_RTT_UNPROT_UNMAP 12a4c7000 18f85c000 18fe00000 0 0 > RMI_ERROR_RTT 2 0 0 0 0
SMC_RMI_RTT_UNPROT_UNMAP 12a4c7000 18fe00000 18fe10000 0 0 > RMI_ERROR_RTT 2 0 0 0 0
SMC_RMI_RTT_UNPROT_UNMAP 12a4c7000 18fe00000 18fe10000 0 0 > RMI_ERROR_RTT 2 0 0 0 0
SMC_RMI_REC_CREATE 12a4c7000 12a392000 12a391000 > RMI_INCOMPLETE 0 40 0 0
SMC_RMI_OP_MEM_DONATE 0 106406098 10 > RMI_INCOMPLETE 10 0 0 0
SMC_RMI_OP_CONTINUE 0 0 > RMI_SUCCESS 0 0
SMC_RMI_REALM_ACTIVATE 12a4c7000 > RMI_SUCCESS
Unhandled write S2_0_C0_C2_2
SMC_RMI_RTT_DATA_MAP 12a4c7000 8ffb4000 8ffb5000 1 129f1d004 > RMI_INCOMPLETE 0 0 0 1
SMC_RMI_OP_CONTINUE 0 0 > RMI_SUCCESS 8ffb5000 0
PSCI_84000000 0 0 0 0 0 0 0 > 10001 0 0 0
PSCI_84000006 0 0 0 0 0 0 0 > ffffffffffffffff 0 0 0
PSCI_8400000a 80000000 0 0 0 0 0 0 > 0 0 0 0
SMC_80000000 0 0 0 0 0 0 0 > 10002 0 0 0
SMC_84000050 1 0 0 ffffacd67d1e8000 ffffacd67d124000 0 0 > ffffffffffffffff 0 0 0
SMC_80000001 80000002 ffff 0 ffffacd67d1e8000 ffffacd67d124000 ffffacd67cd5d000 ffffacd67cd5dd38 > ffffffffffffffff 0 0 0
PSCI_8400000a c4000001 0 0 0 0 0 0 > 0 0 0 0
PSCI_8400000a c4000012 0 0 0 0 0 0 > ffffffffffffffff 0 0 0
PSCI_8400000a c4000015 0 0 0 0 0 0 > ffffffffffffffff 0 0 0
SMC_8600ff01 ffffacd67cfc0740 10002 0 0 0 0 ffffacd67cfb3cc8 > ffffffffffffffff 0 0 0
SMC_RSI_VERSION 10000 > RSI_SUCCESS 10000 10001
SMC_RSI_REALM_CONFIG 81385000 > RSI_SUCCESS
SMC_RSI_IPA_STATE_SET 80000000 90000000 1 0
SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
SMC_RMI_RTT_SET_RIPAS 12a4c7000 12a392000 80000000 90000000 > RMI_ERROR_RTT 3
<... the last message repeats ...>
Thanks,
Gavin