Re: [PATCH v5 11/15] KVM: arm64: Reserve dirty ring space for the HDBSS buffer
From: Tian Zheng
Date: Sun Nov 08 2026 - 23:02:24 EST
On 11/6/2026 11:36 PM, Leonardo Bras wrote:
On Fri, Nov 06, 2026 at 06:36:04PM +0800, Tian Zheng wrote:
On 11/2/2026 10:54 PM, Leonardo Bras wrote:
On Tue, Sep 29, 2026 at 06:36:51PM +0800, Tian Zheng wrote:
The dirty ring reserves room for CPU-side dirty buffers via
kvm_cpu_dirty_log_size(). Report the HDBSS buffer entry count
through it, so a full buffer flush always fits in the ring.
Pin the HDBSS buffer size to PAGE_SIZE when dirty-ring is enabled,
via kvm_arch_dirty_ring_size_updated(): a larger buffer only
shrinks the soft_limit and forces more frequent userspace drains.
Signed-off-by: Tian Zheng <zhengtian10@xxxxxxxxxx>
---
arch/arm64/kvm/arm.c | 8 ++++++++
arch/arm64/kvm/mmu.c | 8 ++++++++
2 files changed, 16 insertions(+)
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 9d7bdece149a..5a7046acb62e 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -215,6 +215,14 @@ static int kvm_arm_default_max_vcpus(void)
return vgic_present ? kvm_vgic_get_max_vcpus() : KVM_MAX_VCPUS;
}
+void kvm_arch_dirty_ring_size_updated(struct kvm *kvm)
+{
+ if (!system_supports_hdbss())
+ return;
+
+ kvm->arch.hdbss_buffer_size = KVM_ARM_HDBSS_DEFAULT_SIZE;
+}
+
On the same subject as a previous patch:
I don't think this is the best approach, on top of adding a new weak
function to take care.
Maybe during the dirty-ring allocation, on vcpu creation, we could take
into account the requested dirty-ring size, and just allocate the HDBSS
buffer taking that into account. That would make it simpler, don't you
agree?
Thanks!
Leo
Hi Leo,
The "set once" guard we're discussing comes from the patch that adds the
ioctl, and it isn't really needed. We discussed this earlier [1]: we can
just initialize hdbss_buffer_size to PAGE_SIZE at VM creation and let
the ring reservation use it directly. **That also means
kvm_arch_dirty_ring_size_updated() can be removed.** Thanks for your
suggestion.
:)
As for why PAGE_SIZE is the right value in ring mode: the HDBSS buffer
is flushed on every VM exit, so a larger buffer doesn't reduce the flush
frequency. It only inflates the ring reservation (64 + size/
sizeof(u64)), which shrinks the soft_limit and forces more frequent
exits to userspace for the ring to be drained. Draining the dirty ring
from QEMU is more expensive than draining the HDBSS buffer from the
kernel, so the smallest buffer gives the best performance. In this mode,
a larger buffer provides no benefit to users.
Humm, if properly dimensioned larger buffers would be benefitial for
memory-intensive workloads, unless they are guest_exiting for other
reasons in the meantime.
Agreed, and the workload I tested is on the "unless" side. The guest
exits constantly for other reasons, so the larger buffer never got a
chance to pay off, and only cost ring capacity.
In my tests, I also saw performance degrade when more space is reserved.
Humm, what kind of workloads did you test?
Did you have the chance to monitor what were the reasons for VM exiting?
For the workload question, it's a Redis server in the guest under redis-
benchmark (SET-only, 2KB values, 1M keys, 100M requests). The traffic is
network-driven, so the guest exits constantly to service requests. The
HDBSS buffer only accumulates entries between two exits and is flushed
on every exit, so it rarely fills regardless of size. A larger buffer
only inflated the ring reservation, which shrinks the soft_limit and
makes the ring-full exit and userspace drain fire more often.
I am not saying this is mandatory for the HDBSS implementation, but the
whole idea of letting the user choose their own HDBSS buffer size is
because different workloads may benefit from different buffer sizes: if a
VM writes a lot to new memory, and exits very little for other reasons,
then a larger buffer will cause the VM to guest_exit much less often.
Does that make sense?
Thanks!
Leo
It does make sense. That's exactly the workload where a larger buffer
pays off, and the next version no longer gets in its way. The two ioctls
are no longer mutually exclusive.
The only coupling left is the reservation. hdbss_buffer_size starts at
SZ_4K at VM creation, so a ring enabled right away reserves for the
default buffer (64 + 512 entries), and the minimum ring size accepted is
unchanged from v5. KVM_CAP_ARM_HDBSS_BUFFER_SIZE is the gate. When the
ring is already enabled, a size is accepted only if its reservation
still fits in the ring, otherwise -EINVAL. In the other order, ring-
enable validation rejects a ring too small for the configured size.
So SZ_4K is just the conservative default for guests that exit often.
Anything larger, within what the ring can reserve, is up to the user.
Does this work for you?
Thanks,
Tian