[kmorph] QEMU test report: takeover OK, a build break, two config traps

[Yun Wang]

Hi Cong,

After your announcement [1] I tried multikernel + kmorph in a QEMU/KVM
guest on x86_64. The takeover itself worked: 10 of 10 runs, with the
predecessor crashed by sysrq-c. I also hit a build break and two
configuration traps that the next tester will probably meet; details
and a fix below.

Tested: multikernel/linux 57fd249 (7.0.0-mk4), kmorph 9ec046e,
GCC 16.2.1, QEMU 11.1.3 (q35, KVM, 4 vCPUs, 4 GiB). The lines named
below are unchanged on master (77a05a4), and kmorph main is still
9ec046e.

1) Build break: CONFIG_INTEL_IOMMU=y with CONFIG_IRQ_REMAP=n

     drivers/iommu/intel/iommu.c:2806:35: error: 'struct intel_iommu'
     has no member named 'ir_table'

   intel_iommu_claim() uses iommu->ir_table unconditionally, but the
   member only exists under CONFIG_IRQ_REMAP (iommu.h:733).
   x86_64_defconfig + kvm_guest.config + the MULTIKERNEL options hits
   it, and so does this one command after "make prepare" on that
   config:

     make O=build drivers/iommu/intel/iommu.o

   The patch below follows the intel_irq_remap_claim() pattern already
   in iommu.h: a small helper in irq_remapping.c, a stub for
   !CONFIG_IRQ_REMAP next to the existing one, and iommu.c calls only
   the helper, so there is no #ifdef in the .c file. checkpatch
   --strict is clean. With it, the config that failed above builds as
   a full bzImage (0 errors, 0 warnings), drivers/iommu/intel/ also
   builds with IRQ_REMAP on, and kmorph takeovers worked on the
   patched IRQ_REMAP=n kernel in QEMU (10 of 10). QEMU has no vIOMMU
   here, so intel_iommu_claim() itself did not run. The three files are
   identical on master (77a05a4), so it applies there as is.

Signed-off-by: Yun Wang <7z0423arsenova@xxxxxxxxx>

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 7cec65cf5..d28928abe 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -2802,8 +2802,8 @@ int intel_iommu_claim(u64 base)
 	up_write(&dmar_global_lock);
 
 	iommu_pmu_register(iommu);
-	pr_info("%s: claimed; DMA translated%s by this kernel\n",
-		iommu->name, iommu->ir_table ? " and interrupts remapped" : "");
+	pr_info("%s: claimed; DMA translated%s by this kernel\n", iommu->name,
+		intel_irq_remap_active(iommu) ? " and interrupts remapped" : "");
 	return 0;
 
 out_unlock:
diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index ad6c2287c..005bec130 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1222,10 +1222,16 @@ static inline bool dmar_unit_foreign(struct intel_iommu *iommu)
 
 #ifdef CONFIG_IRQ_REMAP
 void intel_irq_remap_claim(struct intel_iommu *iommu);
+bool intel_irq_remap_active(struct intel_iommu *iommu);
 #else
 static inline void intel_irq_remap_claim(struct intel_iommu *iommu)
 {
 }
+
+static inline bool intel_irq_remap_active(struct intel_iommu *iommu)
+{
+	return false;
+}
 #endif
 
 void iommu_flush_write_buffer(struct intel_iommu *iommu);
diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c
index 9e5042fdb..a5aa32d1c 100644
--- a/drivers/iommu/intel/irq_remapping.c
+++ b/drivers/iommu/intel/irq_remapping.c
@@ -1205,6 +1205,17 @@ void intel_irq_remap_claim(struct intel_iommu *iommu)
 	raw_spin_unlock_irqrestore(&irq_2_ir_lock, flags);
 }
 
+/**
+ * intel_irq_remap_active - Whether interrupts are remapped through a unit
+ * @iommu: the DMAR unit
+ *
+ * Return: true when @iommu has an interrupt remapping table.
+ */
+bool intel_irq_remap_active(struct intel_iommu *iommu)
+{
+	return !!iommu->ir_table;
+}
+
 static struct intel_iommu *remapping_unit(u64 base)
 {
 	struct dmar_drhd_unit *drhd;

2) "kmorph arm" fails with ENOMEM on a 2 GiB x86_64 guest

     Baseline: failed to allocate 132 MB on node -1: -12
     kmorph: error: pool initialisation failed: Cannot allocate memory

   mk_take_memory() only tries ZONE_MOVABLE and ZONE_NORMAL, and below
   4 GiB x86_64 has neither: /proc/zoneinfo shows Normal and Movable
   with present 0, everything is in DMA32. With -m 4G (2 GiB above the
   PCI hole) "kmorph arm" takes 0.6 to 0.8 s. The README transcript
   shows 2047 MB of RAM, so a clearer error or a README note would
   help. I did not try movablecore=.

3) A successor with no boot-time PCI device adopts no PCI function

   Without devices= in kmorph.conf the takeover logs

     Failed to move device 0000:00:1f.2: -1
     kmorphd: adopted 0 devices: 0 reset, 0 quiesced

   From reading the code: mk_pcifront_host_fenced() returns early when
   the successor has no PCI bus, so mk_pcifront_direct_access stays
   false and mk_pcifront_adopt_function() returns -EPERM
   (pcifront.c:150-153 and 704). The same VM with
   devices=0000:01:00.0 instead logs "Host fenced: config space is
   reached directly through ECAM", adopts three functions, and the
   e1000e comes up as eth1. Without devices=, kmorphd's network
   restore also waits about 3 s per NIC that never shows up, so
   panic -> TAKEN_OVER took 6.8 s instead of 0.9 s (7 runs). The
   README lists devices= as optional. This one is also on the kmorph
   tracker, with the log lines:

     https://github.com/multikernel/kmorph/issues/2

Timing with devices= set and the defaults (5 x 100 ms probes), n = 10,
min / median / max in seconds on one host clock:

   panic -> fence starts                 0.51 / 0.56 / 0.61
   fence starts -> TAKEN_OVER            0.29 / 0.34 / 0.39
   panic -> shell on successor console   0.93 / 0.97 / 1.04

Two more sessions of 10 runs (the second on the patched kernel above)
gave the same picture: median panic -> TAKEN_OVER 0.91 and 0.90 s.
The series were run by scripts Claude generated; I also re-ran the
README commands by hand once (10 of 10, median 0.91 s).

This is a console-only image: no service, no traffic, no real
hardware, no vIOMMU. It says nothing about the nginx figure.

Two smaller things. kmorphd logs through one /dev/kmsg fd, which the
kernel rate-limits to 10 lines per 5 s unless printk.devkmsg=on, so on
a default command line "state TAKEN_OVER" never shows. Details:

   https://github.com/multikernel/kmorph/issues/1

And the successor boots without a swiotlb ("No low mem": its 128 MB
are above 4 GiB). I did not test a 32-bit-DMA device after a takeover,
so that one is a question, not a bug.

The QEMU rig, the logs and the exact commands are available if you
want them.

[1] https://lore.kernel.org/all/au5M_M_RzWfXLr0T@pop-os.localdomain/

Thanks,
Yun Wang