Re: [PATCH v2 2/7] s390: vfio-ap: maintain a shadow of the guest's CRYCB

From: Pierre Morel
Date: Tue May 07 2019 - 04:23:07 EST


On 06/05/2019 21:53, Tony Krowiak wrote:
On 5/6/19 2:49 AM, Pierre Morel wrote:
On 03/05/2019 23:14, Tony Krowiak wrote:
This patch introduces a shadow of the CRYCB being used by a guest. This
will enable to more effectively manage dynamic changes to the AP
resources installed on the host that may be assigned to an mdev device
and being used by a guest. For example:

* AP adapter cards can be dynamically added to and removed from the AP
ÂÂ configuration via the SE or an SCLP command.

* AP resources that disappear and reappear due to hardware malfunctions.

* AP queues bound to and unbound from the vfio_ap device driver by a
ÂÂ root user.

Signed-off-by: Tony Krowiak <akrowiak@xxxxxxxxxxxxx>
---
 drivers/s390/crypto/vfio_ap_ops.c | 91 ++++++++++++++++++++++++++++++++---
 drivers/s390/crypto/vfio_ap_private.h | 2 +
 2 files changed, 87 insertions(+), 6 deletions(-)

diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c
index b88a2a2ba075..44a04b4aa9ae 100644
--- a/drivers/s390/crypto/vfio_ap_ops.c
+++ b/drivers/s390/crypto/vfio_ap_ops.c
@@ -297,6 +297,45 @@ static void vfio_ap_mdev_wait_for_qempty(unsigned long apid, unsigned long apqi)
ÂÂÂÂÂ } while (--retry);
 }
+/*
+ * vfio_ap_mdev_update_crycb
+ *
+ * @matrix_mdev: the mediated matrix device
+ *
+ * Updates the AP matrix in the guest's CRYCB from it's shadow masks.
+ *
+ * Returns zero if the guest's CRYCB is successfully updated; otherwise,
+ * returns -ENODEV if a guest is not running or does not have a CRYCB.
+ */
+static int vfio_ap_mdev_update_crycb(struct ap_matrix_mdev *matrix_mdev)
+{
+ÂÂÂ if (!matrix_mdev->kvm || !matrix_mdev->kvm->arch.crypto.crycbd)
+ÂÂÂÂÂÂÂ return -ENODEV;
+
+ÂÂÂ kvm_arch_crypto_set_masks(matrix_mdev->kvm,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->shadow_crycb->apm,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->shadow_crycb->aqm,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->shadow_crycb->adm);
+
+ÂÂÂ return 0;
+}
+
+static int match_apqn(struct device *dev, void *data)
+{
+ÂÂÂ struct ap_queue *apq = to_ap_queue(dev);
+
+ÂÂÂ return (apq->qid == *(unsigned long *)(data)) ? 1 : 0;
+}
+
+static struct device *vfio_ap_get_queue_dev(unsigned long apid,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ unsigned long apqi)
+{
+ÂÂÂ unsigned long apqn = AP_MKQID(apid, apqi);
+
+ÂÂÂ return driver_find_device(&matrix_dev->vfio_ap_drv->driver, NULL,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ &apqn, match_apqn);
+}
+
 /**
ÂÂ * assign_adapter_store
ÂÂ *
@@ -805,14 +844,9 @@ static int vfio_ap_mdev_group_notifier(struct notifier_block *nb,
ÂÂÂÂÂ if (ret)
ÂÂÂÂÂÂÂÂÂ return NOTIFY_DONE;
-ÂÂÂ /* If there is no CRYCB pointer, then we can't copy the masks */
-ÂÂÂ if (!matrix_mdev->kvm->arch.crypto.crycbd)
+ÂÂÂ if (vfio_ap_mdev_update_crycb(matrix_mdev))
ÂÂÂÂÂÂÂÂÂ return NOTIFY_DONE;
-ÂÂÂ kvm_arch_crypto_set_masks(matrix_mdev->kvm, matrix_mdev->matrix.apm,
-ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->matrix.aqm,
-ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->matrix.adm);
-
ÂÂÂÂÂ return NOTIFY_OK;
 }
@@ -867,12 +901,55 @@ static int vfio_ap_mdev_reset_queues(struct mdev_device *mdev)
ÂÂÂÂÂ return rc;
 }
+static int vfio_ap_mdev_create_shadow_crycb(struct ap_matrix_mdev *matrix_mdev)
+{
+ÂÂÂ unsigned long apid, apqi, domid;
+ÂÂÂ struct device *dev;
+
+ÂÂÂ matrix_mdev->shadow_crycb = kzalloc(sizeof(*matrix_mdev->shadow_crycb),
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ GFP_KERNEL);
+ÂÂÂ if (!matrix_mdev->shadow_crycb)
+ÂÂÂÂÂÂÂ return -ENOMEM;
+
+ÂÂÂ vfio_ap_matrix_init(&matrix_dev->info, matrix_mdev->shadow_crycb);
+
+ÂÂÂ /*
+ÂÂÂÂ * Examine each APQN assigned to the mdev device. Set the APID and APQI
+ÂÂÂÂ * in the shadow CRYCB if and only if the queue device identified by
+ÂÂÂÂ * the APQN is in the configuration.
+ÂÂÂÂ */
+ÂÂÂ for_each_set_bit_inv(apid, matrix_mdev->matrix.apm,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->matrix.apm_max + 1) {
+ÂÂÂÂÂÂÂ for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->matrix.aqm_max + 1) {
+ÂÂÂÂÂÂÂÂÂÂÂ dev = vfio_ap_get_queue_dev(apid, apqi);
+ÂÂÂÂÂÂÂÂÂÂÂ if (dev) {
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ set_bit_inv(apid,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->shadow_crycb->apm);
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ set_bit_inv(apqi,
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ matrix_mdev->shadow_crycb->aqm);
+ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ put_device(dev);
+ÂÂÂÂÂÂÂÂÂÂÂ }

I think that if we do not find a device here we have a problem.
Don't we?

Other than the fact that the guest will not have any AP devices,
what would be the problem? What would you suggest?


Suppose we have in matrix_mdev->matrix:
1-2
1-3
2-2
2-3

We set the shadow_crycb with:
we find 1-2 we set 1 2
we find 1-3 we se 1 3
we find 2-2 we set 2 2
we do not find 2-3

we have set apm(1,2) aqm(2,3)
the guest can access 2-3 but we do not have the device.

Pierre

--
Pierre Morel
Linux/KVM/QEMU in BÃblingen - Germany