Re: [PATCH v4 3/7] s390: ap: associate a ap_vfio_queue and a matrix mdev

[Pierre Morel]

On 26/02/2019 19:14, Tony Krowiak wrote:
> On 2/22/19 10:29 AM, Pierre Morel wrote:
> > We need to associate the ap_vfio_queue, which will hold the
> > per queue information for interrupt with a matrix mediated device
> > which hold the configuration and the way to the CRYCB.
> >
> > Let's do this when assigning a APID or a APQI to the mediated device
> > and clear the relation when unassigning.
> >
> > Queuing the devices on a list of free devices and testing the
> > matrix_mdev pointer to the associated matrix allow us to know
> > if the queue is associated to the matrix device and associated
> > or not to a mediated device.
> >
> > When resetting an AP queue we must wait until there are no more
> > messages in the message queue before considering the queue is really
> > in a clean state.
> >
> > Let's do it and wait until the status response code indicate the
> > queue is empty after issuing a PAPQ/ZAPQ instruction.
> >
> > Being at work on the reset function, let's simplify
> > vfio_ap_mdev_reset_queue and vfio_ap_mdev_reset_queues by using the
> > vfio_ap_queue structure as parameter.
> >
> > Signed-off-by: Pierre Morel <pmorel@xxxxxxxxxxxxx>
> > ---
> > Â drivers/s390/crypto/vfio_ap_ops.c | 385 
> > +++++++++++++++++++-------------------
> > Â 1 file changed, 189 insertions(+), 196 deletions(-)

...snip...

> > +static int vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q)
> > +{
> > +ÂÂÂ struct ap_queue_status status;
> > +ÂÂÂ int retry = 20;
> > +
> > +ÂÂÂ do {
> > +ÂÂÂÂÂÂÂ status = ap_zapq(q->apqn);
> > +ÂÂÂÂÂÂÂ switch (status.response_code) {
> > +ÂÂÂÂÂÂÂ case AP_RESPONSE_NORMAL:
> > +ÂÂÂÂÂÂÂÂÂÂÂ while (!status.queue_empty && retry--) {
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ msleep(20);
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ status = ap_tapq(q->apqn, NULL);
> > +ÂÂÂÂÂÂÂÂÂÂÂ }
>
> I am not sure the above is necessary. I have an email out to the author
> of the architecture doc to verify.

I do not know the question you asked but the documentation is very clear 
on the reset behavior: a queue is completely reseted only after the RC 
of reset/zapq is 0 and the queue_empty bit is set.

> > +ÂÂÂÂÂÂÂÂÂÂÂ if (retry <= 0)
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ pr_warn("%s: queue 0x%04x not empty\n",

...snip...

> > + * @matrix_mdev: the matrix mediated device for which we want to 
> > associate
> > + *ÂÂÂÂÂÂÂÂ all available queues with a given apqi.
> > + * @apid:ÂÂÂÂ The apid which associated with all defined APQI of the
> > + *ÂÂÂÂÂÂÂÂ mediated device will define a AP queue.
> > ÂÂ *
> > - * - If @data contains only an apid value, @data will be flagged as
> > - *ÂÂ reserved if the APID field in the AP queue device matches
> > - *
> > - * - If @data contains only an apqi value, @data will be flagged as
> > - *ÂÂ reserved if the APQI field in the AP queue device matches
> > - *
> > - * Returns 0 to indicate the input to function succeeded. Returns 
> > -EINVAL if
> > - * @data does not contain either an apid or apqi.
> > + * We remove the queue from the list of queues associated with the
> > + * mediated device and put them back to the free list of the matrix
> > + * device and clear the matrix_mdev pointer.
> > ÂÂ */
> > -static int vfio_ap_has_queue(struct device *dev, void *data)
> > +static void vfio_ap_put_all_domains(struct ap_matrix_mdev *matrix_mdev,
> > +ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ int apid)
>
> I would prefer this be named:
>
>  ÂÂÂÂvfio_ap_mdev_free_queues_with_apid()
>
> get/put is typically used to increment/decrement reference counters.
> What you are doing in this function freeing all queues connected to 
> specified card.

OK, I can change this function name and the further one you mentioned.

> > Â {
> > -ÂÂÂ struct vfio_ap_queue_reserved *qres = data;
> > -ÂÂÂ struct ap_queue *ap_queue = to_ap_queue(dev);
> > -ÂÂÂ ap_qid_t qid;
> > -ÂÂÂ unsigned long id;
> > +ÂÂÂ int apqi, apqn;
> > -ÂÂÂ if (qres->apid && qres->apqi) {
> > -ÂÂÂÂÂÂÂ qid = AP_MKQID(*qres->apid, *qres->apqi);
> > -ÂÂÂÂÂÂÂ if (qid == ap_queue->qid)
> > -ÂÂÂÂÂÂÂÂÂÂÂ qres->reserved = true;
> > -ÂÂÂ } else if (qres->apid && !qres->apqi) {
> > -ÂÂÂÂÂÂÂ id = AP_QID_CARD(ap_queue->qid);
> > -ÂÂÂÂÂÂÂ if (id == *qres->apid)
> > -ÂÂÂÂÂÂÂÂÂÂÂ qres->reserved = true;
> > -ÂÂÂ } else if (!qres->apid && qres->apqi) {
> > -ÂÂÂÂÂÂÂ id = AP_QID_QUEUE(ap_queue->qid);
> > -ÂÂÂÂÂÂÂ if (id == *qres->apqi)
> > -ÂÂÂÂÂÂÂÂÂÂÂ qres->reserved = true;
> > -ÂÂÂ } else {
> > -ÂÂÂÂÂÂÂ return -EINVAL;
> > +ÂÂÂ for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm, AP_DOMAINS) {
> > +ÂÂÂÂÂÂÂ apqn = AP_MKQID(apid, apqi);
> > +ÂÂÂÂÂÂÂ vfio_ap_free_queue(apqn, matrix_mdev);
> > ÂÂÂÂÂ }
>
> Maybe you should clear the bit corresponding to apid from the APM here?

I do not think so, this is pure list handling, the APM bit is already 
cleared in the unassign_adapter_store function.

I only answered once for all comments on naming and bit mask but will 
treat them the same way.
Thanks for comments.

Regards,
Pierre



--
Pierre Morel
Linux/KVM/QEMU in BÃblingen - Germany