Re: [PATCH v6 5/5] KVM: selftests: Allowing running dirty_log_perf_test on specific CPUs

[Sean Christopherson]

On Thu, Oct 27, 2022, Wang, Wei W wrote:
> On Wednesday, October 26, 2022 11:44 PM, Sean Christopherson wrote:
> > > I think it would be better to do the thread pinning at the time when
> > > the thread is created by providing a pthread_attr_t attr, e.g. :
> > >
> > > pthread_attr_t attr;
> > >
> > > CPU_SET(vcpu->pcpu, &cpu_set);
> > > pthread_attr_setaffinity_np(&attr, sizeof(cpu_set_t), &cpu_set);
> > > pthread_create(thread, attr,...);
> > >
> > > Also, pinning a vCPU thread to a pCPU is a general operation which
> > > other users would need. I think we could make it more general and put
> > > it to kvm_util.
> > 
> > We could, but it taking advantage of the pinning functionality would require
> > plumbing a command line option for every test, 
> 
> I think we could make this "pinning" be optional (no need to force everyone
> to use it).

Heh, it's definitely optional.

> > If we go this route in the future, we'd need to add a worker trampoline as the
> > pinning needs to happen in the worker task itself to guarantee that the pinning
> > takes effect before the worker does anything useful.  That should be very
> > doable.
> 
> The alternative way is the one I shared before, using this:
> 
> /* Thread created with attribute ATTR will be limited to run only on
>    the processors represented in CPUSET.  */
> extern int pthread_attr_setaffinity_np (pthread_attr_t *__attr,
>                                  size_t __cpusetsize,
>                                  const cpu_set_t *__cpuset)
> 
> Basically, the thread is created on the pCPU as user specified.
> I think this is better than "creating the thread on an arbitrary pCPU
> and then pinning it to the user specified pCPU in the thread's start routine".

Ah, yeah, that's better.

> > I do like the idea of extending __vcpu_thread_create(), but we can do that once
> > __vcpu_thread_create() lands to avoid further delaying this series.
> 
> Sounds good. I can move some of those to vcpu_thread_create() once it's ready later.
> 
> >  struct perf_test_args {
> > @@ -43,8 +41,12 @@ struct perf_test_args {
> >  	bool nested;
> >  	/* True if all vCPUs are pinned to pCPUs */
> >  	bool pin_vcpus;
> > +	/* The vCPU=>pCPU pinning map. Only valid if pin_vcpus is true. */
> > +	uint32_t vcpu_to_pcpu[KVM_MAX_VCPUS];
> 
> How about putting the pcpu id to "struct kvm_vcpu"? (please see below code
> posed to shows how that works). This is helpful when we later make this more generic,
> as kvm_vcpu is used by everyone.

I don't think "pcpu" belongs in kvm_vcpu, even in the long run.  The vast, vast
majority of tests will never care about pinning, which means that vcpu->pcpu can't
be used for anything except the actual pinning.   And for pinning, the "pcpu"
doesn't need to be persistent information, i.e. doesn't need to live in kvm_vcpu.

> Probably we also don't need "bool pin_vcpus".

Yeah, but for selftests shaving bytes is not exactly top priority, and having a
dedicated flag avoids the need for magic numbers.  If Vipin had used -1, I'd
probably be fine with that, but I'm also totally fine using a dedicated flag too.

> We could initialize pcpu_id to -1 to indicate that the vcpu doesn't need
> pinning (this is also what I meant above optional for other users).
> 
> Put the whole changes together (tested and worked fine), FYI:

The big downside of this is forcing all callers of perf_test_create_vm() to pass
in NULL.  I really want to move away from this pattern as it makes what should be
simple code rather difficult to read due to having a bunch of "dead" params
dangling off the end of function calls.