Re: [PATCH 3/3] perf stat: Enable BPF counter with --for-each-cgroup

[Song Liu]

> On Jun 24, 2021, at 3:06 PM, Namhyung Kim <namhyung@xxxxxxxxxx> wrote:
> 
> On Thu, Jun 24, 2021 at 2:41 PM Song Liu <songliubraving@xxxxxx> wrote:
>> 
>> 
>> 
>>> On Jun 24, 2021, at 2:01 PM, Namhyung Kim <namhyung@xxxxxxxxxx> wrote:
>>> 
>>> On Thu, Jun 24, 2021 at 9:20 AM Song Liu <songliubraving@xxxxxx> wrote:
>>>>>>> +
>>>>>>> +// single set of global perf events to measure
>>>>>>> +struct {
>>>>>>> +     __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
>>>>>>> +     __uint(key_size, sizeof(__u32));
>>>>>>> +     __uint(value_size, sizeof(int));
>>>>>>> +     __uint(max_entries, 1);
>>>>>>> +} events SEC(".maps");
>>>>>>> +
>>>>>>> +// from logical cpu number to event index
>>>>>>> +// useful when user wants to count subset of cpus
>>>>>>> +struct {
>>>>>>> +     __uint(type, BPF_MAP_TYPE_HASH);
>>>>>>> +     __uint(key_size, sizeof(__u32));
>>>>>>> +     __uint(value_size, sizeof(__u32));
>>>>>>> +     __uint(max_entries, 1);
>>>>>>> +} cpu_idx SEC(".maps");
>>>>>> 
>>>>>> How about we make cpu_idx a percpu array and use 0,1 for
>>>>>> disable/enable profiling on this cpu?
>>>>> 
>>>>> No, it's to calculate an index to the cgrp_readings map which
>>>>> has the event x cpu x cgroup number of elements.
>>>>> 
>>>>> It controls enabling events with a global (bss) variable.
>>>> 
>>>> If we make cgrp_idx a per cpu array, we probably don't need the
>>>> cpu_idx map?
>>> 
>>> Right.
> 
> Maybe not.  Sometimes we want to profile a subset of cpus only.
> In that case, cpu != idx then I think we still need this.

We can only attach the bpf program on selected CPUs. Say, we want
CPUs 1, 3, 5. We just do 

	for (i in [1, 3, 5]) {
		link = bpf_program__attach_perf_event(skel->progs.on_switch,
						      FD(cgrp_switch, i));
		/* */
	}

The value arrays are still for all cpu, but they will just report zero
for CPU 0, 2, 4, .... 

Would this work? 
	
[...]


>>>>> Maybe.  But I don't know how to access the elements
>>>>> in a per-cpu map from userspace.
>>>> 
>>>> Please refer to bperf__read() reading accum_readings. Basically, we read
>>>> one index of all CPUs with one bpf_map_lookup_elem().
>>> 
>>> Thanks!  So when I use a per-cpu array with 3 elements, I can access
>>> to cpu/elem entries in a row like below, right?
>>> 
>>> 0/0, 0/1, 0/2, 1/0, 1/1, 1/2, 2/0, 2/1, 2/2, 3/0, ...
>> 
>> I am not sure I am following here.
>> 
>> Say the system have 10 cpus, and the array has 3 elements. We can do:
>> 
>>        __u32 values[10];  /* assuming both key and value are __u32 */
>>        __u32 elem;
>>        int cpu;
>> 
>>        for (elem = 0; elem < 3; elem++) {
>>                bpf_map_lookup_elem(map_fd, &elem, values);
>>                for (cpu = 0; cpu < 10; cpu++)
>>                        values[cpu] /* this is the value for cpu/elem */
>>        }
> 
> Thanks for the explanation, I didn't think that way.
> I thought it like below:
> 
>    __u32 elem, value;
> 
>    for (elem = 0; elem < 3 * 10; elem++) {
>        bpf_map_lookup_elem(map_fd, &elem, &value);
>    }
> 
> So in this case, the actual value size is like below, right?
> 
>  value-size = map-value-size * number-of-cpu

This is right (for user space). 

Thanks,
Song