Re: [RFC PATCH] perf/x86/intel/rapl: avoid access unallocate memory

From: Charles (Chas) Williams
Date: Fri Nov 04 2016 - 16:43:12 EST

On 11/04/2016 02:03 PM, Sebastian Andrzej Siewior wrote:
On 2016-11-04 08:20:37 [-0400], Charles (Chas) Williams wrote:
The initial CPU boots and is identified:

[ 0.009018] identify_boot_cpu
[ 0.009174] generic_identify: phys_proc_id is now 0
[ 0.009427] identify_cpu: before c ffffffff81ae2680 logical_proc_id 0 c->phys_proc_id 0
[ 0.009506] identify_cpu: after c ffffffff81ae2680 logical_proc_id 65535 c->phys_proc_id 0

So, this is fine because the APIC hasn't been scanned yet. APIC
now gets scanned:

[ 0.015789] smpboot: APIC(0) Converting physical 0 to logical package 0, cpu 0 (ffff88023fc0a040)
[ 0.015794] smpboot: APIC(1) Converting physical 1 to logical package 1, cpu 1 (ffff88023fd0a040)
[ 0.015797] smpboot: Max logical packages: 2

where is the APICID here is comming from?

This comes from here:

unsigned int apicid = apic->cpu_present_to_apicid(cpu);

if (apicid == BAD_APICID || !apic->apic_id_valid(apicid))
if (!topology_update_package_map(apicid, cpu))

And I think this is the part that is "wrong". The apicid appears to
be a logical CPU id. I believe that in most cases this mapping comes
from x86_bios_cpu_apicid (or x86_cpu_to_apicid) which is generated in
generic_processor_info() which maps apicid's to logical cpu indexes.

Note that apic->cpu_present_to_apicid() is using just the cpu_index.

for_each_present_cpu(cpu) {
unsigned int apicid = apic->cpu_present_to_apicid(cpu);

if (apicid == BAD_APICID || !apic->apic_id_valid(apicid))
if (!topology_update_package_map(apicid, cpu))
pr_warn("CPU %u APICId %x disabled\n", cpu, apicid);
per_cpu(x86_bios_cpu_apicid, cpu) = BAD_APICID;
set_cpu_possible(cpu, false);
set_cpu_present(cpu, false);

So, at this point, I think everything is correct. But now the secondary
CPU's "boot":

[ 0.236569] identify_secondary_cpu
[ 0.236620] generic_identify: phys_proc_id is now 2

so here is where fun starts. Xen has also
arch/x86/xen/smp.c::cpu_bringup() where the phys_proc_id is changed. But
isn't done for vmware but it might a place where they duct tape things.

How is this APIC id different from the earlier? I guess based on your
output that generic_identify() changes the content of phys_proc_id.

[ 0.236745] identify_cpu: before c ffff88023fd0a040 logical_proc_id 65535 c->phys_proc_id 2
[ 0.236747] identify_cpu: after c ffff88023fd0a040 logical_proc_id 65535 c->phys_proc_id 2

So, APIC discovered I have a cpu 0 and 1 but generic_identify() is called
my second CPU, 2. This is >= max_physical_pkg_id, so it is going to get
set to -1.

Now. max_physical_pkg_id is huge. The physical_to_logical_pkg array is
set to -1 on init so slot two has the value -1. That is what you see -
not the -1 because of ">= max_physical_pkg_id".

The comment at the end of identfy_cpu() says:

/* The boot/hotplug time assigment got cleared, restore it */

So, logical_proc_id being wrong here before restoration doesn't bother
me since I assume something in booting the secondary CPU's clears any
existing cpu data.

I know detect_extended_topology() is likely being called for both CPU's
and getting the right values (checking this now). I don't know why
generic_identify() is resetting this value.

I don't know either. But it is clearly reading the apic id twice and
second approach is different from the first which leads to different
results. So if you figure out how the first APICID for the second CPU is
retrieved and then you see how it happens for the second time. There
must be a difference.

The phys core id from generic_identify() comes from the CPU's EBX register
so we _know_ this is right.

if (c->cpuid_level >= 0x00000001) {
c->initial_apicid = (cpuid_ebx(1) >> 24) & 0xFF;
#ifdef CONFIG_X86_32
# ifdef CONFIG_SMP
c->apicid = apic->phys_pkg_id(c->initial_apicid, 0);
# else
c->apicid = c->initial_apicid;
# endif
c->phys_proc_id = c->initial_apicid;

The intel docs
claims this is the Local APIC ID. So it seems likely this is correct
value. It's not clear it matter if this is the right value or not
though. Even if this is the correct apicid, nothing knows about it.

An argument could be made that instead of checking the cpuid level, we
could just use the apicid based on the cpu index just like the other code.
It would be consistent at least then.