Re: [PREEMPT-RT] Oops in rapl_cpu_prepare()

[Sebastian Andrzej Siewior]

On 2016-10-21 17:03:56 [-0400], Charles (Chas) Williams wrote:
> I can't get dedicated access to the specific bare metal since it is
> running as a dedicated hypervisor.  I haven't seen this issue anywhere
> else though with the 4.8 kernel.

That is something :)

> > If a callback (such as CPUHP_PERF_X86_RAPL_PREP) fail then we rollback
> > to the starting point (in case of CPU up it would be CPUHP_OFFLINE.
> 
> You'll like this, I just did a little printk debugging because it was
> easier than trying to get a debugger running:
> 
> 	[    3.107126] init_rapl_pmus: maxpkg 4
there! vmware bug. It probably worked by chance.

> 	[    3.107263] rapl_cpu_prepare: pmu ffff880234faa540  cpu 0  pkgid 0
> 	[    3.107400] rapl_cpu_prepare: pmu ffff880234faa600  cpu 1  pkgid 2
> 	[    3.107537] rapl_cpu_prepare: pmu ffff880234faa6c0  cpu 2  pkgid 65535
> 	[    3.107662] rapl_cpu_online: pmu ffff880234faa540 cpu 0 pkgid 0
> 	[    3.107907] rapl_cpu_online: pmu ffff880234faa600 cpu 1 pkgid 2
> 	[    3.108133] rapl_cpu_online: pmu ffff880234faa6c0 cpu 2 pkgid 65535
> 	[    3.108333] rapl_cpu_online: pmu ffff880234faa6c0 cpu 3 pkgid 65535
> 
> where pkgid is topology_logical_package_id(cpu).
> 
> I can't understand why I don't see a cpu 3 during cpu prepare, when I
> see one later.  

because cpu 2 and 3 share the same package and if your printk is at the
bottom of the function, it will return early.

> The 65535 is a -1 from topology_phys_to_logical_pkg()
> getting assigned to the logical_proc_id apparently.

yes. The topology field is u16.

> So this is pretty puzzling.  Since this is a guest running under VMWare, I
> don't know that there is any particular CPU pinning or emulation of RAPL.

I assume "init_rapl_pmus: maxpkg 4" is from init_rapl_pmus() returning
topology_max_packages(). So it says 4 but then returns 65535 for CPU 2
and 3. That -1 comes probably from topology_update_package_map(). Could
you please send a complete boot log and try the following patch? This
one should fix your boot problem and disable RAPL if the info is
invalid.

diff --git a/arch/x86/events/intel/rapl.c b/arch/x86/events/intel/rapl.c
index 0a535cea8ff3..f5d85f2853d7 100644
--- a/arch/x86/events/intel/rapl.c
+++ b/arch/x86/events/intel/rapl.c
@@ -682,6 +682,15 @@ static int __init init_rapl_pmus(void)
 {
 	int maxpkg = topology_max_packages();
 	size_t size;
+	unsigned int cpu;
+
+	for_each_possible_cpu(cpu) {
+		if (topology_logical_package_id(cpu) >= maxpkg) {
+			pr_err("rapl pmu error: max package: %u but CPU%d belongs to %u\n",
+			       maxpkg, cpu, topology_logical_package_id(cpu));
+			return -EINVAL;
+		}
+	}
 
 	size = sizeof(*rapl_pmus) + maxpkg * sizeof(struct rapl_pmu *);
 	rapl_pmus = kzalloc(size, GFP_KERNEL);

Sebastian