Re: [PATCH v8 RESEND 0/4] Support kdump for AMD secure memory encryption(SME)

From: lijiang
Date: Fri Oct 05 2018 - 01:52:52 EST

Next message: Wolfgang Grandegger: "Re: [PATCH 2/2] can: tcan4x5x: Add tcan4x5x driver to the kernel"
Previous message: Souptick Joarder: "Re: [PATCH v2] mm: Introduce new function vm_insert_kmem_page"
In reply to: Borislav Petkov: "Re: [PATCH v8 RESEND 0/4] Support kdump for AMD secure memory encryption(SME)"
Next in thread: Borislav Petkov: "Re: [PATCH v8 RESEND 0/4] Support kdump for AMD secure memory encryption(SME)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

å 2018å10æ05æ 03:02, Borislav Petkov åé:
> On Thu, Oct 04, 2018 at 05:33:14PM +0800, lijiang wrote:
>> I have tested the patch again based on upstream 4.19.0-rc6, it works very well.
>
> How have you tested this?
>
> Please describe the steps in detail.
>

There are eight steps:

Step 1: prepare for test tools, you might refer to the cover-letter.
a. makedumpfile
b. crash-7.2.3
c. kexec-tools-2.0.17

Compile and install these test tools.

Step 2: make sure that the kernel option is enabled if this machine has SME feature.
CONFIG_AMD_MEM_ENCRYPT=y

Step 3: apply these patches based on upstream v4.19-rc6, compile and install kernel
#git am xxxx.patch
#make ARCH=x86_64 -j32
#make ARCH=x86_64 modules_install -j32
#make ARCH=x86_64 install

Step 4: configure kdump and modify some parameters for SME
a. configure kdump.conf
#cat /etc/kdump.conf
path /var/crash
core_collector makedumpfile -l --message-level 1 -d 31

b. add the parameter "mem_encrypt=on" for kernel command-line to grub.cfg, if
this machine has SME feature. And also add crashkernel=xx, which will reserve
memory for kdump.

Step 5: reboot, and then load the crash kernel image and kdump initramfs.

a: When SME is enabled, i use this command to load them:

#kexec -p /boot/vmlinuz-4.19.0-rc6+ --initrd=/boot/initramfs-4.19.0-rc6+kdump.img --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap mem_encrypt=on console=ttyS0,115200n81 LANG=en_US.UTF-8 earlyprintk=serial debug irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never disable_cpu_apicid=0"

b: When SME is disabled, i use this command to load them:

#kexec -p /boot/vmlinuz-4.19.0-rc6+ --initrd=/boot/initramfs-4.19.0-rc6+kdump.img --command-line="root=/dev/mapper/rhel_hp--dl385g10--03-root ro rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap mem_encrypt=off console=ttyS0,115200n81 LANG=en_US.UTF-8 earlyprintk=serial debug irqpoll nr_cpus=1 reset_devices cgroup_disable=memory mce=off numa=off udev.children-max=2 panic=10 rootflags=nofail acpi_no_memhotplug transparent_hugepage=never disable_cpu_apicid=0"

Step 6: trigger panic by sysrq
#echo c > /proc/sysrq-trigger

Step 7: check whether the vmcore has been created.

[root@hp-dl385g10-03 linux]# ls -al /var/crash/*
/var/crash/127.0.0.1-2018-10-05-01:20:20:
drwxr-xr-x. 2 root root 44 10æ 5 01:20 .
drwxr-xr-x. 3 root root 107 10æ 5 01:20 ..
-rw-------. 1 root root 1179265928 10æ 5 01:20 vmcore
-rw-r--r--. 1 root root 126571 10æ 5 01:20 vmcore-dmesg.txt

/var/crash/127.0.0.1-2018-10-05-01:35:21:
drwxr-xr-x. 2 root root 44 10æ 5 01:35 .
drwxr-xr-x. 4 root root 144 10æ 5 01:35 ..
-rw-------. 1 root root 1084270120 10æ 5 01:35 vmcore
-rw-r--r--. 1 root root 125578 10æ 5 01:35 vmcore-dmesg.txt

Step 8: check whether the crash tool can parse the vmcore
a. When SME is enabled.
#crash vmlinux /var/crash/127.0.0.1-2018-10-05-01\:20\:20/vmcore

crash 7.2.3++
Copyright (C) 2002-2017 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [308MB]: patching 85986 gdb minimal_symbol values

KERNEL: vmlinux
DUMPFILE: /var/crash/127.0.0.1-2018-10-05-01:20:20/vmcore [PARTIAL DUMP]
CPUS: 32
DATE: Fri Oct 5 01:19:40 2018
UPTIME: 00:04:04
LOAD AVERAGE: 0.18, 0.33, 0.16
TASKS: 462
NODENAME: hp-dl385g10-03.lab.eng.pek2.redhat.com
RELEASE: 4.19.0-rc6+
VERSION: #223 SMP Fri Oct 5 01:05:56 EDT 2018
MACHINE: x86_64 (2095 Mhz)
MEMORY: 31.8 GB
PANIC: "sysrq: SysRq : Trigger a crash"
PID: 9451
COMMAND: "bash"
TASK: ffff9d53c5f8c500 [THREAD_INFO: ffff9d53c5f8c500]
CPU: 26
STATE: TASK_RUNNING (SYSRQ)

crash> log
[ 0.000000] Linux version 4.19.0-rc6+ (root@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-34) (GCC)) #223 SMP Fri Oct 5 01:05:56 EDT 2018
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.19.0-rc6+ root=/dev/mapper/rhel_hp--dl385g10--03-root ro mem_encrypt=on crashkernel=2G,high rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap console=ttyS0,115200n81 LANG=en_US.UTF-8
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008bfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000008c000-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000029920fff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000029921000-0x0000000029921fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000029922000-0x0000000062242fff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000062243000-0x0000000062342fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000062343000-0x0000000062355fff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x0000000062356000-0x0000000062356fff] ACPI NVS
[ 0.000000] BIOS-e820: [mem 0x0000000062357000-0x00000000623d5fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000623d6000-0x0000000062615fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000062616000-0x0000000062637fff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000062638000-0x0000000062697fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000062698000-0x0000000062757fff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000062758000-0x0000000062758fff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x0000000062759000-0x0000000062789fff] usable
[ 0.000000] BIOS-e820: [mem 0x000000006278a000-0x000000006278cfff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x000000006278d000-0x00000000627d6fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000627d7000-0x00000000627d7fff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x00000000627d8000-0x000000006286afff] usable
[ 0.000000] BIOS-e820: [mem 0x000000006286b000-0x000000006286efff] reserved
[ 0.000000] BIOS-e820: [mem 0x000000006286f000-0x00000000682f8fff] usable
...
...
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS0] enabled
[ 0.000000] AMD Secure Memory Encryption (SME) active
[ 0.000000] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
[ 0.000000] ACPI: Core revision 20180810
...
...

b. When SME is disabled.
#crash vmlinux /var/crash/127.0.0.1-2018-10-05-01\:35\:21/vmcore

crash 7.2.3++
Copyright (C) 2002-2017 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [576MB]: patching 85986 gdb minimal_symbol values

KERNEL: vmlinux
DUMPFILE: /var/crash/127.0.0.1-2018-10-05-01:35:21/vmcore [PARTIAL DUMP]
CPUS: 32
DATE: Fri Oct 5 01:34:44 2018
UPTIME: 00:01:43
LOAD AVERAGE: 0.31, 0.20, 0.08
TASKS: 456
NODENAME: hp-dl385g10-03.lab.eng.pek2.redhat.com
RELEASE: 4.19.0-rc6+
VERSION: #223 SMP Fri Oct 5 01:05:56 EDT 2018
MACHINE: x86_64 (2095 Mhz)
MEMORY: 31.8 GB
PANIC: "sysrq: SysRq : Trigger a crash"
PID: 2093
COMMAND: "bash"
TASK: ffff9be9aa062e00 [THREAD_INFO: ffff9be9aa062e00]
CPU: 12
STATE: TASK_RUNNING (SYSRQ)

crash> log
[ 0.000000] Linux version 4.19.0-rc6+ (root@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-34) (GCC)) #223 SMP Fri Oct 5 01:05:56 EDT 2018
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.19.0-rc6+ root=/dev/mapper/rhel_hp--dl385g10--03-root ro mem_encrypt=off crashkernel=2G,high rd.lvm.lv=rhel_hp-dl385g10-03/root rd.lvm.lv=rhel_hp-dl385g10-03/swap console=ttyS0,115200n81 LANG=en_US.UTF-8
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008bfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000008c000-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000029920fff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000029921000-0x0000000029921fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000029922000-0x00000000622dbfff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000622dc000-0x000000006261bfff] reserved
[ 0.000000] BIOS-e820: [mem 0x000000006261c000-0x000000006263dfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000006263e000-0x000000006269dfff] reserved
[ 0.000000] BIOS-e820: [mem 0x000000006269e000-0x00000000627d9fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000627da000-0x00000000627ecfff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x00000000627ed000-0x00000000627edfff] ACPI NVS
[ 0.000000] BIOS-e820: [mem 0x00000000627ee000-0x00000000627f1fff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x00000000627f2000-0x00000000627f3fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000627f4000-0x00000000627f4fff] ACPI data
[ 0.000000] BIOS-e820: [mem 0x00000000627f5000-0x000000006286afff] usable
[ 0.000000] BIOS-e820: [mem 0x000000006286b000-0x000000006286efff] reserved
[ 0.000000] BIOS-e820: [mem 0x000000006286f000-0x00000000682f8fff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000682f9000-0x0000000068b05fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000068b06000-0x0000000068b09fff] ACPI NVS
[ 0.000000] BIOS-e820: [mem 0x0000000068b0a000-0x0000000068b1afff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000068b1b000-0x0000000068b1dfff] ACPI NVS
...
...
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS0] enabled
[ 0.000000] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
[ 0.000000] ACPI: Core revision 20180810
...
...

Regards,
Lianbo

> Thx.
>

Next message: Wolfgang Grandegger: "Re: [PATCH 2/2] can: tcan4x5x: Add tcan4x5x driver to the kernel"
Previous message: Souptick Joarder: "Re: [PATCH v2] mm: Introduce new function vm_insert_kmem_page"
In reply to: Borislav Petkov: "Re: [PATCH v8 RESEND 0/4] Support kdump for AMD secure memory encryption(SME)"
Next in thread: Borislav Petkov: "Re: [PATCH v8 RESEND 0/4] Support kdump for AMD secure memory encryption(SME)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]