Re: [RFC PATCH v3 0/7] device backed vmemmap crash dump support

[Zhijian Li (Fujitsu)]

ping again,

I would appreciate feedback from the nvdimm and linux-mm communities

Thanks
Zhijian


On 21/03/2024 13:40, Li Zhijian wrote:
> ping
> 
> 
> Any comment is welcome.
> 
> 
> On 06/03/2024 18:28, Li Zhijian wrote:
>> Hello folks,
>>
>> Compared with the V2[1] I posted a long time ago, this time it is a
>> completely new proposal design.
>>
>> ### Background and motivate overview ###
>> ---
>> Crash dump is an important feature for troubleshooting the kernel. It is the
>> final way to chase what happened at the kernel panic, slowdown, and so on. It
>> is one of the most important tools for customer support.
>>
>> Currently, there are 2 syscalls(kexec_file_load(2) and kexec_load(2)) to
>> configure the dumpable regions. Generally, (A)iomem resources registered with
>> flags (IORESOURCE_SYSTEM_RAM | IORESOUCE_BUSY) for kexec_file_load(2) or
>> (B)iomem resources registered with "System RAM" name prefix for kexec_load(2)
>> are dumpable.
>>
>> The pmem use cases including fsdax and devdax, could map their vmemmap to
>> their own devices. In this case, these part of vmemmap will not be dumped when
>> crash happened since these regions are satisfied with neither the above (A)
>> nor (B).
>>
>> In fsdax, the vmemmap(struct page array) becomes very important, it is one of
>> the key data to find status of reverse map. Lacking of the information may
>> cause difficulty to analyze trouble around pmem (especially Filesystem-DAX).
>> That means troubleshooters are unable to check more details about pmem from
>> the dumpfile.
>>
>> ### Proposal ###
>> ---
>> In this proposal, register the device backed vmemmap as a separate resource.
>> This resource has its own new flag and name, and then teaches kexec_file_load(2)
>> and kexec_load(2) to mark it as dumpable.
>>
>> Proposed flag: IORESOURCE_DEVICE_BACKED_VMEMMAP
>> Proposed name: "Device Backed Vmemmap"
>>
>> NOTE: crash-utils also needs to adapt to this new name for kexec_load()
>>
>> With current proposal, the /proc/iomem should show as following for device
>> backed vmemmap
>> # cat /proc/iomem
>> ...
>> fffc0000-ffffffff : Reserved
>> 100000000-13fffffff : Persistent Memory
>>    100000000-10fffffff : namespace0.0
>>      100000000-1005fffff : Device Backed Vmemmap  # fsdax
>> a80000000-b7fffffff : CXL Window 0
>>    a80000000-affffffff : Persistent Memory
>>      a80000000-affffffff : region1
>>        a80000000-a811fffff : namespace1.0
>>          a80000000-a811fffff : Device Backed Vmemmap # devdax
>>        a81200000-abfffffff : dax1.0
>> b80000000-c7fffffff : CXL Window 1
>> c80000000-147fffffff : PCI Bus 0000:00
>>    c80000000-c801fffff : PCI Bus 0000:01
>> ...
>>
>> ### Kdump service reloading ###
>> ---
>> Once the kdump service is loaded, if changes to CPUs or memory occur,
>> either by hot un/plug or off/onlining, the crash elfcorehdr should also
>> be updated. There are 2 approaches to make the reloading more efficient.
>> 1) Use udev rules to watch CPU and memory events, then reload kdump
>> 2) Enable kernel crash hotplug to automatically reload elfcorehdr (>= 6.5)
>>
>> This reloading also needed when device backed vmemmap layouts change, Similar
>> to what 1) does now, one could add the following as the first lines to the
>> RHEL udev rule file /usr/lib/udev/rules.d/98-kexec.rules:
>>
>> # namespace updated: watch daxX.Y(devdax) and pfnX.Y(fsdax) of nd
>> SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="bind", GOTO="kdump_reload"
>> SUBSYSTEM=="nd", KERNEL=="[dp][af][xn][0-9].*", ACTION=="unbind", GOTO="kdump_reload"
>> # devdax <-> system-ram updated: watch daxX.Y of dax
>> SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="bind", GOTO="kdump_reload"
>> SUBSYSTEM=="dax", KERNEL=="dax[0-9].*", ACTION=="unbind", GOTO="kdump_reload"
>>
>> Regarding 2), my idea is that it would need to call the memory_notify() in
>> devm_memremap_pages_release() and devm_memremap_pages() to trigger the crash
>> hotplug. This part is not yet mature, but it does not affect the whole feature
>> because we can still use method 1) alternatively.
>>
>> [1] https://lore.kernel.org/lkml/02066f0f-dbc0-0388-4233-8e24b6f8435b@xxxxxxxxxxx/T/
>> --------------------------------------------
>> changes from V2[1]
>> - new proposal design
>>
>> CC: Alison Schofield <alison.schofield@xxxxxxxxx>
>> CC: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
>> CC: Baoquan He <bhe@xxxxxxxxxx>
>> CC: Borislav Petkov <bp@xxxxxxxxx>
>> CC: Dan Williams <dan.j.williams@xxxxxxxxx>
>> CC: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
>> CC: Dave Jiang <dave.jiang@xxxxxxxxx>
>> CC: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
>> CC: "H. Peter Anvin" <hpa@xxxxxxxxx>
>> CC: Ingo Molnar <mingo@xxxxxxxxxx>
>> CC: Ira Weiny <ira.weiny@xxxxxxxxx>
>> CC: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
>> CC: Vishal Verma <vishal.l.verma@xxxxxxxxx>
>> CC: linux-cxl@xxxxxxxxxxxxxxx
>> CC: linux-mm@xxxxxxxxx
>> CC: nvdimm@xxxxxxxxxxxxxxx
>> CC: x86@xxxxxxxxxx
>>
>> Li Zhijian (7):
>>    mm: memremap: register/unregister altmap region to a separate resource
>>    mm: memremap: add pgmap_parent_resource() helper
>>    nvdimm: pmem: assign a parent resource for vmemmap region for the
>>      fsdax
>>    dax: pmem: assign a parent resource for vmemmap region for the devdax
>>    resource: Introduce walk device_backed_vmemmap res() helper
>>    x86/crash: make device backed vmemmap dumpable for kexec_file_load
>>    nvdimm: set force_raw=1 in kdump kernel
>>
>>   arch/x86/kernel/crash.c         |  5 +++++
>>   drivers/dax/pmem.c              |  8 ++++++--
>>   drivers/nvdimm/namespace_devs.c |  3 +++
>>   drivers/nvdimm/pmem.c           |  9 ++++++---
>>   include/linux/ioport.h          |  4 ++++
>>   include/linux/memremap.h        |  4 ++++
>>   kernel/resource.c               | 13 +++++++++++++
>>   mm/memremap.c                   | 30 +++++++++++++++++++++++++++++-
>>   8 files changed, 70 insertions(+), 6 deletions(-)
>>