Re: [PATCH 0/3] x86/platform/UV: Update Memory Block Size Setting

From: Mike Travis
Date: Tue May 15 2018 - 13:41:07 EST


[re-including everyone for any other comments about these patches...]

On 5/10/2018 11:48 PM, Michal Hocko wrote:
On Thu 10-05-18 18:18:32, mike.travis@xxxxxxx wrote:

Update support for the UV kernel to accommodate Intel BIOS changes in
NVDIMM alignment, which caused UV BIOS to align the memory boundaries
on different blocks than the previous UV standard of 2GB.

Please elaborate (much) more. What is the actual problem and how is the
patchset addressing it.


On 5/15/2018 1:55 AM, Michal Hocko wrote:
> On Fri 11-05-18 08:08:14, Mike Travis wrote:
> [...]
>> If you think I need this more detailed explanation in the patch descriptions
>> themselves, I'll add it.
>
> Yes we definitely want this information along with a high level
> description of how this got fixed. Incomplete memblocks need to be
> handled gracefully (especially when blocks are 2GB in size).
>
> Thanks!
>

Hi Michal,

I will add more info but this patch does not address anything about incomplete memblocks. They have existed in 2GB mem block size form since 2009 (v2.6) with the first UV1 system release. I am not changing any of that handling.

The fixing part was to adapt to what Intel BIOS was using as a different boundary to align the PMEM NVDIMMs. This is explained in both patch 1 and patch 2:

"Add a new function to "adjust" the current fixed UV memory block size of 2GB so it can be changed to a different physical boundary. This is out of necessity so UV BIOS can accommodate Intel BIOS changes for NVDIMM's, which can align these new PMEM modules at other than 2GB boundaries."

"Add a call to the new function to "adjust" the current fixed UV memory block size of 2GB so it can be changed to a different physical boundary. This accommodates changes in the Intel BIOS, and therefore UV BIOS, which now can align boundaries different than the previous UV standard of 2GB. It also flags any UV Mem boundaries that cause a change in the mem block size boundary."

I can't explain why the Intel BIOS changed the boundaries, it probably has something to do with accommodating other areas of the NVDIMMs physical address space. This caused a boundary alignment to be less than 2GB which UV had been using. From one of the UV BIOS engineers I found out that Intel really only guarantees a 64MB boundary (which is actually less than the current Linux minimum of 128MB.) Our own UVHUB requirements dictated that 2GB was an acceptable boundary up until now.

So let me know how much more info is needed on how I need to explain _this_ change.

Thanks,
Mike