Re: [PATCH v3 0/3] create sysfs representation of ACPI HMAT

[Anshuman Khandual]

On 12/22/2017 04:01 PM, Kogut, Jaroslaw wrote:
>> ... first thinking about redesigning the NUMA for
>> heterogeneous memory may not be a good idea. Will look into this further.
> I agree with comment that first a direction should be defined how to handle heterogeneous memory system.
> 
>> https://linuxplumbersconf.org/2017/ocw//system/presentations/4656/original/
>> Hierarchical_NUMA_Design_Plumbers_2017.pdf
> I miss in the presentation a user perspective of the new approach, e.g.
> - How does application developer see/understand the heterogeneous memory system?

>From user perspective

- Each memory node (with or without CPU) is a NUMA node with attributes
- User should detect these NUMA nodes from sysfs (not part of proposal)
- User allocates/operates/destroys VMA with new sys calls (_mattr based)

> - How does app developer use the heterogeneous memory system?

- Through existing and new system calls

> - What are modification in API/sys interfaces?

- The presentation has possible addition of new system calls with 'u64
  _mattr' representation for memory attributes which can be used while
  requesting different kinds of memory from the kernel

> 
> In other hand, if we assume that separate memory NUMA node has different memory capabilities/attributes from stand point of particular CPU, it is easy to explain for user how to describe/handle heterogeneous memory. 
> 
> Of course, current numa design is not sufficient in kernel in following areas today:
> - Exposing memory attributes that describe heterogeneous memory system
> - Interfaces to use the heterogeneous memory system, e.g. more sophisticated policies
> - Internal mechanism in memory management, e.g. automigration, maybe something else.

Right, we would need

- Representation of NUMA with attributes
- APIs/syscalls for accessing the intended memory from user space
- Memory management policies and algorithms navigating trough all these
  new attributes in various situations

IMHO, we should not consider sysfs interfaces for heterogeneous memory
(which will be an ABI going forward and hence cannot be changed easily)
before we get the NUMA redesign right.