Re: [PATCH] arm64: add NUMA emulation support

[Shuah Khan]

On 09/11/2018 03:11 AM, Michal Hocko wrote:
> On Mon 10-09-18 20:02:05, Shuah Khan wrote:
>> Hi Michal,
>>
>> On 09/10/2018 07:48 AM, Michal Hocko wrote:
>>> On Fri 07-09-18 16:30:59, Shuah Khan wrote:
>>>> On 09/07/2018 02:34 AM, Michal Hocko wrote:
>>>>> On Thu 06-09-18 15:53:34, Shuah Khan wrote:
>> [....]
>>>>
>>>> In addition to isolation, being able to reserve a block instead is one of the
>>>> issues I am looking to address. Unfortunately memory cgroups won't address that
>>>> issue.
>>>
>>> Could you be more specific why you need reservations other than
>>> isolation.
>>>
>>
>> Taking automotive as a specific example, there are two classes of applications:
>> 1. critical applications that must run
>> 2. Infotainment and misc. user-space.
>>
>> In this case, being able to reserve a block of memory for critical applications
>> will ensure the memory is available for them. If a critical application has to
>> restart and/or when an on-demand critical application starts, it might not be able
>> to allocate memory if it is not reserved.
>>
>> When a flat system has multiple memory blocks, with NUMA emulation in conjunction with
>> cpusets, one or more block can be reserved for critical applications configuring a set
>> of cpus and one of more memory nodes for them.
>>
>> Memory cgroups will not support such reservation. Hope this helps explain the use-case
>> I am trying to address with this patch.
> 
> OK, that is more clear. I still believe that you either have to have a
> very good control over memory allocations or a good luck to not see
> unexpected kernel allocations in your reserved memory which might easily
> break guarantees you would like to accomplish.
> 

Thanks. Right. I am with you on the possibility that root cgroup can eat into
the reserved memory. However, with this solution I proposed, there is a guarantee
that the cpuset cgroup that is configured for non-critical Infotainment and misc.
user-space application will not be able to allocate from the reserved memory node.

I am hoping the proposed patch will allow critical apps. reserving memory with the
exception that root cgroup and kernel can still allocate from it when needed. Perhaps
cpuset exclusive logic could be extended to look for non-exclusive memory nodes first
if it doesn't already do that. This is inline with the current cpuset approach is that
the critical kernel allocations aren't starved to ensure memory reservations.

If you don't think this solution isn't ideal/good, do you have other suggestions
for solving the problem? If not would it be okay to start with what I proposed and
build on top of as needed?

thanks,
-- Shuah