Re: [PATCH v2 1/3] sched/numa: make free-space tiering promotion bypass configurable

[Bharata B Rao]

On 29-Oct-26 3:16 PM, Li Zhe wrote:
> [You don't often get email from lizhe.67@xxxxxxxxxxxxx. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
> 
> On 10/29/26 5:34 PM, Bharata B Rao wrote:
>> On 28-Oct-26 9:42 AM, Li Zhe wrote:
>>> Memory tiering normally promotes any recently accessed slow-tier page if
>>> the target fast-tier node has enough free space. This preserves the
>>> existing policy of filling available fast memory before relying on
>>> hotness and rate-limit control.
>>>
>>> That is not always the desired policy. Some deployments may intentionally
>>> keep part of a workload's memory on CXL-attached memory, for example to
>>> preserve a desired DRAM/CXL capacity split or bandwidth distribution. With
>>> the free-space bypass, short cold accesses and bursts of hot accesses can
>>> refill the fast tier and pull the workload away from that distribution
>>> whenever DRAM has free space.
>>>
>>> Redis testing exercises this policy issue. On a CXL system with node0 as
>>> DRAM and node3 as slow-tier memory, a Redis test was run with
>>> memory-tiering-only mode enabled. The test used a Redis dataset with 12
>>> million 4KB values. It reclaimed memory until about 85% of Redis
>>> anonymous memory was on node3, used a Redis benchmark configuration with
>>> a 10% hot-key working set, and then issued a short 256MB access burst to
>>> cold keys outside that working set. The cold-key burst mainly touches
>>> memory outside the hot working set, which is expected to remain mostly on
>>> the slow tier.
>> Can you please clarify if 85% of redis memory ended up on lower tier naturally
>> due to overflow from toptier? I ask this question because, if it is a natural
>> overflow/demotion, then it will be hard to find head room in the toptier to
>> allow promotions that bypass hotness or rate limit checks."
> 
> 
> It was not a natural overflow from a full top tier.

Thanks for confirming this.

> 
> The test intentionally reclaimed/demoted Redis memory to the lower tier
> to model a deployment that keeps only a limited fraction of the workload
> in DRAM and leaves the rest on CXL memory.  In that setup the top tier can
> still have enough free space, so the current free-space bypass can promote
> short cold accesses back to DRAM.
With pghot, I have been providing benchmark numbers from two scenarios:

1. Non-overcommitted case: The workload memory is pushed to lower tier before
the measurement phase begins, the toptier has enough memory to accept the
the incoming promotions.

2. Overcommitted case: The workload memory naturally overflows/demotes into
the lower tier and every promotion undergoes the hotness + ratelimiting check.

I gather that your measurement technique is similar to case 1 above.

The feedback I have received for pghot is this: Non-overcommitted case is not
the right methodology to use but the natural spillover is. Hence I am not sure
if optimizing the free-space bypass case would benefit any naturally run
cases.

Regards,
Bharata.