[LSF/MM TOPIC] Fine-grained Data Access Pattern and Memory Management Optimizations

[SeongJae Park]

From: SeongJae Park <sjpark@xxxxxxxxx>

DAMON and DAMOS
---------------

DAMON[1] is a framework for general data access monitoring of kernel
subsystems.  It provides best-effort high quality monitoring results while
incurring only minimal and upper-bounded overhead, due to its practical
overhead-accuracy tradeoff mechanism.  On a production machine utilizing 70 GB
memory, it can repeatedly scan accesses to the whole memory for every 5ms,
while consuming only 1% single CPU time.

On top of it, a data access pattern-oriented memory management engine called
DAMON-based Operation Schemes (DAMOS) is implemented.  It allows clients to
implement their access pattern oriented memory management logic with very
simple scheme descriptions.  We implemented fine-grained access-aware THP and
proactive reclamation using this engine in three lines of scheme and achieved
remarkable improvements[2].

As of this writing (2021-05-28), the code is not in the mainline but available
at its development tree[3], and regularly posted to LKML as patchsets[4,5,6].
Nevertheless, the code has already merged in the public Amazon Linux kernel
trees[7,8], and all Amazon Linux users can use DAMON/DAMOS off the box. We are
also supporting the two latest upstream LTS stable kernels[9,10].

Agenda
------

Now we can improve Linux kernel's memory management using DAMON's fine-grained
access information.  We could do that by modifying existing subsystems to use
DAMON internally, or implementing new ones on top of DAMOS and let users
select.  The two ways would have their own pros and cons.

I am currently working for implementation of another practical and lightweight
proactive reclamation in the latter way, because this is a new subsystem.  I'd
like to further extend this work to other parts of the kernel including THP,
NUMA and tiered-memory balancing, LRU list arrangement, and KVM, though I don't
have concrete idea as of now but only shallow thought.

I'd like to briefly introduce DAMON/DAMOS, my progress and plans, and
discuss with people for
- thoughts on the programming interface of DAMON/DAMOS,
- pros and cons of the two approaches for this kind of work,
- how DAMON/DAMOS could collaborate with other similar works,
- concerns regarding my planned future works, and
- what additional DAMON/DAMOS-based optimizations would be possible.


[1] https://damonitor.github.io (https://damonitor.github.io/)
[2] https://damonitor.github.io/doc/html/latest/vm/damon/eval.html
[3] https://https://github.com/sjp38/linux/tree/damon/master (https://https//github.com/sjp38/linux/tree/damon/master)
[4] https://lore.kernel.org/linux-mm/20210520075629.4332-1-sj38.park@xxxxxxxxx/
[5] https://lore.kernel.org/linux-mm/20201216084404.23183-1-sjpark@xxxxxxxxxx/
[6] https://lore.kernel.org/linux-mm/20201216094221.11898-1-sjpark@xxxxxxxxxx/
[7] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon
[8] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damon
[9] https://github.com/sjp38/linux/tree/damon/for-v5.4.y
[10] https://github.com/sjp38/linux/tree/damon/for-v5.10.y