Re: [LSF/MM/BPF TOPIC] Towards Unified and Extensible Memory Reclaim (reclaim_ext)

[Michal Hocko]

On Wed 25-03-26 14:06:37, Shakeel Butt wrote:
> The Problem
> -----------
> 
> Memory reclaim in the kernel is a mess. We ship two completely separate
> eviction algorithms -- traditional LRU and MGLRU -- in the same file.
> mm/vmscan.c is over 8,000 lines. 40% of it is MGLRU-specific code that
> duplicates functionality already present in the traditional path. Every
> bug fix, every optimization, every feature has to be done twice or it
> only works for half the users. This is not sustainable. It has to stop.

While I do agree that having 2 implementations available and to maintain
them is not long term sustainable I would disagree with your above line
of argumentation. We are not aiming to have the two in feature parity
nor they are overlapping in bug space all that much.

> We should unify both algorithms into a single code path. In this path,
> both algorithms are a set of hooks called from that path.

Isn't this the case from a large part? MGRLU tends to have couple of
entry points in the shared code base (node/memcg scanning code).

> Everyone
> maintains, understands, and evolves a single codebase. Optimizations are
> now evaluated against -- and available to -- both algorithms. And the
> next time someone develops a new LRU algorithm, they can do so in a way
> that does not add churn to existing code.

I think we should focus to make a single canonical reclaim
implementation work well. I.e. we deal with most (or ideally all) known
regressions of MGLRU. In the initial presentation of the MGRLU framework
we were told that the implemenation should be extensible to provide more
creative aging algorithms etc.

> The Fix: One Reclaim, Pluggable and Extensible
> -----------------------------------------------
> 
> We need one reclaim system, not two. One code path that everyone
> maintains, everyone tests, and everyone benefits from. But it needs to
> be pluggable as there will always be cases where someone wants some
> customization for their specialized workload or wants to explore some
> new techniques/ideas, and we do not want to get into the current mess
> again.

I would go that way only if/after we are done with MGLRU unification and
after we will have depleted the potential of that approach and hit cases
where we cannot implement new extensions without going $foo_ext way. TBH
I am not convinced "make it pluginable to solve hard problems" is the
best way forward.

-- 
Michal Hocko
SUSE Labs