Re: [RFC][PATCH 0/4] Memory controller soft limit patches (v2)

[Balbir Singh]

* KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> [2009-02-17 13:03:52]:

> On Tue, 17 Feb 2009 08:35:26 +0530
> Balbir Singh <balbir@xxxxxxxxxxxxxxxxxx> wrote:
> 
> > * KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> [2009-02-17 09:05:23]:
> > 
> > > On Mon, 16 Feb 2009 16:38:44 +0530
> > > Balbir Singh <balbir@xxxxxxxxxxxxxxxxxx> wrote:
> > > 
> > > > 
> > > > From: Balbir Singh <balbir@xxxxxxxxxxxxxxxxxx>
> > > > 
> > > > Changelog v2...v1
> > > > 1. Soft limits now support hierarchies
> > > > 2. Use spinlocks instead of mutexes for synchronization of the RB tree
> > > > 
> > > > Here is v2 of the new soft limit implementation. Soft limits is a new feature
> > > > for the memory resource controller, something similar has existed in the
> > > > group scheduler in the form of shares. The CPU controllers interpretation
> > > > of shares is very different though. We'll compare shares and soft limits
> > > > below.
> > > > 
> > > > Soft limits are the most useful feature to have for environments where
> > > > the administrator wants to overcommit the system, such that only on memory
> > > > contention do the limits become active. The current soft limits implementation
> > > > provides a soft_limit_in_bytes interface for the memory controller and not
> > > > for memory+swap controller. The implementation maintains an RB-Tree of groups
> > > > that exceed their soft limit and starts reclaiming from the group that
> > > > exceeds this limit by the maximum amount.
> > > > 
> > > > This is an RFC implementation and is not meant for inclusion
> > > > 
> > > 
> > > some thoughts after reading patch.
> > > 
> > > 1. As I pointed out, cpuset/mempolicy case is not handled yet.
> > 
> > That should be esy to do with zonelists passed from reclaim path
> >
> plz do.
> 
> > 
> > > 2. I don't like to change usual direct-memory-reclaim path. It will be obstacles
> > >    for VM-maintaners to improve memory reclaim. memcg's LRU is designed for
> > >    shrinking memory usage and not for avoiding memory shortage. IOW, it's slow routine
> > >    for reclaiming memory for memory shortage.
> > 
> > I don't think I agree here. Direct reclaim is the first indication of
> > shortage and if order 0 pages are short, memcg's above their soft
> > limit can be targetted first.
> > 
> My "slow" means "the overhead seems to be big". The latency will increase.
> 
> About 0-order
> In patch 4/4
> +	did_some_progress = mem_cgroup_soft_limit_reclaim(gfp_mask);
> +	/*
> should be
>         if (!order)
>             did_some_progress = mem....
> 

OK, will do

> 
> I don't want to add any new big burden to kernel hackers of memory management,
> they work hard to improve memory reclaim. This patch will change the behavior.
> 

I don't think I agree, this approach suggests that before doing global
reclaim, there are several groups that are using more than their
share of memory, so it makes sense to reclaim from them first.


> BTW, in typical bad case, several threads on cpus goes into memory recalim at once and
> all thread will visit this memcg's soft-limit tree at once and soft-limit will
> not work as desired anyway.
> You can't avoid this problem at alloc_page() hot-path.

Even if all threads go into soft-reclaim at once, the tree will become
empty after a point and we will just return saying there are no more
memcg's to reclaim from (we remove the memcg from the tree when
reclaiming), then those threads will go into regular reclaim if there
is still memory pressure.

> 
> > > 3. After this patch, res_counter is no longer for general purpose res_counter...
> > >    It seems to have too many unnecessary accessories for general purpose.  
> > 
> > Why not? Soft limits are a feature of any controller. The return of
> > highest ancestor might be the only policy we impose right now. But as
> > new controllers start using res_counter, we can clearly add a policy
> > callback.
> > 
> I think you forget that memroy cgroups is an only controller in which the kernel
> can reduce the usage of resource without any harmful to users.
> soft-limit is nonsense for general resources, I think.
> 

Really? Even for CPUs? soft-limit is a form of shares (please don't
confuse with cpu.shares). Soft limits is used as a way of implementing
work conserving controllers.

-- 
	Balbir
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/