Re: [RFC][PATCH 4/6] Flat hierarchical reclaim by ID

[Balbir Singh]

* KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> [2008-12-09 23:28:32]:

> Balbir Singh said:
> > * KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> [2008-12-09
> > 20:09:15]:
> >
> >>
> >> From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx>
> >>
> >> Implement hierarchy reclaim by cgroup_id.
> >>
> >> What changes:
> >> 	- Page reclaim is not done by tree-walk algorithm
> >> 	- mem_cgroup->last_schan_child is changed to be ID, not pointer.
> >> 	- no cgroup_lock, done under RCU.
> >> 	- scanning order is just defined by ID's order.
> >> 	  (Scan by round-robin logic.)
> >>
> >> Changelog: v3 -> v4
> >> 	- adjusted to changes in base kernel.
> >> 	- is_acnestor() is moved to other patch.
> >>
> >> Changelog: v2 -> v3
> >> 	- fixed use_hierarchy==0 case
> >>
> >> Changelog: v1 -> v2
> >> 	- make use of css_tryget();
> >> 	- count # of loops rather than remembering position.
> >>
> >> Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxx>
> >
> > I have not yet run the patch, but the heuristics seem a lot like
> > magic. I am not against scanning by order, but is order the right way
> > to scan groups?
> My consideration is
>   - Both of current your implementation and this round robin is just
>     an example. I never think some kind of search algorighm detemined by
>     shape of tree is the best way.
>   - No one knows what order is the best, now. We have to find it.
>   - The best order will be determined by some kind of calculation rather
>     than shape of tree and must pass by tons of tests.

Yes, the shape of the tree just limits where to reclaim from

>     This needs much amount of time and patient work. VM management is not
>     so easy thing.
>     I think your soft-limit idea can be easily merged onto this patch set.
> 

Yes, potentially. With soft limit, the general expectation is this

Let us say you have group A and B

        groupA, soft limit = 1G
        groupB, soft limit = 2G

Now assume the system has 4G. When groupB is not using its memory,
group A can grab all 4G, but when groupB kicks in and tries to use 2G
or more, then the expectation is that

group A will get 1/3 * 4 = 4/3G
group B will get 2/3 * 4 = 8/3G

Similar to CPU shares currently.

> > Does this order reflect their position in the hierarchy?
>   No. just scan IDs from last scannned one in RR.
>   BTW, can you show what an algorithm works well in following case ?
>   ex)
>     groupA/   limit=1G     usage=300M
>           01/ limit=600M   usage=600M
>           02/ limit=700M   usage=70M
>           03/ limit=100M   usage=30M
>    Which one should be shrinked at first and why ?
>    1) when group_A hit limits.

With tree reclaim, reclaim will first reclaim from A and stop if
successful, otherwise it will go to 01, 02 and 03 and then go back to
A.

>    2) when group_A/01 hit limits.

This will reclaim only from 01, since A is under its limit

>    3) when group_A/02 hit limits.

This will reclaim only from 02 since A is under its limit

Does RR do the same right now?

>    I can't now.
> 
>    This patch itself uses round-robin and have no special order.
>    I think implenting good algorithm under this needs some amount of time.
> 

I agree that fine tuning it will require time, but what we need is
something usable that will not have hard to debug or understand corner cases.

> > Shouldn't id's belong to cgroups instead of just memory controller?
> If Paul rejects, I'll move this to memcg. But bio-cgroup people also use
> ID and, in this summer, I posted swap-cgroup-ID patch and asked to
> implement IDs under cgroup rather than subsys. (asked by Paul or you.)
> 

We should talk to Paul and convince him.

> >From implementation, hierarchy code management at el. should go into
> cgroup.c and it gives us clear view rather than implemented under memcg.
> 

cgroup has hierarchy management already, in the form of children and
sibling. Walking those structures is up to us, that is all we do
currently :)

> -Kame
> > I would push back ids to cgroups infrastructure.
> >
> 
> 
> 

-- 
	Balbir
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/