Re: [PATCH][1/2] adjust dirty threshold for lowmem-only mappings

[Rik van Riel]

On Sat, 25 Dec 2004, Andrea Arcangeli wrote:
> On Fri, Dec 24, 2004 at 05:12:32PM -0500, Rik van Riel wrote:
> > The process 'dd', and all the other processes, live in
> > the highmem zone, which has 2.5GB of memory free. Now
> > tell me again why you think the swap token has any
> > relevance to those 950MB of pagecache that is filling
> > up lowmem ?
>
> If 2.5G of ram is really free, then how can the oom killer be invoked in
> the first place? If that happens it means you're under a lowmem
> shortage, something you apparently ruled out when you said
> lowmem_reserve couldn't help your workload.

Let me explain a 3rd time:

1) run dd if=/dev/zero of=/dev/hdaN on a system with 4GB RAM

2) the pagecache mapping for /dev/hdaN can only come from
   lowmem, of which we have roughly 900MB

3) the dirty_limit is 40% of 4GB, or roughly 1.6GB - the dd
   from (1) will not throttle itself at all, but will just
   fill up lowmem without limitation

4) any memory that could be affected by the swap token (process
   text, data, stack, ...) is allocated with __GFP_HIGHMEM, so
   that all lives in the highmem zone with 2.5GB free

5) since dd is not being paged out at all, and can dirty memory
   without limit, the VM gets backed into a corner and will
   trigger an OOM kill - even though most of lowmem is simply
   dirty page cache

6) an unpatched 2.6.10-rc kernel will OOM kill in minutes on
   a test system here

6) Andrew's total_pages patch, marcelo's vm-writeout-throttle patch
   and my two patches improve the situation a lot, and the OOM kill
   takes a day or so to be triggered

If you have any more questions as to why the bug happens, don't
hesitate to ask and I'll explain you why this problem happens.

--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/