Re: New dcache not using slab allocator?

Kevin Buhr (buhr@stat.wisc.edu)
04 Aug 1997 14:48:08 -0500


Erik Andersen <andersee@debian.org> writes:
>
> Umm. kmalloc()/kfree() are now built on top of the slab allocator.
> Take a look at linux/mm/slab.c and you will find these nice functions.
> That means nobody has to do anything to take advantage of slab.
> If you have used kmalloc in the kernel, you are already using slab.

Not really. It's worthwhile reading the Usenix article referenced in
"linux/mm/slab.c", since it explains some of the rationale for having
a slab allocator in the first place. Just because the kmalloc/kfree
functions use the underlying slab allocator does *not* mean they give
you all the benefits of having a dedicated slab allocator for your
objects.

"dcache" entries are an ideal example. They are allocated and freed
en mass by the kernel at a tremendous rate. Because of the widely
varying lifetimes of dcache entries, this can lead to horrible
fragmentation of the kmalloc pool (with one or more long-lived dcache
entries tying up almost every available page), and it's probably
behind the reports of failed "fork" calls that have cropped up over
2.1.46 and 2.1.47.

Having dcache entries in their own slab cache helps out here. They
are effectively isolated from other memory objects, so the number of
pages that can be tied up by long-lived entries is greatly reduced.

I, for one, have patched "dcache.c" to go through the slab allocator
using a special "dentry" cache, and my previous "fork" out-of-memory
and swap-attack problems have disappeared. It's not a complete
solution, since the filenames (allocated via kmalloc) are still
capable of causing fragmentation, but that's a more difficult patch.

For those interested, here's my patch-in-progress against vanilla 2.1.47.

Kevin <buhr@stat.wisc.edu>

* * *

Index: linux/fs/dcache.c
diff -u linux/fs/dcache.c:1.1.1.2 linux/fs/dcache.c:1.2
--- linux/fs/dcache.c:1.1.1.2 Tue Jul 29 18:21:47 1997
+++ linux/fs/dcache.c Sun Aug 3 12:47:47 1997
@@ -17,6 +17,7 @@
#include <linux/mm.h>
#include <linux/fs.h>
#include <linux/malloc.h>
+#include <linux/slab.h>
#include <linux/init.h>

/*
@@ -34,10 +35,12 @@
static struct list_head dentry_hashtable[D_HASHSIZE];
static LIST_HEAD(dentry_unused);

+static kmem_cache_t *de_cachep;
+
void d_free(struct dentry *dentry)
{
kfree(dentry->d_name.name);
- kfree(dentry);
+ kmem_cache_free(de_cachep, dentry);
}

/*
@@ -133,13 +136,13 @@
char * str;
struct dentry *dentry;

- dentry = kmalloc(sizeof(struct dentry), GFP_KERNEL);
+ dentry = kmem_cache_alloc(de_cachep, SLAB_KERNEL);
if (!dentry)
return NULL;

str = kmalloc(NAME_ALLOC_LEN(name->len), GFP_KERNEL);
if (!str) {
- kfree(dentry);
+ kmem_cache_free(de_cachep, dentry);
return NULL;
}

@@ -387,4 +390,10 @@
d++;
i--;
} while (i);
+ de_cachep = kmem_cache_create("dentry",
+ sizeof(struct dentry),
+ 0,
+ SLAB_HWCACHE_ALIGN, NULL, NULL);
+ if(!de_cachep)
+ panic("Cannot create dentry SLAB cache\n");
}
Index: linux/mm/vmscan.c
diff -u linux/mm/vmscan.c:1.1.1.2 linux/mm/vmscan.c:1.4
--- linux/mm/vmscan.c:1.1.1.2 Tue Jul 29 18:27:22 1997
+++ linux/mm/vmscan.c Sun Aug 3 12:49:52 1997
@@ -365,13 +365,7 @@
shrink_dcache();
state = 2;
case 2:
- /*
- * We shouldn't have a priority here:
- * If we're low on memory we should
- * unconditionally throw away _all_
- * kmalloc caches!
- */
- if (kmem_cache_reap(0, dma, wait))
+ if (kmem_cache_reap(i, dma, wait))
return 1;
state = 3;
case 3:
@@ -385,6 +379,10 @@
i--;
} while ((i - stop) >= 0);
}
+#if 0
+ printk("do_try_to_free_page: failed pri=%d dma=%d wait=%d\n",
+ priority, dma, wait);
+#endif
return 0;
}

@@ -401,6 +399,11 @@

lock_kernel();
retval = do_try_to_free_page(priority,dma,wait);
+#if 0
+ if (!retval)
+ printk("try_to_free_page: failed pri=%d dma=%d wait=%d\n",
+ priority, dma, wait);
+#endif
unlock_kernel();
return retval;
}