Re: Horrible L2 cache effects from kernel compile

From: William Lee Irwin III (wli@holomorphy.com)
Date: Mon Feb 24 2003 - 19:59:22 EST


On Mon, Feb 24, 2003 at 04:41:37PM -0800, Dave Hansen wrote:
> c01cc948 334779 8.82177 __copy_to_user_ll
> c015797c 314165 8.27857 d_lookup
> c012c964 157716 4.15598 find_get_page
> c013cae0 156345 4.11985 page_add_rmap
> c0176557 148116 3.90301 .text.lock.inode
> c013cc38 132228 3.48434 page_remove_rmap
> c0176338 90851 2.39402 ext3_dirty_inode
> c0117320 80918 2.13227 scheduler_tick
> c01ccae0 79475 2.09425 atomic_dec_and_lock
> c0138cd4 75367 1.986 do_no_page
> c0139b60 75317 1.98468 vm_enough_memory
> c017d78c 72929 1.92175 start_this_handle

Your profile was upside down. I've re-sorted it.
It probably confused people who were wondering why the numbers
at the top of the profile were lower than the ones below them.

On Mon, Feb 24, 2003 at 04:41:37PM -0800, Dave Hansen wrote:
> a snippet from d_lookup(), annotated. I've seen oprofile be off by a
> line here, but we can be pretty sure it is in this area.
> smp_read_barrier_depends();
> /* 106 0.002793% */
> dentry = list_entry(tmp, struct dentry, d_hash);
> /* if lookup ends up in a different bucket
> * due to concurrent rename, fail it
> */
> /* 154991 4.084% */
> if (unlikely(dentry->d_bucket != head))
> break;
> /* to avoid race if dentry keep coming back to original
> * bucket due to double moves
> */
> /* 634 0.01671% */
> if (unlikely(++lookup_count > max_dentries))
> break;
> ...

Well, you're walking hash chains, you're going to get a lot of cache
misses, about the whole length of the chain's worth. You could try
changing the dcache to use something like a B+ tree or some such
nonsense to reduce its cache footprint.

I'd collect statistics to see if the dcache hash function is badly
distributing things. That's relatively easily fixable if so.

-- wli
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Feb 28 2003 - 22:00:24 EST