Re: breaking up the pagemap_lru_lock (was in rmap, now everywhere ;-) )

From: Martin J. Bligh (Martin.Bligh@us.ibm.com)
Date: Sat Mar 09 2002 - 00:20:31 EST


> but anyways, can you show where do you see this high contenction on the
> pagemap_lru lock? Maybe that's more a sympthom that the rmap is doing
> something silly with the lock acquired, can you measure high contention
> also on my tree on similar workloads? I think we should worry about the
> pagecache_lock, before the pagemap_lru lock.

OK, we've now got some of the other problems on NUMA-Q fixed,
so I can see lock contention better. This is running on a
16-way NUMA-Q, which now has proper page_local allocation
working, compiling a kernel with a VM that's based on the
standard 2.4.18 (I haven't tried the latest -aa tree stuff
yet - if there's any particular part that might help this,
please let me know)

 20.2% 57.1% 5.4us( 86us) 111us( 16ms)(14.7%) 1014988 42.9% 57.1% 0% pagemap_lru_lock
 0.06% 17.2% 4.4us( 46us) 73us(2511us)(0.01%) 3751 82.8% 17.2% 0% activate_page+0xc
 10.4% 55.9% 5.6us( 86us) 123us( 16ms)( 8.1%) 507541 44.1% 55.9% 0% lru_cache_add+0x1c
  9.6% 58.5% 5.2us( 84us) 98us( 12ms)( 6.7%) 503696 41.5% 58.5% 0% lru_cache_del+0xc

Contention is bad, max hold time is good, max wait time is bad.
Looks like it's getting bounced around inside one node - I'll
try John Stultz's msclock stuff on this to fix that, but this
lock's still getting hammered, and could do with some medical
aid ;-) I have something to break it up per zone for rmap, but
I think your tree has a global lru, if I'm not mistaken?
Not sure what we can do about that one.

Pagecache_lock doesn't look good either, but is less of a problem:

 17.5% 31.3% 7.5us( 99us) 52us(4023us)( 2.4%) 631988 68.7% 31.3% 0% pagecache_lock
 13.5% 33.2% 7.7us( 82us) 53us(3663us)( 1.9%) 475603 66.8% 33.2% 0% __find_get_page+0x18
 0.12% 14.7% 2.8us( 54us) 43us(1714us)(0.02%) 11570 85.3% 14.7% 0% __find_lock_page+0x10
 0.10% 10.7% 6.6us( 58us) 43us( 354us)(0.00%) 4270 89.3% 10.7% 0% add_to_page_cache_unique+0x18
  3.8% 28.1% 8.0us( 99us) 52us(4023us)(0.43%) 127606 71.9% 28.1% 0% do_generic_file_read+0x1a4
 0.00% 16.9% 8.1us( 34us) 42us( 108us)(0.00%) 65 83.1% 16.9% 0% do_generic_file_read+0x370
 0.02% 16.1% 0.9us( 20us) 20us( 161us)(0.00%) 5580 83.9% 16.1% 0% filemap_fdatasync+0x20
 0.02% 16.6% 0.9us( 21us) 20us( 145us)(0.00%) 5580 83.4% 16.6% 0% filemap_fdatawait+0x14
 0.00% 1.7% 1.0us( 13us) 19us( 26us)(0.00%) 173 98.3% 1.7% 0% find_or_create_page+0x44
 0.00% 1.7% 3.0us( 45us) 41us( 89us)(0.00%) 173 98.3% 1.7% 0% find_or_create_page+0x88
 0.00% 2.4% 1.9us( 25us) 47us( 211us)(0.00%) 663 97.6% 2.4% 0% remove_inode_page+0x18
 0.00% 16.7% 4.0us( 22us) 108us( 306us)(0.00%) 42 83.3% 16.7% 0% truncate_inode_pages+0x3c
 0.00% 2.6% 0.8us( 30us) 80us( 864us)(0.00%) 663 97.4% 2.6% 0% truncate_list_pages+0x204

None of the hold times look too bad, but the lock's still getting
hammered. Would be nice to break this up per zone/node.

dcache_lock is also a pain. I'm working with Hanna's stuff on that,
and making good progress. Still some way to go.

BKL is a pain in the rear, as usual ;-)

M.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Fri Mar 15 2002 - 22:00:12 EST