"time make -j32 bzImage" is now down to 23 seconds.
(16 way NUMA-Q, 700MHz P3's, 4Gb RAM).
Below is a description of which patches helped get there.
Start (2.4.18)
47s
{make NUMA local memory allocation work}
memalloc-15setup (Pat Gaughen)
memalloc-16discont (Pat Gaughen)
pageallocnull fix + force CONFIG_NUMA (Martin Bligh)
27s
{O(1) scheduler}
sched-O1-2.4.18-pre8-K3.patch (Ingo Molnar)
25s
{NUMA scheduler}
numaK3.patch (Mike Kravetz)
24s
{dcache cacheline bouncing fixes}
dcache/fast_walkA2-2.4.18.patch (Hanna Linder)
23s
Appling Ingo's patch alone took time from 47s to 30s.
The benefits after the local mem stuff aren't quite as
stunning, but still good.
Top 10 profile hitters left:
21439 total 0.0228
9112 default_idle 175.2308
3364 _text_lock_swap 62.2963
790 lru_cache_add 8.5870
750 _text_lock_namei 0.7184
587 do_anonymous_page 1.7681
572 lru_cache_del 26.0000
569 do_generic_file_read 0.5117
510 __free_pages_ok 0.9733
421 _text_lock_dec_and_lock 17.5417
318 _text_lock_read_write 2.6949
Big locks left:
pagemap_lru_lock
20.2% 57.1% 5.4us( 86us) 111us( 16ms)(14.7%) 1014988 42.9% 57.1% 0%
pagecache_lock
17.5% 31.3% 7.5us( 99us) 52us(4023us)( 2.4%) 631988 68.7% 31.3% 0%
others:
dcache_lock (much improved, but still work to be done)
BKL (isn't it always ;-)
Planned work next:
1. Try John Stultz's mcslocks
(note high max wait vs low max hold currently)
2. Try rmap + pagemap_lru_breakup from Arjan
3. Try radix tree pagecache.
4. Try grafting NUMA-Q page local alloc onto -aa tree
5. Try SGI NUMA zone ordering stuff.
6. [HARD] Break up ZONE_NORMAL between nodes
(all currently on node 0).
Any other suggestions are welcome. I'd also be interested
to know if 23s is fast for make bzImage, or if other big
iron machines can kick this around the room.
Thanks,
Martin.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Fri Mar 15 2002 - 22:00:11 EST