In between 2.5.56 and 2.5.57 IO speeds have dropped if CONFIG_PREEMPT is
set.
The machine is a 4 processor Xeon with 8GB of RAM. The external storage is
attached to the machine via qlogic 2200 HBAs using the qlogicfc driver.
The filesystems on the external devices is reiserfs, mounted with noatime
as the only change from defaults. The config file is attached below. The
same config was used for both 2.5.56 and 2.5.57. The config has
preemptable kernel enabled. I know preemption will lower your throughput,
but it was enabled in 2.5.56, and speed was good.
The test involved the simultaneous writing to all 8 filesystems on the
external storage. On 2.5.56, the vmstat output is below
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy
id wa
8 1 0 7108 87280 7505164 0 0 20 2989 144 56 0 21
77 1
8 2 0 8444 85376 7505440 0 0 0 115536 1387 2391 0 93
1 5
9 2 0 7684 89364 7502132 0 0 0 112348 1455 2308 0 94
1 5
4 2 0 7072 94704 7495600 0 0 0 129468 1382 4314 0 51
32 18
For 2.5.57, the io bo rate drops, and the system time pegs at 100%
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r b swpd free buff cache si so bi bo in cs us sy
id wa
8 1 0 8000 61196 7535276 0 0 36 4158 147 22 1 40
58 1
9 1 0 7468 62316 7533832 0 0 1 79852 1223 303 0 100
0 0
9 1 0 6740 64884 7532368 0 0 0 79655 1288 182 0 100
0 0
9 0 0 7004 66720 7528828 0 0 0 78841 1261 354 0 100
0 0
9 1 0 8272 67604 7526844 0 0 0 77958 1190 204 0 100
0 0
The profiling information for 2.5.57 is below
/usr/sbin/readprofile -m /boot/System.map-2.5.57-test2 -v | sort -nr +2 |
head -15
00000000 total 436997 2.9703
c011bb60 __preempt_spin_lock 293278 3054.9792
c0107020 default_idle 83500 1304.6875
c011bc22 .text.lock.sched 55920 185.1656
c0119fc0 __wake_up 2371 29.6375
c0118960 kmap_atomic 941 7.3516
c0121920 do_softirq 393 1.8894
c01189f0 kmap_atomic_to_page 129 1.1518
c011bb00 __might_sleep 128 1.3333
c0119af0 schedule 126 0.1270
c0113e50 flush_tlb_mm 47 0.3264
c011bbc0 __preempt_write_lock 35 0.3571
c01178c0 do_page_fault 31 0.0251
c01189e0 kunmap_atomic 17 1.0625
c0124cb0 add_timer 15 0.0446
If I recompile 2.5.57 with preemption off, the vmstat rates go back to the
rates similar to the 2.5.56 w/ preemption turned on.
Is this expected behaviour now?
Thanks in advance.
Robert
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Wed Jan 15 2003 - 22:00:55 EST