2.6.35-rc3: Load average climbing to 3+ with no apparent reason:CPU 98% idle, with hardly no I/O

[TÃrÃk Edwin]

Hi,

I just noticed that my load average is 2.99 and climbing (it is 3.11
right now).
CPU is 98% idle, with hardly any I/O at all so I don't know what is
causing this:
 10:32:55 up  1:01,  5 users,  load average: 3.28, 3.31, 3.09

$ vmstat 5
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0 492412 490320 1716264    0    0   122    79  331  419  2  1 93  4
 0  0      0 492388 490320 1716264    0    0     0    13  755  983  0  1 99  0
 0  0      0 492632 490324 1716040    0    0     1    71 1013 1455  1  1 98  0
 1  0      0 492132 490340 1716264    0    0     4  1651  947 1223  2  1 96  1
 0  0      0 491972 490340 1716272    0    0     0    69 1122 1586  2  2 96  0
 0  0      0 491788 490340 1716272    0    0     0    41 1527 2517  3  2 95  0
 0  0      0 491884 490340 1716272    0    0     0   107 1419 2193  2
 1 97  0

This happens with 2.6.35-rc3-00001-g6bdebf9 (where the -00001 patch is
this bugfix required for networking to work at all: "net: fix deliver_no_wcard regression on loopback device")

I have attached the output of cfs-debug-info.sh: cfs-debug-info-2010.07.01-10.29.57.gz

I don't see anything special in dmesg, just the continous reset of ata9 (CDROM) that I reported about already:
http://lkml.org/lkml/2010/6/27/83
Could this cause load average calculation to go wrong?

[ 1022.445484] ata9: soft resetting link
[ 1022.640654] ata9.00: configured for UDMA/33
[ 1022.640739] ata9.01: failed to set xfermode (err_mask=0x100)
[ 1023.439682] irq 19: nobody cared (try booting with the "irqpoll"
option)
[ 1023.439687] Pid: 0, comm: swapper Not tainted
2.6.35-rc3-00001-g6bdebf9 #82
[ 1023.439689] Call Trace:
[ 1023.439691]  <IRQ>  [<ffffffff810871d2>] __report_bad_irq+0x38/0x87
[ 1023.439700]  [<ffffffff81087334>] note_interrupt+0x113/0x177
[ 1023.439703]  [<ffffffff81087ab3>] handle_fasteoi_irq+0xa8/0xd2
[ 1023.439707]  [<ffffffff81004d97>] handle_irq+0x1f/0x28
[ 1023.439710]  [<ffffffff810043ba>] do_IRQ+0x57/0xbe
[ 1023.439713]  [<ffffffff813ce313>] ret_from_intr+0x0/0xa
[ 1023.439715]  <EOI>  [<ffffffffa001e25b>] ? acpi_safe_halt+0x24/0x37
[processor]
[ 1023.439733]  [<ffffffffa001e34d>] acpi_idle_enter_c1+0x71/0xc8
[processor]
[ 1023.439737]  [<ffffffff81339d85>] ? menu_select+0x163/0x276
[ 1023.439745]  [<ffffffff81338ead>] cpuidle_idle_call+0x91/0x11c
[ 1023.439747]  [<ffffffff81001cbd>] cpu_idle+0x5c/0x9c
[ 1023.439750]  [<ffffffff813bb081>] rest_init+0xb5/0xba
[ 1023.439753]  [<ffffffff81787d00>] start_kernel+0x385/0x390
[ 1023.439755]  [<ffffffff817872a3>] x86_64_start_reservations+0xb3/0xb7
[ 1023.439758]  [<ffffffff8178738b>] x86_64_start_kernel+0xe4/0xeb
[ 1023.439759] handlers:
[ 1023.439760] [<ffffffff812f006f>] (ata_bmdma_interrupt+0x0/0x1ab)
[ 1023.439764] [<ffffffffa00562ae>] (usb_hcd_irq+0x0/0x60 [usbcore])
[ 1023.439773] Disabling IRQ #19
[ 1027.601538] ata9: soft resetting link
[ 1027.774161] ata9.00: configured for UDMA/33
[ 1027.789039] ata9.01: configured for UDMA/33
[ 1028.248557] ata9: EH complete

Best regards,
--Edwin

Attachment: cfs-debug-info-2010.07.01-10.29.57.gz
Description: GNU Zip compressed data