Disk Corruption with ide hpt-366 controller & software raid5

Tom Livingston (tsl@volition.org)
Mon, 13 Sep 1999 01:30:40 -0700


Hello all,

I am encountering reproducible read time errors while reading from my
existing RAID array with at least one disk running on a HPT-366 driven
channel. This manifests itself as random errors... e.g. if you read the
same thing five times, you will get five different answers. I first
encountered this as massive corruption in my superblocks and file systems,
but I have included set of dd results below to make the explanation more
scientific.

As I point out below with the dd tests, this problem does not occur when
accessing the disk on the hpt-366 controller alone, and not through the raid
set /dev/md0. Also, this same raid set "passes" the tests when none of the
disks are connected to the hpt-366.

I am running redhat-6.0+updates, linux kernel 2.2.12, with
raid0145-19990824-2.2.11.gz and 2.2.12.uniform-ide-6.20.hydra.patch.gz Note
that the raid patch is for 2.2.11, however it applies cleanly, minus one
reject in include/linux/fs.h. This patch in the raid code was already
applied to 2.2.12.

Hardware is Abit BP-6 motherboard, single celeron 366, 128MB ram. IDE disk
controllers present: PIIX4 (onboard), three PDC20246 (3 pci cards), HPT-366
(onboard). These controllers run 12 drives, 10 in the raid5 set. All 10
raid disks are Maxtor 9084[05]D[456] (e.g. 90845D4) drives. All related
dmesg type output is included at the bottom.

This array had been running using only the PIIX4 and the PDC20246's, with
two channels hosting two drives in the raid set, the others holding 1
drive/channel. Set up like this, the RAID system has worked fine for
months.

[note that the entire time these tests were run, the raid system was running
in degraded mode, as I had forcibly removed hde1 from the raid array before
these tests began, in order to do a little testing with the hpt-366 before I
committed to moving the active parts of the raid. these tests ran fine]

[root@music /root]# for i in 1 2 3 4 5
> do
> dd if=/dev/md0 count=500000 2> /dev/null | md5sum
> done
48ed7a0be7aac6a7f27be473737ee097 -
48ed7a0be7aac6a7f27be473737ee097 -
48ed7a0be7aac6a7f27be473737ee097 -
48ed7a0be7aac6a7f27be473737ee097 -
48ed7a0be7aac6a7f27be473737ee097 -

However, when a disk from the existing raid set is moved from a PDC channel
to an HPT channel, dd shows me reading unreliably from the /dev/md0 set:

[root@music /root]# for i in 1 2 3 4 5
> do
> dd if=/dev/md0 count=500000 2> /dev/null | md5sum
> done
08d3b2b34dfc667ca96c549f8a8a3c15 -
cee7aa5dd1ee81ff63a93bba3830ca31 -
a577d2d50f9ebc535b9e49905c29631c -
f8c6aea89094543aaf2982ef6504285d -
596f99e3047d18eef9798634a091670b -

Interestingly, this does not manifest itself if you read directly from the
lone drive on the hpt-366, ignoring the raid set:

[root@music /root]# for i in 1 2 3 4 5
> do
> dd if=/dev/hdq1 count=500000 2> /dev/null | md5sum
> done
eada67dc62f41de461b96f0daac0983d -
eada67dc62f41de461b96f0daac0983d -
eada67dc62f41de461b96f0daac0983d -
eada67dc62f41de461b96f0daac0983d -
eada67dc62f41de461b96f0daac0983d -

The amount of corruption, seems to be minimal. I tested one 300000 block
sample twice, and found that only 82 bytes were different:

[root@music /root]# dd if=/dev/md0 count=300000 of=test1
[root@music /root]# dd if=/dev/md0 count=300000 of=test2

[root@music /root]# cmp -l test1 test2
6031611 243 253 [ed note: binary: 10100011 10101011]
6031612 323 134 [ed note: binary: 11010011 01011100]
6293755 256 246 [ed note: binary: 10101110 10100110]
6293756 261 76 [ed note: binary: 10110001 00111110]
7641340 102 166 [ed note: binary: 01000010 01110110]
7660796 147 255 [ed note: binary: 01100111 10101101]
10296572 35 175
11292924 235 300
11369724 5 304
11555068 165 50
11631868 166 267
[ed note: etc.. full test available]

[root@music /root]# cmp -l test1 test2 | wc
82 246 1422

The output of my boot when a drive is connected to the HPT-266 is as
follows:

Sep 12 23:45:45 music syslog: syslogd startup succeeded
Sep 12 23:45:45 music syslog: klogd startup succeeded
Sep 12 23:45:45 music kernel: klogd 1.3-3, log source = /proc/kmsg started.
Sep 12 23:45:45 music kernel: Inspecting /boot/System.map
Sep 12 23:45:45 music kernel: Loaded 5217 symbols from /boot/System.map.
Sep 12 23:45:45 music kernel: Symbols match kernel version 2.2.12.
Sep 12 23:45:45 music kernel: No module symbols loaded - kernel modules not
enabled.
Sep 12 23:45:45 music kernel: Linux version 2.2.12 (root@music.volition.org)
(gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 re
lease)) #1 Fri Sep 10 18:46:26 PDT 1999
Sep 12 23:45:45 music kernel: Detected 367505372 Hz processor.
Sep 12 23:45:45 music kernel: Console: colour VGA+ 80x25
Sep 12 23:45:45 music kernel: Calibrating delay loop... 367.00 BogoMIPS
Sep 12 23:45:45 music kernel: Memory: 128080k/131072k available (992k kernel
code, 416k reserved, 1528k data, 56k init)
Sep 12 23:45:45 music kernel: CPU: Intel Celeron (Mendocino) stepping 05
Sep 12 23:45:45 music kernel: Checking 386/387 coupling... OK, FPU using
exception 16 error reporting.
Sep 12 23:45:45 music kernel: Checking 'hlt' instruction... OK.
Sep 12 23:45:45 music kernel: POSIX conformance testing by UNIFIX
Sep 12 23:45:45 music kernel: PCI: PCI BIOS revision 2.10 entry at 0xfb420
Sep 12 23:45:45 music kernel: PCI: Using configuration type 1
Sep 12 23:45:45 music kernel: PCI: Probing PCI hardware
Sep 12 23:45:45 music kernel: Linux NET4.0 for Linux 2.2
Sep 12 23:45:45 music kernel: Based upon Swansea University Computer Society
NET3.039
Sep 12 23:45:45 music kernel: NET4: Unix domain sockets 1.0 for Linux
NET4.0.
Sep 12 23:45:45 music kernel: NET4: Linux TCP/IP 1.0 for NET4.0
Sep 12 23:45:45 music kernel: IP Protocols: ICMP, UDP, TCP
Sep 12 23:45:45 music kernel: Initializing RT netlink socket
Sep 12 23:45:45 music kernel: Starting kswapd v 1.5
Sep 12 23:45:45 music kernel: Detected PS/2 Mouse Port.
Sep 12 23:45:45 music kernel: Serial driver version 4.27 with no serial
options enabled
Sep 12 23:45:45 music kernel: pty: 256 Unix98 ptys configured
Sep 12 23:45:45 music kernel: Uniform Multi-Platform E-IDE driver Revision:
6.20
Sep 12 23:45:45 music kernel: PIIX4: IDE controller on PCI bus 00 dev 39
Sep 12 23:45:45 music kernel: PIIX4: not 100% native mode: will probe irqs
later
Sep 12 23:45:45 music kernel: ide0: BM-DMA at 0xf000-0xf007, BIOS
settings: hda:DMA, hdb:DMA
Sep 12 23:45:45 music kernel: ide1: BM-DMA at 0xf008-0xf00f, BIOS
settings: hdc:DMA, hdd:DMA
Sep 12 23:45:45 music kernel: PDC20246: IDE controller on PCI bus 00 dev 58
Sep 12 23:45:45 music kernel: PDC20246: not 100% native mode: will probe
irqs later
Sep 12 23:45:45 music kernel: PDC20246: ROM enabled at 0xe6000000
Sep 12 23:45:45 music kernel: PDC20246: (U)DMA Burst Bit ENABLED Primary PCI
Mode Secondary PCI Mode.
Sep 12 23:45:45 music kernel: ide2: BM-DMA at 0xa400-0xa407, BIOS
settings: hde:DMA, hdf:DMA
Sep 12 23:45:45 music kernel: ide3: BM-DMA at 0xa408-0xa40f, BIOS
settings: hdg:DMA, hdh:DMA
Sep 12 23:45:45 music kernel: PDC20246: IDE controller on PCI bus 00 dev 68
Sep 12 23:45:45 music kernel: PDC20246: not 100% native mode: will probe
irqs later
Sep 12 23:45:45 music kernel: PDC20246: ROM enabled at 0xe7000000
Sep 12 23:45:45 music kernel: PDC20246: (U)DMA Burst Bit ENABLED Primary PCI
Mode Secondary PCI Mode.
Sep 12 23:45:46 music atd: atd startup succeeded
Sep 12 23:45:45 music kernel: ide4: BM-DMA at 0xb800-0xb807, BIOS
settings: hdi:DMA, hdj:pio
Sep 12 23:45:45 music kernel: ide5: BM-DMA at 0xb808-0xb80f, BIOS
settings: hdk:DMA, hdl:pio
Sep 12 23:45:45 music kernel: PDC20246: IDE controller on PCI bus 00 dev 78
Sep 12 23:45:45 music kernel: PDC20246: not 100% native mode: will probe
irqs later
Sep 12 23:45:45 music kernel: PDC20246: ROM enabled at 0xe8000000
Sep 12 23:45:45 music kernel: PDC20246: (U)DMA Burst Bit DISABLED Primary
PCI Mode Secondary PCI Mode.
Sep 12 23:45:45 music kernel: PDC20246: FORCING BURST BIT 0x00 -> 0x01
ACTIVE
Sep 12 23:45:45 music kernel: ide6: BM-DMA at 0xcc00-0xcc07, BIOS
settings: hdm:pio, hdn:pio
Sep 12 23:45:45 music kernel: ide7: BM-DMA at 0xcc08-0xcc0f, BIOS
settings: hdo:DMA, hdp:pio
Sep 12 23:45:45 music kernel: HPT366: onboard version of chipset, pin1=1
pin2=2
Sep 12 23:45:45 music kernel: HPT366: IDE controller on PCI bus 00 dev 98
Sep 12 23:45:45 music kernel: HPT366: not 100% native mode: will probe irqs
later
Sep 12 23:45:45 music kernel: HPT366: reg5ah=0x03 ATA-33 Cable Port0
Sep 12 23:45:45 music kernel: ide8: BM-DMA at 0xdc00-0xdc07, BIOS
settings: hdq:pio, hdr:pio
Sep 12 23:45:45 music kernel: HPT366: IDE controller on PCI bus 00 dev 99
Sep 12 23:45:45 music kernel: HPT366: not 100% native mode: will probe irqs
later
Sep 12 23:45:45 music kernel: HPT366: reg5ah=0x01 ATA-66 Cable Port1
Sep 12 23:45:45 music kernel: ide9: BM-DMA at 0xe800-0xe807, BIOS
settings: hds:pio, hdt:pio
Sep 12 23:45:45 music kernel: hda: QUANTUM FIREBALL ST4.3A, ATA DISK drive
Sep 12 23:45:45 music kernel: hdb: Maxtor 90845D4, ATA DISK drive
Sep 12 23:45:45 music kernel: hdc: Maxtor 84320D4, ATA DISK drive
Sep 12 23:45:45 music kernel: hdd: Maxtor 90845D4, ATA DISK drive
Sep 12 23:45:45 music kernel: hde: Maxtor 90845D4, ATA DISK drive
Sep 12 23:45:45 music kernel: hdf: Maxtor 90840D6, ATA DISK drive
Sep 12 23:45:45 music kernel: hdg: Maxtor 90845D4, ATA DISK drive
Sep 12 23:45:45 music kernel: hdi: Maxtor 90840D6, ATA DISK drive
Sep 12 23:45:46 music crond: crond startup succeeded
Sep 12 23:45:45 music kernel: hdk: Maxtor 90840D5, ATA DISK drive
Sep 12 23:45:45 music kernel: hdm: Maxtor 90845D4, ATA DISK drive
Sep 12 23:45:45 music kernel: hdo: Maxtor 90840D6, ATA DISK drive
Sep 12 23:45:45 music kernel: hdq: Maxtor 90845D4, ATA DISK drive
Sep 12 23:45:45 music kernel: ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Sep 12 23:45:45 music kernel: ide1 at 0x170-0x177,0x376 on irq 15
Sep 12 23:45:45 music kernel: ide2 at 0x9400-0x9407,0x9806 on irq 11
Sep 12 23:45:45 music kernel: ide3 at 0x9c00-0x9c07,0xa006 on irq 11
Sep 12 23:45:45 music kernel: ide4 at 0xa800-0xa807,0xac06 on irq 7
Sep 12 23:45:45 music kernel: ide5 at 0xb000-0xb007,0xb406 on irq 7
Sep 12 23:45:45 music kernel: ide6 at 0xbc00-0xbc07,0xc002 on irq 5
Sep 12 23:45:45 music kernel: ide7 at 0xc400-0xc407,0xc802 on irq 5
Sep 12 23:45:45 music kernel: ide8 at 0xd400-0xd407,0xd802 on irq 11
Sep 12 23:45:45 music kernel: hda: QUANTUM FIREBALL ST4.3A, 4110MB w/81kB
Cache, CHS=524/255/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdb: Maxtor 90845D4, 8063MB w/256kB Cache,
CHS=16383/16/63, UDMA(33)
Sep 12 16:45:39 music rc.sysinit: Loading default keymap succeeded
Sep 12 16:45:39 music rc.sysinit: Setting default font succeeded
Sep 12 16:45:39 music swapon: swapon: warning: /dev/hda5 has insecure
permissions 0660, 0600 suggested
Sep 12 16:45:39 music swapon: swapon: warning: /dev/hda6 has insecure
permissions 0660, 0600 suggested
Sep 12 16:45:39 music rc.sysinit: Activating swap partitions succeeded
Sep 12 16:45:39 music rc.sysinit: Setting hostname music.volition.org
succeeded
Sep 12 16:45:39 music fsck: /dev/hda7: clean, 82206/514000 files,
1488680/2048001 blocks
Sep 12 16:45:39 music rc.sysinit: Checking root filesystem succeeded
Sep 12 16:45:39 music rc.sysinit: Remounting root filesystem in read-write
mode succeeded
Sep 12 16:45:39 music fsck: /dev/hda1: clean, 58/52208 files, 26401/208813
blocks
Sep 12 16:45:39 music rc.sysinit: Checking filesystems succeeded
Sep 12 16:45:40 music rc.sysinit: Mounting local filesystems succeeded
Sep 12 23:45:45 music kernel: hdc: Maxtor 84320D4, 4120MB w/256kB Cache,
CHS=8930/15/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdd: Maxtor 90845D4, 8063MB w/256kB Cache,
CHS=16383/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hde: Maxtor 90845D4, 8063MB w/256kB Cache,
CHS=16383/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdf: Maxtor 90840D6, 8010MB w/256kB Cache,
CHS=16276/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdg: Maxtor 90845D4, 8063MB w/256kB Cache,
CHS=16383/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdi: Maxtor 90840D6, 8010MB w/256kB Cache,
CHS=16276/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdk: Maxtor 90840D5, 8047MB w/256kB Cache,
CHS=16351/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdm: Maxtor 90845D4, 8063MB w/256kB Cache,
CHS=16383/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdo: Maxtor 90840D6, 8010MB w/256kB Cache,
CHS=16276/16/63, UDMA(33)
Sep 12 23:45:45 music kernel: hdq: Maxtor 90845D4, 8063MB w/256kB Cache,
CHS=16383/16/63, (U)DMA
Sep 12 23:45:45 music kernel: Floppy drive(s): fd0 is 1.44M
Sep 12 23:45:46 music inet: inetd startup succeeded
Sep 12 16:45:40 music rc.sysinit: Turning on user and group quotas for local
filesystems succeeded
Sep 12 23:45:42 music date: Sun Sep 12 23:45:42 PDT 1999
Sep 12 23:45:42 music rc.sysinit: Setting clock : Sun Sep 12 23:45:42 PDT
1999 succeeded
Sep 12 23:45:42 music rc.sysinit: Enabling swap space succeeded
Sep 12 23:45:42 music init: Entering runlevel: 3
Sep 12 23:45:44 music network: Bringing up interface lo succeeded
Sep 12 23:45:44 music network: Bringing up interface eth0 succeeded
Sep 12 23:45:45 music portmap: portmap startup succeeded
Sep 12 23:45:45 music random: Initializing random number generator succeeded
Sep 12 23:45:45 music kernel: FDC 0 is a post-1991 82077
Sep 12 23:45:45 music kernel: md driver 0.90.0 MAX_MD_DEVS=256, MAX_REAL=12
Sep 12 23:45:45 music kernel: translucent personality registered
Sep 12 23:45:45 music kernel: linear personality registered
Sep 12 23:45:45 music kernel: raid0 personality registered
Sep 12 23:45:45 music kernel: raid1 personality registered
Sep 12 23:45:45 music kernel: raid5 personality registered
Sep 12 23:45:45 music kernel: raid5: measuring checksumming speed
Sep 12 23:45:45 music kernel: raid5: MMX detected, trying high-speed MMX
checksum routines
Sep 12 23:45:45 music kernel: pII_mmx : 817.626 MB/sec
Sep 12 23:45:45 music kernel: p5_mmx : 858.774 MB/sec
Sep 12 23:45:45 music kernel: 8regs : 631.317 MB/sec
Sep 12 23:45:45 music kernel: 32regs : 353.949 MB/sec
Sep 12 23:45:45 music kernel: using fastest function: p5_mmx (858.774
MB/sec)
Sep 12 23:45:45 music kernel: scsi : 0 hosts.
Sep 12 23:45:45 music kernel: scsi : detected total.
Sep 12 23:45:47 music named[316]: starting. named 8.2 Wed Mar 31 10:57:12
EST 1999 ^Iroot@porky.devel.redhat.com:/usr/src/bs/BUILD/
bind-8.2/src/bin/named
Sep 12 23:45:47 music named[316]: cache zone "" (IN) loaded (serial 0)
Sep 12 23:45:47 music named[316]: Zone "0.0.127.in-addr.arpa" (file
named.local): No default TTL set using SOA minimum instead
Sep 12 23:45:47 music named[316]: master zone "0.0.127.in-addr.arpa" (IN)
loaded (serial 1997022700)
Sep 12 23:45:47 music named[316]: listening on [127.0.0.1].53 (lo)
Sep 12 23:45:47 music named[316]: listening on [10.10.10.199].53 (eth0)
Sep 12 23:45:47 music named[316]: Forwarding source address is
[0.0.0.0].1024
Sep 12 23:45:45 music kernel: 3c59x.c:v0.99H 11/17/98 Donald Becker
http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html
Sep 12 23:45:45 music kernel: eth0: 3Com 3c905 Boomerang 100baseTx at
0xd000, 00:60:08:b0:2e:3c, IRQ 10
Sep 12 23:45:45 music kernel: 8K word-wide RAM 3:5 Rx:Tx split,
autoselect/MII interface.
Sep 12 23:45:45 music kernel: MII transceiver found at address 24, status
786f.
Sep 12 23:45:45 music kernel: Enabling bus-master transmits and
whole-frame receives.
Sep 12 23:45:45 music kernel: md.c: sizeof(mdp_super_t) = 4096
Sep 12 23:45:45 music kernel: Partition check:
Sep 12 23:45:45 music kernel: hda: hda1 hda2 < hda5 hda6 hda7 >
Sep 12 23:45:45 music kernel: hdb: hdb1
Sep 12 23:45:45 music kernel: hdc: [PTBL] [525/255/63] hdc1 hdc2 < hdc5
hdc6 hdc7 >
Sep 12 23:45:45 music kernel: hdd: hdd1
Sep 12 23:45:45 music kernel: hde: hde1
Sep 12 23:45:45 music kernel: hdf: hdf1
Sep 12 23:45:45 music kernel: hdg: hdg1
Sep 12 23:45:45 music kernel: hdi: hdi1
Sep 12 23:45:45 music kernel: hdk: hdk1
Sep 12 23:45:45 music kernel: hdm: hdm1
Sep 12 23:45:45 music kernel: hdo: hdo1
Sep 12 23:45:47 music named: named startup succeeded
Sep 12 23:45:47 music named[317]: Ready to answer queries.
Sep 12 23:45:45 music kernel: hdq: hdq1
Sep 12 23:45:45 music kernel: autodetecting RAID arrays
Sep 12 23:45:45 music kernel: (read) hdb1's sb offset: 8256896 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdd1's sb offset: 8256896 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hde1's sb offset: 8256896 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdf1's sb offset: 8203008 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdg1's sb offset: 8256896 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdi1's sb offset: 8203008 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdk1's sb offset: 8240768 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdm1's sb offset: 8256896 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdo1's sb offset: 8203008 [events:
00000002]
Sep 12 23:45:45 music kernel: (read) hdq1's sb offset: 8256896 [events:
00000002]
Sep 12 23:45:45 music kernel: autorun ...
Sep 12 23:45:45 music kernel: considering hdq1 ...
Sep 12 23:45:47 music keytable: Loading keymap:
Sep 12 23:45:45 music kernel: adding hdq1 ...
Sep 12 23:45:45 music kernel: adding hdo1 ...
Sep 12 23:45:45 music kernel: adding hdm1 ...
Sep 12 23:45:45 music kernel: adding hdk1 ...
Sep 12 23:45:45 music kernel: adding hdi1 ...
Sep 12 23:45:45 music kernel: adding hdg1 ...
Sep 12 23:45:45 music kernel: adding hdf1 ...
Sep 12 23:45:45 music kernel: adding hdd1 ...
Sep 12 23:45:45 music kernel: adding hdb1 ...
Sep 12 23:45:45 music kernel: created md0
Sep 12 23:45:45 music kernel: bind<hdb1,1>
Sep 12 23:45:45 music kernel: bind<hdd1,2>
Sep 12 23:45:45 music kernel: bind<hdf1,3>
Sep 12 23:45:45 music kernel: bind<hdg1,4>
Sep 12 23:45:45 music kernel: bind<hdi1,5>
Sep 12 23:45:45 music kernel: bind<hdk1,6>
Sep 12 23:45:45 music kernel: bind<hdm1,7>
Sep 12 23:45:45 music kernel: bind<hdo1,8>
Sep 12 23:45:45 music kernel: bind<hdq1,9>
Sep 12 23:45:45 music kernel: running:
<hdq1><hdo1><hdm1><hdk1><hdi1><hdg1><hdf1><hdd1><hdb1>
Sep 12 23:45:45 music kernel: now!
Sep 12 23:45:45 music kernel: hdq1's event counter: 00000002
Sep 12 23:45:45 music kernel: hdo1's event counter: 00000002
Sep 12 23:45:48 music keytable: Loading
/usr/lib/kbd/keymaps/i386/qwerty/us.kmap.gz
Sep 12 23:45:45 music kernel: hdm1's event counter: 00000002
Sep 12 23:45:45 music kernel: hdk1's event counter: 00000002
Sep 12 23:45:45 music kernel: hdi1's event counter: 00000002
Sep 12 23:45:45 music kernel: hdg1's event counter: 00000002
Sep 12 23:45:45 music kernel: hdf1's event counter: 00000002
Sep 12 23:45:45 music kernel: hdd1's event counter: 00000002
Sep 12 23:45:45 music kernel: hdb1's event counter: 00000002
Sep 12 23:45:45 music kernel: md: device name has changed from hdh1 to hdq1
since last import!
Sep 12 23:45:45 music kernel: md0: max total readahead window set to 4608k
Sep 12 23:45:45 music kernel: md0: 9 data-disks, max readahead per
data-disk: 512k
Sep 12 23:45:45 music kernel: raid5: device hdq1 operational as raid disk 6
Sep 12 23:45:45 music kernel: raid5: device hdo1 operational as raid disk 7
Sep 12 23:45:45 music kernel: raid5: device hdm1 operational as raid disk 4
Sep 12 23:45:45 music kernel: raid5: device hdk1 operational as raid disk 9
Sep 12 23:45:48 music keytable: Loading system font:
Sep 12 23:45:45 music kernel: raid5: device hdi1 operational as raid disk 3
Sep 12 23:45:45 music kernel: raid5: device hdg1 operational as raid disk 1
Sep 12 23:45:45 music kernel: raid5: device hdf1 operational as raid disk 2
Sep 12 23:45:45 music kernel: raid5: device hdd1 operational as raid disk 5
Sep 12 23:45:45 music kernel: raid5: device hdb1 operational as raid disk 0
Sep 12 23:45:45 music kernel: raid5: md0, not all disks are operational --
trying to recover array
Sep 12 23:45:45 music kernel: raid5: allocated 10554kB for md0
Sep 12 23:45:45 music kernel: raid5: raid level 5 set md0 active with 9 out
of 10 devices, algorithm 2
Sep 12 23:45:45 music kernel: RAID5 conf printout:
Sep 12 23:45:45 music kernel: --- rd:10 wd:9 fd:1
Sep 12 23:45:45 music kernel: disk 0, s:0, o:1, n:0 rd:0 us:1 dev:hdb1
Sep 12 23:45:45 music kernel: disk 1, s:0, o:1, n:1 rd:1 us:1 dev:hdg1
Sep 12 23:45:45 music kernel: disk 2, s:0, o:1, n:2 rd:2 us:1 dev:hdf1
Sep 12 23:45:45 music kernel: disk 3, s:0, o:1, n:3 rd:3 us:1 dev:hdi1
Sep 12 23:45:45 music kernel: disk 4, s:0, o:1, n:4 rd:4 us:1 dev:hdm1
Sep 12 23:45:49 music rc: Starting keytable succeeded
Sep 12 23:45:45 music kernel: disk 5, s:0, o:1, n:5 rd:5 us:1 dev:hdd1
Sep 12 23:45:45 music kernel: disk 6, s:0, o:1, n:6 rd:6 us:1 dev:hdq1
Sep 12 23:45:45 music kernel: disk 7, s:0, o:1, n:7 rd:7 us:1 dev:hdo1
Sep 12 23:45:45 music kernel: disk 8, s:0, o:0, n:8 rd:8 us:1 dev:[dev
00:00]
Sep 12 23:45:45 music kernel: disk 9, s:0, o:1, n:9 rd:9 us:1 dev:hdk1
Sep 12 23:45:45 music kernel: disk 10, s:0, o:0, n:0 rd:0 us:0 dev:[dev
00:00]
Sep 12 23:45:45 music kernel: disk 11, s:0, o:0, n:0 rd:0 us:0 dev:[dev
00:00]
Sep 12 23:45:45 music kernel: RAID5 conf printout:
Sep 12 23:45:45 music kernel: --- rd:10 wd:9 fd:1
Sep 12 23:45:45 music kernel: disk 0, s:0, o:1, n:0 rd:0 us:1 dev:hdb1
Sep 12 23:45:45 music kernel: disk 1, s:0, o:1, n:1 rd:1 us:1 dev:hdg1
Sep 12 23:45:45 music kernel: disk 2, s:0, o:1, n:2 rd:2 us:1 dev:hdf1
Sep 12 23:45:45 music kernel: disk 3, s:0, o:1, n:3 rd:3 us:1 dev:hdi1
Sep 12 23:45:45 music kernel: disk 4, s:0, o:1, n:4 rd:4 us:1 dev:hdm1
Sep 12 23:45:45 music kernel: disk 5, s:0, o:1, n:5 rd:5 us:1 dev:hdd1
Sep 12 23:45:45 music kernel: disk 6, s:0, o:1, n:6 rd:6 us:1 dev:hdq1
Sep 12 23:45:45 music kernel: disk 7, s:0, o:1, n:7 rd:7 us:1 dev:hdo1
Sep 12 23:45:45 music kernel: disk 8, s:0, o:0, n:8 rd:8 us:1 dev:[dev
00:00]
Sep 12 23:45:45 music kernel: disk 9, s:0, o:1, n:9 rd:9 us:1 dev:hdk1
Sep 12 23:45:45 music kernel: disk 10, s:0, o:0, n:0 rd:0 us:0 dev:[dev
00:00]
Sep 12 23:45:45 music kernel: disk 11, s:0, o:0, n:0 rd:0 us:0 dev:[dev
00:00]
Sep 12 23:45:45 music kernel: md: updating md0 RAID superblock on device
Sep 12 23:45:45 music kernel: hdq1 [events: 00000003](write) hdq1's sb
offset: 8256896
Sep 12 23:45:45 music kernel: md: recovery thread got woken up ...
Sep 12 23:45:45 music kernel: md0: no spare disk to reconstruct array! --
continuing in degraded mode
Sep 12 23:45:45 music kernel: md: recovery thread finished ...
Sep 12 23:45:45 music kernel: hdo1 [events: 00000003](write) hdo1's sb
offset: 8203008
Sep 12 23:45:45 music kernel: hdm1 [events: 00000003](write) hdm1's sb
offset: 8256896
Sep 12 23:45:52 music sendmail: sendmail startup succeeded
Sep 12 23:45:45 music kernel: hdk1 [events: 00000003](write) hdk1's sb
offset: 8240768
Sep 12 23:45:45 music kernel: hdi1 [events: 00000003](write) hdi1's sb
offset: 8203008
Sep 12 23:45:45 music kernel: hdg1 [events: 00000003](write) hdg1's sb
offset: 8256896
Sep 12 23:45:45 music kernel: hdf1 [events: 00000003](write) hdf1's sb
offset: 8203008
Sep 12 23:45:45 music kernel: hdd1 [events: 00000003](write) hdd1's sb
offset: 8256896
Sep 12 23:45:45 music kernel: hdb1 [events: 00000003](write) hdb1's sb
offset: 8256896
Sep 12 23:45:45 music kernel: .

Let me know if I can be of any more help.

Tom

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/