Fedora 9 will not boot (mdadm/RAID problems, cannot find "/dev/md0", etc.)

[northwind]

The problem:

Decompressing linux... done.
Booting the kernel.
Red Hat nash version 6.0.52 starting
mdadm: /dev/md1 started with 2 drives.
mdadm: /dev/md2 started with 2 drives.
                Welcome to Fedora
                Press 'I' to enter interactive startup.
Starting udev:                                             [  OK  ]
.
.
.
    e2fsck -b 8193 <device>

fsck.ext3: No such file or directory while trying to open /dev/md3
/dev/md3:
The superblock could not be read or does not descripbe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

fsck.ext3: No such file or directory while trying to open /dev/md0
/dev/md0:
The superblock could not be read or does not descripbe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

                                                           [FAILED]

*** An error occurred during the file system check.
*** Dropping you to a shell; the system will reboot
*** when you leave the shell.
Give root password for maintenance
(or type Control-D to continue):

As the above data shows, some of the RAID disks are being mounted while others are failing. This was after doing a rather braindead yum update m* n* (ok, very braindead update). Will I need to use a Fedora Rescue CD to get my system back? Or is there another way? This archived thread seems to be somewhat relevant (as well as this FAQ from Red Hat).

When I enter the root password, it drops me into a shell where I have read-only access. Not good for fixing the problem; but it at least allows me to get the following information.

The /etc/mdadm.conf file:

# mdadm.conf written out by anaconda
DEVICE partitions
MAILADDR root

ARRAY /dev/md3 level=raid1 num-devices=2 UUID=87c0a06f:820a9af2:102d1836:64db2361
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=9803901f:3f22e151:e5ccdc5e:9ec14483
ARRAY /dev/md4 level=raid1 num-devices=2 UUID=f1952d99:8adda65a:1a02cae5:75bf90a9
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=d8a16f21:e9cc53e6:d7395991:2916fcf9
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=bd49febf:06cf2dbe:cdfbe4da:304c4a31

The /etc/fstab file:

/dev/md2        /            ext3    defaults        1 1
/dev/md4        /tmp         ext3    defaults        1 2
/dev/md3        /var         ext3    defaults        1 2
/dev/md0        /boot        ext3    defaults        1 2
tmpfs           /dev/shm     tmpfs   defaults        0 0
devpts          /dev/pts     devpts  gid=5,mode=620  0 0
sysfs           /sys         sysfs   defaults        0 0
/dev/md1        swap         swap    defaults        0 0

The results from ls /dev/[hms]d*

/dev/md1  /dev/sda1  /dev/sda4  /dev/sdb   /dev/sdb3  /dev/sda6
/dev/md2  /dev/sda2  /dev/sda5  /dev/sdb1  /dev/sdb4
/dev/sda  /dev/sda3  /dev/sda6  /dev/sdb2  /dev/sdb5

The results from fdisk /dev/sda

The number of cylinders for this disk is set to 14593.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old version of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sda: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x017486f0

   Device Boot    Start        End       Blocks    Id  System
/dev/sda1   *         1         13       104391    fd  Linux raid autodetect
/dev/sda2            14      14036    112639747+   fd  Linux raid autodetect
/dev/sda3         14037      14214      1429785    fd  Linux raid autodetect
/dev/sda4         14215      14593      3044317+    5  Extended
/dev/sda5         14215      14469      2048256    fd  Linux raid autodetect
/dev/sda6         14470      14593       995998+   fd  Linux raid autodetect

The results from fdisk /dev/sdb

The number of cylinders for this disk is set to 14593.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old version of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdb: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x7b8e46b2

   Device Boot    Start        End       Blocks    Id  System
/dev/sdb1   *         1         13       104391    fd  Linux raid autodetect
/dev/sdb2            14      14036    112639747+   fd  Linux raid autodetect
/dev/sdb3         14037      14214      1429785    fd  Linux raid autodetect
/dev/sdb4         14215      14593      3044317+    5  Extended
/dev/sdb5         14215      14469      2048256    fd  Linux raid autodetect
/dev/sdb6         14470      14593       995998+   fd  Linux raid autodetect

[brittminer]

I had a similar issue with a new Fedora 11 install.

Pertinent info:
* I was receiving the same error you are (except my device is /dev/dm-2).
* I'm using hardware raid.
* If I booted from a USB stick, I could successfully mount the raid file system manually after the boot up with no problem.

I seem to have solved the problem simply by NOT doing a file system check on the raid array during boot up.

In /etc/fstab, try replacing your "1 2" options with "1 0" on the lines pertaining to your raided file systems:

/dev/md2        /            ext3    defaults        1 0
/dev/md4        /tmp         ext3    defaults        1 0
/dev/md3        /var         ext3    defaults        1 0
/dev/md0        /boot        ext3    defaults        1 0