redundant array of crappy disks.

| | Comments (1)
Let me describe to you how we're partitioning the current machines.  The goal is to have everything mirrored, so that we can survive an outage on our cheap and crappy SATA disks.  (I mean, they're Seagate and Maxtors.  Could be much worse.  But they're still very commodity.  You know how it goes.  Worse is better.)

Anyway, let's make a table.  All of the md devices are mirrored:

/dev/md0         /             10G
/dev/sd(1,0)     (swap)     (memsize)
/dev/md1         LVM PV  5G

The rest of each drive is an LVM PV.  The three PVs (one mirrored, two not) are then ganged into a VG.  (One of the first things that we do is create an LV to use for /var, sized the same as swap.  This is so that we can save the domUs at machine shutdown and restore them on reboot.  We might just make / bigger, though.)

When we create a domU, the backing store gets created something like this:

 # lvcreate -L 5G -m 1 vgname

Which puts each leg of the mirror on a separate disk, and the mirror log on /dev/md2.

This is fine and dandy.  One problem, though, is that GRUB isn't installed properly on both devices.  If drive 0 fails in a way that renders it invisible to the BIOS, the machine won't boot.  Here's the solution:

 grub> root (hd1,0)
 grub> install /boot/grub/stage1 (hd1) (hd1)1+15 p (hd0,0)/boot/grub/stage2 /boot/grub/grub.conf

Unfortunately, if the failed drive 0 is still visible to the BIOS, GRUB won't load at all.  My understanding of GRUB isn't complete enough to specify a stage2 relative to the boot sector, although it seems like the sort of thing that should be possible.

1 Comments

how much reliability might / on flash (or netbooting but putting / on md) might add, do you think?

Leave a comment