birds back up after unplanned downtime.

| | Comments (1)
no data loss, but everyone was down for a day.  I will credit all customers on birds 1 month for that.  the problem was that we got a bad disk and md did not fail it, and we were unable to fail it manually.   we tried a reboot but the good drive did not have the grub bootloader installed.   ugh.

anyhow, birds is back but it's running on only one drive;   my replacement drive is DOA or there is a cabling problem.   I will try another drive tonight as that is easier than jimming with the cables.   If there will be a reboot I will give you some warning.

also, we either need to learn how to properly tune md so it fails drives rather than hanging when they are bad, or we need to move away from md.   I will write more about that after I fix the raid.  

at least there was no data loss. 



note, disk will suck for a while as everyone's domain finishes FSCK and 'warms up' pagecache.

Leave a comment

About this Entry

This page contains a single entry by luke published on August 5, 2009 6:51 PM.

problems with billing (or, this is why I send invoices rather than direct billing) was the previous entry in this blog.

I/O hang and unplanned (unclean) reboot of girdle is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.