outage: June 2011 Archives

We are planning to move the servers apples, cerberus, bowl and branch 
on Tuesday evening this week (June 21, 2011) to the new data center.
Expect the downtime to start sometime after 9PM PDT, and everything
should be back up by midnight. Branch needs a new disk also, so we are
going to take it down earlier and rebuild the raid with the new disk before
the move. It will be down starting at 7PM PDT , and when it finishes
rebuilding we will start shutting down the other 3 servers.

If that goes well, hopefully we can move the second group of servers from the eliteweb cabinet on Wednesday night (knife, cauldron and council). See http://book.xen.prgmr.com/mediawiki/index.php/EGI_Moving
If you have any questions, please email support@prgmr.com. It might be delayed if there is much support email to answer from the earlier move or just regular tickets also though. Thanks!

Update: We're starting later for the first move. Now it will start more like midnight or after.

Update 00:08  -  we are rebooting and rebuilding branch before the move.  expect branch downtime to start in a few minutes,  expect downtime for the other servers to start in maybe three hours.

Update 00:35: Branch is down and the RAID is rebuilding.     nobody else is rebooting yet.

Update 05:54:  branch is done rebuilding, apples, cerberus, bowl down for moving

Update: 07:29:  all servers are coming  back up, but all network connectivity is down.  At this point, my provider seems to think it's something between, and not my network, but it's too early to say.  

Update: 08:26:  Cerberus doesn't want to boot 'cause I removed a bad drive (a spare had replaced it in the RAID)   -  I put the drive back and it boots.    Bowl is also down, I don't know what that problem is yet.  

Update:08:38: bowl and branch are both starting xendomains, the problem with bowl was an upgrade blew out the menu.st file.  the problem with branch was that we incorrectly set power on after power fail to off.

update: 08:51:  all users should now be up and running.  complain otherwise.   




About this Archive

This page is a archive of entries in the outage category from June 2011.

outage: May 2011 is the previous archive.

outage: September 2011 is the next archive.

Find recent content on the main index or look in the archives to find all content.