power outage at he.net, network outage at SVTIX

| | Comments (0)
According to the incident report I got from my upstream, there was a PG&e outage  that caused power loss to parts of southern Alameda County.

He.net fremont 1 was affected when the automatic transfer switches properly started the backup generator but failed to cutover.   The incident lasted approximately one hour.   

now, this affected network connectivity to svtix, as right now svtix is he.net bandwidth only, and it goes through he.net fremont 1, so we had an hour or so of network interruption at svtix.  

I'll be taking steps to make my network at svtix more diverse later today.  For now, all xen servers at he.net save for ingot should be back up.   45 minute ETA on ingot.  

Update: kvm guests up.   coral and ingot still down, weird disk issues.  I'm bringing hardware and boot disks down to the co-lo to support nick in person.  

edit: Ingot is up   Eta on coral 110 minutes.   

Note, the main prgmr.com website is on coral, as is our support@prgmr.com email system.  lsc@prgmr.com still works

edit: Coral is up.    

edit:  we missed table.prgmr.com;  it's still down.   nick is working on it now.  





Leave a comment

About this Entry

This page contains a single entry by luke published on May 6, 2011 9:11 AM.

moving servers to market post tower was the previous entry in this blog.

table outage is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.