Here is the report from our upstream on the recent network outages. Please note we received this a couple of days ago but it had a confidentiality clause so we waited to hear back on how much could be shared before posting. We’re also trying to track down some transient packet loss.


  • January 20, 2016: At approximately 13:20 PST, utility power was interrupted, and the building transferred successfully to the back-up power source. At approximately 16:30 PST, utility power was restored. During restoration, the UPS experienced a fault causing it to shut down. The UPS was manually put into maintenance bypass and service was restored. A service call was placed to the UPS vendor and technicians dispatched. As a diagnostic step, the battery string was replaced.

  • January 25, 2016: At approximately 12:10 PST we attempted to bring the UPS out of maintenance bypass resulting in the UPS shutting down during the cutover. The unit was placed in maintenance bypass and service was restored shortly thereafter. We engaged the UPS vendor for assistance in troubleshooting.

  • January 27, 2016: The UPS was inspected, diagnostics run and the main fuse replaced. At approximately 14:30 PST an attempt was made to bring the UPS online. The UPS shut down during the cutover. The unit was placed into maintenance bypass and service was restored.

They say there will be a separate maintenance announcement for replacing the UPS.