I screwed up.

| | Comments (0)
Holmes was having problems after the move;  it wouldn't boot.   I pxe-booted holmes (as is standard practice when a system won't boot) without first verifying that I had removed the .ks file and appended 'rescue' to the kernel in the pxelinux.cfg file.   (policy is to do that as soon as you install a new system, before it goes production, but it's not automated, so it doesn't always happen.)

clearpart was run, and a new / partition was laid down and formatted (but  no customer data is on the / partition, save for your public keys)  

Clearpart means that the metadata for the LVM partitions was removed, but I stopped the install before new data was written to the LVM, so it's possible we will be able to recover the data.   

Either way, I'm clearly not in any shape to handle root. I hope the problem is insufficent sleep.  I will sleep and report back in the morning. 

Note, as of now?  all xen hosts are either in yd33 at 55 s. market or in coresite santa clara 2972 stender.    I still have dedicated servers to do, but that won't impact the VPS customers.

Leave a comment

About this Entry

This page contains a single entry by luke published on May 27, 2013 3:25 AM.

Hey, make sure you disable tso and gso in your guest was the previous entry in this blog.

what exactly happened on holmes and how we will prevent that from happening again is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.