partial network outage

| | Comments (0)
While we were making changes to the BGP announcements, some subset of the internet stopped accepting our routes. For example, from my home ISP and some other providers there were no issues reaching all of our network, but there were issues between our servers at Sacramento and some of our network.

We eventually isolated this to an MTU change to support jumbo frames which should have been OK - the switches and our upstream were configured for jumbo frames and we checked this in advance of making our change.  We believed the MTU not matching our upstream may have been causing some more minor packet loss, which is why we changed it.

We found our BGP session session was continually being re-established. When the connection times out the routes were dropped and immediately picked up again but it was not long enough to propagate to the transit providers who apparently do not cache routes. 
Since the MTU was supposed to work and were other changes to BGP configuration we only found the problem by process of elimination.

 When the MTU change was reverted everything started working properly again. Now that this is resolved I am continuing finalizing the network changes.

Leave a comment

About this Entry

This page contains a single entry by srn published on April 22, 2015 3:02 AM.

Network maintenance starting tonight 20:00 -0700 - minimal downtime expected was the previous entry in this blog.

router transition mostly complete is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.