Due to problems at the company that controls our Name Server, we were down for most of today. The short synopsis is about 3am Pacific, a Denial Of Service attack/ HUGE influx if DNS queries bombarded their main facility in Seattle. In the following hours everything on that network became extremely slow making most of the services provided on that network (DNS,URL forwarding, Email Forwarding, websites, DB) appear unavailable or really slow to anyone outside the network. This kind of activity has happened before but never of this magnitude. The unfortunate side effect of this activity is that it overloaded both the primary and secondary firewalls causing them to reset connections about every 2 minutes. Their senior network engineer was woken up and after having no luck with a remote fix headed to our co-lo facility. He arrived to find the firewalls rebooting under a large deluge of traffic. He couldn't even get information off of the firewalls about what was actually happening.
In the meantime, the downtime at our co-lo in Seattle caused all DNS to be directed to our east coast facility. The facility also was brought down by the volume in traffic. As they tried to diagnose what the problem was so that they could know what to cut off, the traffic just kept coming. After some conflict with the co-lo provider, finally at 5pm PST, they filtered out all traffic destined for the nameserver in the Seattle co-lo. This immediately enabled all services on that network to the outside world.
Isn't this stuff fun?
Ben
-= VPC OffLine Reader 2.1 =-
Registered to: Ben Chiu
-OLR.PL v1.80-
Printer-friendly
version of this topic
Bookmark this topic
(Registered users only)
VPC Tech
![[Admin]](http://www.flightadventures.com/Forum/htdocs/dcforum/Images/4bar.gif)
VPC unreachable 12-09-02
![[Sysop]](http://www.flightadventures.com/Forum/htdocs/dcforum/Images/3bar.gif)
