During the scheduled maintenance window on Friday, February 26th, we experienced two intervals of network instability, each resulting in significantly reduced connectivity. Our network experienced increased latency and in many cases servers being unreachable from certain external networks. While technically not a network outage, we are aware that many of our clients were seriously impacted from approximately 01:00 until 01:10, and again from 02:05 until 02:35.

The root causes of both network events were directly related to configuration conflicts within our new core routers. These conflicts were very different in nature, and manifested themselves in very different ways. Following each of the impacted intervals we successfully isolated and mitigated the specific configuration conflict. Although unrelated to each other, both of the configurations that caused these issues are essential to the safe and stable operation of our network.

Our ongoing mission is to maintain, and strive to improve, the reliability and performance of the critical systems that are the hallmark of digital.forest's service. As stated in our support blog post of February 24th:

"In February of 2009, we commenced a comprehensive analysis of our current network infrastructure, comparing it to what are now the most trusted and efficient network technologies available. Appraising our current and projected future requirements alongside available mature technologies, we initiated the process of testing numerous network hardware and configurations. "

"Utilizing current models of Cisco carrier-grade network equipment, our new core infrastructure is designed to be fault-tolerant, easily expandable, fast and flexible."

We are constantly evaluating our processes and protocols, and as a direct result of client feedback we are implementing a new notification system to provide significantly improved advance notification of upgrades and maintenance. digital.forest is also working rapidly to provide a truly fault-tolerant support website and helpdesk. We will provide a direct announcement, including any necessary instructions, to all of our clients as soon as these services have been enhanced.

The entire staff at digital.forest wishes to express our sincerest regret for the inconvenience to our clients and their customers during our network upgrade. Be assured that we will continue to work tirelessly to improve both the systems and processes we employ to deliver the service and support that you have come to expect.