4. Wrap up & Summary

We're pleased to report that the repair on our HVAC system is complete, and finished without incident. The final bit of work required brazing & welding within the unit itself. To mitigate any risk of having the pre-action fire suppression system discharging its gasses, we had our vendor Fire Chief, come out and disable the system. Part of our annual maintenance procedure for the fire suppression system involves the shut down of the HVAC system anyway, so Fire Chief took advantage of the situation to perform that maintenance.

Above: Technicians from Fire Chief perform preventative maintenance on the Fire Detection and Suppression system.

During the HVAC system shutdown, digital.forest staff monitored temperatures in various locations around the datacenter, while our Facilities Manager bounced between the roof and the datacenter monitoring our vendors. Below you can see digital.forest Tech Support member Will Winslow and Facilities Manager Kevin Teker in the darkened datacenter just after the HVAC shutdown occurred. They're carrying their temperature monitors and about to spread out to their stations. You can see the high-CFM fans mentioned earlier today in the open door behind them.

All of our preparation paid off, plus a bit of luck from the weather (it stayed very cool, plus it didn't rain) so that the natural tendency for the facility to warm up was mitigated by the combination of pre-cooling and the fans pulling outside air into the facility. We're happy to report that our highest temperature reached was about what we see here on a "normal" day. Our temporary portable HVAC units never even needed to be turned on.

Interesting conclusions:
Electrical capacity is a hot topic in the datacenter management business these days. There are various rules of thumb concerning the estimataion of power usage split between "floor" (meaning the servers) and "mechanical" (meaning the HVAC systems to cool the servers.) The variable is the delta between outside and inside ambient temperatures. The hotter it is outside, the harder the HVAC systems have to work to chill the inside. We're blessed to be located in a very moderate climate here in Seattle. It rarely gets very hot here. Nor does it get very cold. Our average temperature is actually quite a bit lower than ideal datacenter temperature. Even in summer, it cools enough at night to keep our average right at ideal datacenter temperature. We monitor electricity usage at several points, along the flow for a lot of reasons, but on our main panel in the datacenter we can check at a glance and see how much power is being used in total. The ammeter for example read this way earlier today when we were running the rooftop HVAC and 100% using outside air:

That reads 274 Amps. That is 274 Amps of 3-phase power as it comes in off the grid. Our feed is 2000 Amps so as you can see we have a lot of room for growth with regards to electricity. This is one of the things that really attracted us to this facility when we moved here just over two years ago. With so many datacenter operations running at nearly 100% of their power capacity we felt it important to be able to accommodate our clients expanding needs and requirements. This maintenance interval provided us some real-time data concerning the power needs of our mechanical infrastructure. Those rules of thumb mentioned earlier say "for every 1 amp you feed the floor, you feed the mechanical 1 to 1.75 amps." This seems to have been proven in our experience, but rounded down due to our temperate, if not downright cool location here in Seattle. Here is a shot of the ammeter with the HVAC system shut down completely:

That is 219 Amps of 3-phase power. Looking at our monitoring history, we hit our maximum of 400 Amps last July when we had a week of temperatures in the 90-95° F (32-35°C) range. That means we are running at a roughly 1:1 floor:mechanical ratio in terms of electricity at our peak consumption. If anything we are favoring the floor, which is a great advantage in this industry.

Yet another benefit of colocation at digital.forest in cool Seattle!