1. Preparations

In our world Electricity is transformed into Bits, with the by-product of BTUs (heat). Our job is to handle (route) bits and manage (cool) BTUs. Despite the fact that we are fairly certain that the technicians can get their work done with minimal downtime of the HVAC system, we are living by the old adage "hope for the best, but prepare for the worst." To that end we have performed the following preparations. We are pretty intimate with our facility and know where the heavy users of electricity are located. We have the "hot spots" identified and covered by portable AC units.

We also have the ability to pull outside air into the facility in large volume, and use smaller local fans to provide ventilation to "warm spots." The outside temperature at the moment is 39°F/4°C, so it is a fairly good day to be performing this task.

This process of course requires a bit of preparation itself. High CFM fans and portable HVAC units are not exactly light users of electricity themselves, and to protect the servers you depend upon we can't just plug them in wherever there's an open outlet. Mechanical motors put variable strains on electrical circuits and it is not smart to put them on the same circuit being used by computers. Therefore we have used building electricity circuits for these devices, rather than the power from our PDUs that feeds the racks. We've taken the extra step to lay extension cords to the various mechanical units, and gaffer-taped those to the floor. Additionally our Facilities Manager has diagramed the circuits and breakers involved in feeding the mechanical units and calculated the amperage loads so we can avoid popping breakers.

We have also deployed some temperature probes in critical locations to monitor the ambient temperatures in "cool rows" to see what the intake air is like for servers. Finally, overnight we dropped the datacenter temperature several degrees below our normal 65°F/18°C to provide some "breathing room".

More info coming soon.