******SERVICE IMPACTING UPS SYSTEM MAINTENANCE******

On Thursday, March 19th starting at 07:00 hrs PST and ending at 18:30 hrs PST, we will perform maintenance on our UPS systems. This maintenance will include replacing the batteries in UPS 1 and upgrading the capacitors in UPS 3.

******Impacted Services******

Our FileMaker 3, 4, 5, and 6, Lasso 3, 4, 5, and 6, and legacy Mac shared web hosting environments will be taken offline for 30 minutes in the morning at approximately 08:00 hrs PST and again for 30 minutes at approximately 18:00 hrs PST.

******************
For the duration of this maintenance our datacenter electrical load will be transferred to generator power.

This combined maintenance window has been selected based on factors to provide the greatest degree of protection for our clients. These factors include the availability of the most senior technicians from our UPS vendor and the ability to mobilize parts and additional service personnel in the very unlikely event of a component malfunction or failure during the maintenance. Additionally, by servicing both UPS systems during the same maintenance window, we eliminate an additional operation of our maintenance bypass switch thus limiting exposure to a potential voltage drop to datacenter critical equipment.

The schedule for this maintenance is as follows:

07:00 Datacenter Temperature Reduction: We will force the temperature of the datacenter toward the lower portion of the ASHRAE allowable envelope prior to transferring power to the back-up generator. When the transfer takes place our HVAC systems will power-cycle causing a small (2 to 4 degree Fahrenheit) thermal inclination in the datacenter. By lowering the overall space temperature we can be assured that equipment temperatures will not be adversely impacted by the 10-minute restart period of the HVAC systems.

07:00 to 07:30 Generator pre-flight evaluation: Check fluids, connections, air inlets/exhaust, fuel supply, fuel lines, filters and separators. Log results and announce startup to internal staff.

07:30 Generator start-up and warm-up: Start the generator and monitor performance stats, check for fluid leaks, supply artificial load and evaluate voltage, amperage, frequency and engine stats against established baselines.

07:45 FileMaker 3, 4, 5, and 6, Lasso 3, 4, 5, and 6, and legacy Mac shared web hosting environment shutdown: We will begin shutting down the shared hosting environment at this time. The shutdown will be complete by 08:00 hrs and server environment restarts will begin shortly after 08:00 hrs.

08:00 Transfer from grid power to generator power and wrap power around UPS: Manually operate the ATS and UPS maintenance bypass to force the datacenter electrical load to the generator. At this time our HVAC system will power cycle as described above.

08:00 to 12:00 Replace batteries in UPS 1: The UPS system must be completely powered off for life safety while the batteries are removed and replaced. During this operation UPS 3 will remain online and provide UPS power to clients with A+B power.

12:00 to 12:30 Power on UPS system and perform artificial load testing: Following the removal and replacement of components, the UPS systems will be powered on and connected to an artificial load (not datacenter equipment, servers or other infrastructure) for testing. With this equipment we will test the transfer process and mechanism as well as simulate a load of 80% of the UPS system capacity.

13:00 Transfer datacenter load back to UPS 1: Once the process and mechanism has been validated we will transfer the datacenter load back to UPS 1 though power will continue to be supplied to the UPS from the back-up generator.

13:30 to 17:30 Replace capacitors in UPS 3: The UPS system must be completely powered off for life safety while the capacitors are removed and replaced. During this operation UPS 1 will remain online and provide UPS power to clients with A+B power

17:30 to 18:00 Power on UPS 3 and transfer datacenter load: Once the capacitors have been replaced and charged we will transfer the datacenter load back onto UPS 3.

17:45 FileMaker 3, 4, 5, and 6, Lasso 3, 4, 5, and 6, and legacy Mac shared web hosting environment shutdown: We will begin shutting down the shared hosting environment at this time. The shutdown will be complete by 18:00 hrs and server environment restarts will begin shortly after 18:00 hrs

18:00 Transfer Datacenter live load back to grid power: Once all testing has been completed and both UPS systems are online and functioning within specifications, we will transfer the datacenter live load back to grid power.

18:00 to 18:30 Generator Cool-Down and Post-Flight Evaluation: Following the operation of our generator we will perform the same evaluations we performed during the pre-flight as well as allow the generator to cool prior to shutting down.

One to two weeks following this maintenance, we will perform a battery preventive maintenance on UPS system 1 to validate the condition of the new batteries and replace any questionable jars. This preventive maintenance will follow the same procedure as above but require significantly less time to complete. A notice will be posted to our support blog prior to this future maintenance.

digital.forest remains committed to providing our customers with the highest level of service, the greatest degree of protection, and the most transparent communications. If you have any questions or concerns about the above maintenance, please contact your account manager. Our account management staff is available Monday through Friday from 08:00 hrs PST until 17:00 hrs PST at 877-720-0483 Option 2.

Just a reminder, tonight at 11 PM we will be performing an emergency maintenance on part of our electrical system. This will have an impact on a portion of our shared web and database servers, and eight dedicated and colocated servers. The shared hosting servers affected are:

acacia, acorn, alder, aralia, arrowwood, ash, aspen, avocado, balsa, balsam, bamboo, banana, banyan, bayberry, beech, bigleaf, blackberry, boojum, boxwood, bubinga, buckeye, cactus, cedar, cherry, chestnut, cholla, cinnamon, clover, columbia, commerce2, commerce3, cork, cottonwood, db, deerwood, dogwood, ebony, elm, evergreen, ficus, fig, filbert, fuji, gingko, grape, grapevine, hackberry, hazel, hemlock, hickory, ironbark, ivy, kentia, kola, kudzu, larch, laurel, lilac, lime, madrona, magnolia, mango, mangrove, maple, mesquite, mimosa, moringa, mulberry, myrtle, newninewire, ninewire2, olive, orchid, palmetto, papaya, pear, pecan, plum, poplar, privet, quince, redbud, sassafras, savin, sequoia, sherwood, snowberry, spiceberry, spruce, strawberry, sycamore, tamarack, teak, truffula, tutsan, walnut, woodpecker, yucca, yulan/fern

Every effort will be made to minimize the downtime. Our electricians have estimated the time required to complete the task at 2 hours.

Thank you for your patience and understanding while we strive to build a better facility.

Update 1:15 AM: The maintenance went very smoothly and was completed in under 90 minutes. We were able to supply alternate power to the few dedicated and colo servers involved in the maintenance. We also took the opportunity to replace some failed parts in a couple of servers (acorn for example whose fan inside the power supply had stopped working.)

Again, thank you so much for your patience during this critical maintenance interval.

Olive is presently down for a much needed disk upgrade.

Total downtime should be less than 15 minutes; however, FTP access to Olive will be disabled for a longer period of time while we migrate data to the new disk. Service will be fully restored by this afternoon.

12:18 PST
The upgrade is complete. FTP access to Olive has been restored.