For the past several weeks we've noticed a growing problem in delivering mail to yahoo.com email addresses. On all of our servers, at any time of day or night, the mail queues going to yahoo.com are backlogged. It seems Yahoo! has taken the extreme step of "greylisting" all inbound mail in order to avoid spam. They are basically slowing down the rate at which they accept mail, and throwing up error messages telling our server to try again later. Our servers then have to wait some prescribed amount of time before trying to send their queued mail. In the meantime more mail queues up for delivery. When the prescribed time arrives, our server tries again, and is able to deliver a small portion of the queue, when yahoo.com's mail servers once again call a halt and ask to retry later. In the meantime even more mail queues up for delivery to yahoo.com.

lather, rinse, repeat.

This means that at any given moment there are hundreds, if not thousands of mails queued on our servers for yahoo.com, and they aren't willing to take them at a good enough rate to relieve the backlog. It is like a short-cycling stop light at rush hour, except here rush hour never ends.

To add injury to insult, they are being very critical of message contents. many of our customers have reported large numbers of mails being returned as undeliverable with error messages such as:

message text rejected by mx1.mail.yahoo.com: 451

We have no idea what criteria yahoo.com is using to determine what text they are rejecting. The messages we've had forwarded to us looked very "un-spammy"... no styled text, no HTML, no mentions of investment advice or pharmaceutical life enhancements. We're as confused and helpless as you are.

Yahoo.com has no human beings we can talk to in order to resolve this issue, unlike most ISPs. As an autonomous network operator, we often can pick up the phone and call other ISPs and network providers and ask to talk to the people who run mail ops, and actually talk to them. We realize that might be an odd concept in this highly networked and automated world, but it still holds true. We attend technical conferences and meet with our peers, and exchange business cards. We help each other out when bad things happen. We make friends and call in favors. In most cases, if there is a problem between two networks, you can find the person on the other side who can work with us to fix it.

Except with Yahoo.

They are seemingly impenetrable, at least with regards to mail operations. They have some web forms to fill out and hopefully start the process (we can completely understand their desire to hide from user-level support issues since they make no revenue from those users, but they shouldn't hide from network peers!) but the web forms go nowhere and apparently don't do anything.

We're not alone in this. Others have the same problem. While it is a bit comforting to know that we aren't being singled out for poor treatment, it does nothing to help the mail get moving again.

As yahoo!mail is a free service, it is a classic case of "getting what they are paying for."

If you are a digital.forest client and you are FORWARDING your mail to a yahoo.com account, we strongly suggest that you stop doing that - many of you do, which is only adding to the problem. You are likely not getting your mail at all, or if you are, it is VERY late. We suggest you configure your mail client to check your mail directly on our servers. We support POP, IMAP, and webmail. Our servers have a very configurable webmail that you will probably like just as much, if not more than Yahoo's.

If you are a digital.forest client and you are trying to send mail to an address in yahoo.com, we suggest that you contact those people another way, and get an alternative email from them, as the same situation applies, they are not getting your mail, or at the least not in a timely fashion.

If you are a digital.forest client and you maintain mailing lists for communicating with your customers, we strongly suggest that you make every effort to keep those lists "clean" and remove addresses that bounce or cause problems like this. If you are mailing to non-existing addresses then you are clogging mail queues with useless, time-consuming items that only serve to create or enlarge an existing problem.

We'll keep an eye on this, but we suspect it won't be getting better anytime soon. If it does, we'll update this site. Thanks for your patience.

--Chuck Goolsbee
V.P., Technical Operations
digital.forest

UPDATE: 10-26-06 16:19 - Mail issues have been resolved.

We are currently experiencing an issue with our primary mail server. Technicians are working on the server and expect a resolution within the hour.

It has been a while since we updated the support pages with news of our datacenter expansion. Our first client for the space experienced a month delay, so work was paused for a while in September. They are now moving in November 1st, so the finishing touches are being applied. Here is a familiar view, taken from the northwest corner:

It is a composite panorama shot, with the first client's area at the left, their cage walls being constructed around them, the rest of the colocation space around it, in the foreground, and all the way back to the wall, Our Power Distribution Unit near the far door, and the ladder rack extending down the right side of the frame. Our existing datacenter is beyond the double doors at the far right.

Behind the photographer is a ~900sq' private datacenter suite. Beyond that is another ~1200sq' private datacenter suite, or perhaps subdivided cage suite area that will be developed as soon as we fill the existing space.

We have enough available electrical power to power several hundred racks at our current usage level, and the building can support cooling capacity between 109 to 130 Watts per square foot. These are amazing numbers, and we are thrilled to have found a facility with this sort of expansion capability. The work above is just the start of what digital.forest can grow to become.

Above: Facilities Manager Damian Amrhein installs seismic supports to the cage walls being installed.

In other news, we are offering a new line of server enclosures from APC. These units have some very nice features with regards to airflow, power distribution, and cable management. Shown below are the cabinets chosen by our latest client. They were delivered earlier this week and will be bolted to the floor over the weekend. They are very nice units.

Space is going fast, with about 40% of it already spoken for, so if you need datacenter space in the Seattle market, give your digital.forest sales professional a call today at 877-720-0483, option 2.

--Chuck Goolsbee
VP, Technical Operations
digital.forest

During our scheduled maintenance window we will be applying Windows updates to our servers. These updates will require server reboots.

If your server is down, wait a few minutes and check again. If it is still down you may call our support line @ 1-206-838-1630 option 3 (toll free 1-877-720-0483)

Reboots will occur between 11:00 pm and 2:00 am

The hosting server Titan is currently down for maintenance. We expect to have all of the sites operational by mid-morning on October 10th.

The streaming server Columbia has experienced what appears to be an OS failure. We are currently working to restore service on this server and will have an update by 12:00 PM on Saturday, October 7th.