Yesterday started out like a not so fun Friday for thousands, perhaps millions of small businesses using Bluehost, HostGator and HostMonster to host their domain, websites and emails. If you were one of those businesses, then you are already know first hand about the huge outage Bluehost, HostGator and HostMonster experienced yesterday and its impacts.

According to the Bluehost, the website and email outages appear to be related to a scheduled server maintenance at the Provo, Utah, data center and issues with some of their major network switches according to updates provided on twitter and their blog.

No doubt that some noticed sooner than others. I personally noticed when trying to check my email at the beginning of the work day like others whose email is hosted by Bluehost, HostGator or HostMonster, at least those impacted by the outage. That’s a ton of email accounts!

The first sign of the outage announced on Twitter

It became clear that the outage were serious when even the Bluehost.com website was offline and phone calls to the support line went unanswered (has never happened to me in the last 4 years as a Bluehost customer) due to an overwhelming number of support calls from the widespread outage. In fact, the phone rang and then went to a fast busy signal which usually means all circuits are being utilized. One can just imagine the panic and pandemonium then taking place in Bluehost data center and its call centers.

At 4:01 am, it happened. The first public announcement of the outage on Twitter.

FYI we had issues during maintenance last night. We’ve been working through the night to fix things ASAP, stay tuned for more updates.  Bluehost (@bluehost) August 2, 2013

At 9:54 am

@bluehost both of our websites are done and email this is unacceptable! Also can’t load your website to get support’s phone number SpoonDrawer Media (@SpoonDrawer) August 2, 2013

By 10:54 am and several tweets later, this tweet is issued regarding phone support

@jskilling Sorry, our phones have been overloaded with calls from customers. We are working to resolve the network issues.  Bluehost (@bluehost) August 2, 2013

By 11:48 and a few dozen tweets later,

FYI: Chats/Phones are impacted by the network issues. Best to follow us @bluehost and facebook for updates until things calm down. Thanks!

Bluehost (@bluehost) August 2, 2013

By 2:56 pm and several hundred tweets later, this tweet

FYI this issue affects other brands of our parent company, so we launched a central website to provide ongoing info http://t.co/rCqBSlTesE  Bluehost (@bluehost) August 2, 2013

Twitter quickly became the primary way to receive near real terms updates from Bluehost on status (a sign of the times), with update tweets to individual users being posted every few minutes before redirecting users to a blog set up by the brands parent company, Endurance, who eventually set up a blog to provide Official Updates from the Endurance International Group on the outage.

Here you will find the official, public details currently available. Can you imagine the opportunity cost to Bluehost, HostGator and HostMonster for having their websites down for an extended period of time not that many reading this will necessarily care about that aspect, but as a business owner, you can rest assured that these companies were feeling our pain and throwing every resources to deal with what is known in the communications industry as a Sev 1 or Severity 1 Support issue – i.e. hugely business impacting for many.

Valiant Effort by Bluehost Support

Kudos to the Bluehost team for pumping out 635 tweets during this incident providing personalized, transparent access to information through a channel that many of its clients use. To provide some perspective, at the time of this post, the Bluehost twitter account was 479 days old with 3688 tweets during the first 478 days or an average of just under 8 tweets per day, then one day later, 635 tweets within 24 hrs all on providing information and feedback to individuals during the outage.

That should give you an insight into the breadth of impact and Bluehost’s dedication to addressing and providing information/transparency to its users. Creating the blog for official updates was the only way to put a lid on this information overflow while providing a consolidated point of updates for all the Endurance brands beyond Bluehost.

eBiz ROI has been a loyal, satisfied Bluehost customer for nearly four years, having hosted eBiz ROI’s business website and blog and client’s domain, email and websites there successfully. The Choice to Use Bluehost as a Domain Registrar, Website and Email Host was Driven by Due Diligence finding that:

1. Bluehost  has a reputation for providing tremendous customer service,
2. Highly rated by the BBB (Better Business Bureau) A+ (report)
3. Their size as measured by the number of domains/website they host in the millions according to the bluehost.com website

For more information on choosing a web host, see  Top 5 Tips for Selecting a Web host.

Bluehost impact of Widespread Outage

There Bluehost website claims “Over 20,000 New Customers a Month & Hosting Millions of Domains!” on one of their sales pages. That’s a lot of folks impacted on Bluehost alone. If you do the math, Bluehost lost 667 new customers yesterday (20,000 per month / 30 days per month). That does not count cancellations or renewals or even those performing due diligence that were going to buy today or tomorrow.

Now to be fair, websites on Bluehost, at least the eBiz ROI clients and email, where accessible intermittently throughout the day. Emails that were not accessible when  sent  were queued and eventually delivered (no lost data), but the impact to businesses beyond Bluehost was significant to say the least.

Reliability you Say, SLAs?

Bluehost, HostGator and HostMonster all support what is called in the industry as 3 9s of reliability and the terms of service are based on providing this amount of uptime. That means 99.9% uptime each year. When you are hosting, or a hosting customer, your business is based on uptime, just like brick and mortar stores are based on access to their stores being open to conduct business. Just ask the folks of the bridge that was demolished between NY and Vermont a few years back.

If you do the math, 99.9% uptime, which is the typical SLA (Service Level Agreement) for web and email hosting, this equates to 8.76 hours of allowed downtime per year. The math works out like this: (.1% * 24 hours * 365 days in a year). Depending on how uptime is measured yesterday, clearly the hosting outage at Bluehost, HostGator and HostMonster used most if not all or more of this annual SLA downtime allowance, typically reserved for maintenance windows that typically result in minutes, not hours of downtime.

Will Bluehost, HostGator and HostMonster Compensate their Customers for Epic Outage?

From a business perspective, it is almost impossible to see how they can get away with not doing so. Their collective businesses where impacted in the following three ways yesterday:

1. Loss of new business as their websites (sales portals) were down a good part of the day yesterday
2. Loss of renewals as many whose hosting is just coming due are likely shopping today
3. Cancellations of orders requesting a refund due to extended outage

In addition, it’s possible that some Bluehost affiliates will rethink their participation in the program due to potential impacts of brand association.

What I find interesting from a customer relations perspective, is that Bluehost (or Endurance) has not yet, at the time of this post, sent any official email communication to their customers (or at least eBiz ROI) with an official apology. Hopefully they are working on now. This level of outage effecting so many for so long deserves some kind of acknowledgement and some kind of compensation in order to maintain existing accounts and attract new business.

Bluehost, Hostgator and Host Monster Not Alone – GoDaddy Experienced Widespread Outage in May 2013

Before you jump ship, realize that this kind of widespread outage has happened before and will happen again somewhere. GoDaddy had experienced a large outage this past May 2013 and performed a detailed GoDaddy Site Outage Investigation with the findings published here. I strongly recommend that Bluehost do the same. For their own benefit and the benefit of their customers.

Many Bluehost renewals are coming due for eBiz ROI and it’s Internet marketing clients, each of which we will confer with to determine if they want to renew with Bluehost or shop around for other options. This is what we will tell them:

“Our experience with Bluehost over the last 4 years has been highly positive and our recommendation is to wait to see what the official report issued from Bluehost / Endurance says about what happened and the steps to be taken to avoid outages of this magnitude in the future before throwing the baby out with the bath water. “

Web outages by the large web hosts are unfortunate and will no doubt happen again. It’s how those situations are handled from a customer service and PR perspective that can make or break a businesses and help users decide if they will stay or go. My money is on Bluehost and by extension, Endurance stepping up to the plate and taking care of their loyal customers.

Will see.