Case Studies | New Rules of Sales & Service
This blog, along with others hosted by TypePad, has been down for an extended period of time due to criminals executing a prolonged DDoS (distributed denial-of-service) attack.
A non-techie definition of a DDoS attack is when a huge flood of Web traffic is sent to overwhelm a website or network. Sometimes the criminals who do this demand ransom to stop, which appears to have been the case with TypePad according to TechCrunch.
A similar attack occurred recently taking down Basecamp. The post mortem about the Basecamp attack is interesting in its details on the attack.
The attack started late last week as the US was entering a holiday weekend. It appeared to have affected all TypePad blogs. The TypePad team were able to restore service to those blogs using a TypePad domain URL reasonably quickly, but blogs like mine, with custom domains, took several days longer to restore.
Early this morning is the first time I’ve been able to access my blog since last week so I am firing off this post with the hope that the problem has been solved for good.
Coping with customer complaints
Throughout the attack, people with TypePad blogs were obviously frustrated. TypePad support staff worked close to 24x7 over the weekend to restore service and to communicate what was happening.
Since TypePad’s own customer support communications relies on the TypePad service, it was tough to get word out about what was happening. Communications was primarily on @TypePad Twitter with thousands of tweets of both a general nature and as specific responses to people, including me, sent in real-time.
What about my data!?!
In the early stages of the attack, I was most concerned about the content of my blog. Would the data be safe? I’ve written this blog on TypePad for ten years and the thought of well over 1,000 blog posts being compromised worried the hell out of me. Scanning the TypePad Twitter feed, it seemed that others had the same worry, and TypePad continually assured people that the data was safe.
As the attack wore on into the 3rd and 4th and 5th day, I just watched and wondered what it was like to be working on the problem and communicating to customers. I cond of got mesmerized by the TypePad Twitter feed as I was thinking about what it must be like to be working there during the outage.
People were obviously frustrated and some chose to vent, sometimes with venom, to the people at TypePad trying to fix the problem. People will be people and I understand this. Some companies rely on TypePad for a Website that represents their sole livelihood. If you run an ecommerce site, imagine seeing your business fall to zero. That would certainly raise the blood pressure.
What would you do if you were working Twitter support full time?
I watched the Twitter feed carefully. Here are some takeaways from how TypePad responded to people:
Consistency - I’d see the same messages such as “We’re going to get you back up and running as soon as possible” pop up frequently in response to people. This was interesting in that anybody scanning the TypePad twitter feed would know the status within seconds. Perhaps they were using TextExpander on the communications desk. That’s what I would be doing.
Keep your cool - The TypePad Twitter feed continually delivered thoughtful and measured responses in the face of what must have been a huge barrage of very negative inbound traffic from customers.
Don’t over promise – Many people demanded to know when the problem would be fixed. Despite being pushed, TypePad never over promised by giving a date or time. This was a smart move because had they said something like “hopefully tomorrow” and missed that line in the sand by several days, they would really frustrate people.
Be human – The responses to people were genuine. People were cheering TypePad and they replied appropriately. Indeed, TypePad deployed humor now and then when they could. This served to calm people and many ended up thanking TypePad staff for their hard work or even offering to send the office hugs and cookies.
Plan for a crisis – I’m not sure if TypePad had a crisis plan in place, but a huge lesson from this is that a major problem can affect anyone. Even you. Especially you. Are you prepared?
