On backups and not saving customer data...

matthew's picture

So I had an interesting week. We're nearing a "freeze" period at work -- a period in which we're allowed to make no major changes to the infrastructure -- and that means an incredibly intense workload as everyone tries to get their changes in before the freeze arrives. Add to that, the hard drive on my web server just up and died.

Now, I wasn't too worried about my data. I've made good backups and taken care of it. But I have a number of webhosting customers on my server, too -- all good friends, it's a small, low-cost, low-revenue business -- who didn't take my "keep your own backups" advice to heart. So I've been running around like a chicken with my head cut off attempting to restore data from a patched-together amalgam of current database backups, legacy data, and online sync when available from the failing server.

At this point, it looks like all the domains but one are fully moved. That last one, alas, is not one for which I had a good backup, but at least I know the web developer who does.

I've learned some important lessons from this disaster, and hope to apply them to better serve my small customer base in the future.

Lesson 1: Never trust your customers.

They may be geniuses in their field, but they are hiring you for a reason: they don't know how to do this stuff themselves. If they knew what was good for them, they'd be doing it. And the truth is, in any moderately technical arena, your customers don't actually know what you're talking about. They just know what they want, and they want you to give that to them.

Lesson 2: Be the hero.

When the shit hits the fan, customers don't want to be told what to do. Customers want you to do it. They just want things back the way they were -- or, more importantly, doing what they pay you for it to do -- in as short a period as possible. They want you to step in, pick up the ball, and get things running again.
Don't waste time trying to work with them or negotiate. Just get the service back up and running as fast as you can, then tell them how it is. THEN you can negotiate the particulars. A customer is more comfortable being told "Hey, we moved, here's your new info" than you asking them what they want in this case.

Lesson 3: Cover your ass.

Sure, you made it plain to your customer what they had to do when they signed up for the service. You were explicit in your user agreements; it really is their fault if they don't live up to their end of the agreement. Well, cover your ass in case they screw up anyway. See rule #1: no matter what you trust your customers to do, they probably won't do it. Or they will do it, but in the worst possible way. Or they just won't understand what you're asking, and will ignore the requirement until the day things fail.
Just figure out every conceivable way to cover your ass, then do every one of those things that you can within your financial and time constraints. At least then you can show due diligence at doing what you didn't need to do in the first place if by some reason you can't hold up your end of the bargain.
But you'll still probably lose that customer. Which is what you want to avoid in the first place.

Lesson 4: Relationships won't keep your customers.

At the end of this, I've simply decided to replace my vendor for web hosting service that I re-sell. The moves were painful but quick, taking place over two days. Forty-eight hours of my life later, and I have no further ties to those guys. They are getting a call on Monday to stop my service, and they can find another customer.

And that last bit is the most important. Offer a valuable service, and do it better and at lower cost than your competitors. I have most of my customers because of pre-existing relationships with them, and they trust my advice and expertise. But I won't keep them because of that relationship; I have to keep delivering value, particularly in times of crisis like a dead hard drive.

--Matt B.