06.01.13

Where do I start. We had a major hiccup during last night’s update – as you will know if you were around to witness the fallout. The database that runs the current checkout software crashed, mid update, and starting throwing out crazy error messages and redirecting customers to another website (I assume we share that database with them). I panicked. Tony panicked. Everyone panicked. I pulled the update down, sent out an urgent request for technical support, and tried to think what would be best to do. To add insult to injury, my laptop crashed, with the Blue Screen of Death, right in the middle of it. I have no idea, it wasn’t tied in to the database crash, it couldn’t have been, it was just terrible coincidental timing.

People, I am not the girl you want in a crisis. I screech and flap and run round in circles. My initial decision was to postpone until Monday night, but this presented problems of its own: my employees can only work on a Monday, so it would have affected them, and also as was becoming evident by some upset comments on Ravelry, some customers had put themselves out to be there, taken time off work, got up in the wee hours. So when I got a call to say everything was fixed, and only ten minutes had elapsed since my postponement announcement, I decided to reinstate the update to 8pm Sunday. It was a judgement call, made under intense pressure. I knew I was risking upsetting people who may have seen the Monday postponement announcement and gone offline. It was a damned if you do, damned if you don’t situation. I did what I thought best. I know that some of you were annoyed.

I’m sorry. I still don’t know what I could have done that would have been the ‘right’ decision. Someone would have been disappointed, whatever I decided.

Technology sucks. Our utter dependence on it sucks. When it fails, we are totally at its mercy. This is the first time in five years of using this system that anything like that has happened, and the most spectacular fail we’ve had in the history of Posh. I got it all sorted out in the end, even though that took until 11.30pm.

We are almost, oh so close, ready to roll out the new system. Of course, any database can crash, and no system is fool proof. The new system has the huge advantage of having Meg at the helm, ready to handle situations like last night. But it is still hosted on my current server, so as far as I know, even if we’d been using the new system, it still would have cracked up last night. Technology is not fool proof. Things happen. And we do our feeble best to deal with the situation, as best we can.

I really do extend my apologies to any of you that were upset, annoyed, or disappointed by last night’s unravelling. And I want to say a huge thank you, with all my love, to those of you who sent me patient, understanding, sympathetic messages, by email, on Ravelry, on Facebook. That helped a lot.

Finally, I want to reassure you that it was just a database issue, the website was not hacked, you don’t need to be concerned about shopping with us. Well, you don’t pay on the Posh Yarn site, and your details are not stored on the database, so you wouldn’t be at risk anyway, but I know some of you were a bit panicky, so I wanted to just reassure you on that score.

3 thoughts on “06.01.13

  1. What an horrific situation. I am glad it was sorted fairly quickly. Hope the shopper fallout hasn’t been to nasty and that you are all breatjing a bit more easily today.

  2. It was just the usual ant bone in the data wire ;-) ). Sometimes this happens in our surgery, too… on a very busy Monday morning…Oh, how I AM sympathetic with you !

  3. Sorry to hear of the troubles. Sometimes servers and databases will crash for no apparent reason. We have problems at work and we have large amounts of people employed to look after them.
    Hugs if you were further stressed out by shoppers. Hope it wasn’t too bad. And worst case, if some get in a snit and stop buying from you it just means the rest of us have a better chance to buy yarn :)