Those of us who write software know that we make mistakes. Developers don't consider specific scenarios, Testers miss certain tests and with the best will in the world, even the simplest applications can have bugs.

BUT

The main happy paths should work largely fine. If something goes seriously wrong for a large-scale public-facing web site, one of a number of things absolutely must happen:

  1. Ideally, the company will already know because they will get an error message emailed/displayed on a big screen/whatever
  2. If it is more subtle, maybe a user will contact the company and if this happens, it is embarrassing, so you act immediately, especially when the bug relates to a happy path that you ABSOLUTELY should have tested
  3. If it is something with a non-obvious workaround (or none at all) the Development Team make it number 1 priority and work flat out, 24/7 if required, until it is fixed. Why? Because it was a screw up that something so serious got out the door and it is a matter of quality and corporate pride that it gets fixed and quickly.
  4. The Test Manager gets a serious talking to along the lines of, if this happens again, you're fired.
  5. The Technical Team has a serious review about how this was allowed to happen and puts in place real measures to prevent a repeat the next time. This is fed back to the Management Team so that people can be accountable where they need to be - the Management Team need to ensure they are getting the whole truth, not just what someone might say to cover their own back.
What isn't OK is:

  1. Not putting any kind of banner on the web site to say that you are experiencing problems
  2. Not working with whoever found the problem to quickly work out exactly what has happened and why
  3. Telling users to delete their cookies to make it work
  4. Telling users that only some users are having the problem (as if that makes it better that it's broken for me)
  5. Not properly testing updates to consider not just the new site in a clean happy place, but what happens when a user with existing cookies and a number of different browsers comes back to a new site.
  6. Acting like a serious bug report from an end user is just business-as-usual rather than, "I'm really sorry, I'm just going to call the Software Manager to tell them" or even, "We know of a problem and the Team are still trying to find exactly what causes it".