I must admit, these thoughts were not inspired by software projects, although the more I think about it, the more this applies to many types of projects across countless industries, including software.

I was driving to Liverpool from home last week, a journey of 140 miles and which, on a good run, would take about 2.5 hours. It is almost entirely a motorway journey and I usually plan not to stop en-route to save time. It took me 4 hours to get there and 4.5 to get back. Not quite nightmare material but pretty poor nonetheless. The reason? I had chosen to travel on the Saturday and Sunday between Christmas and New Year, clearly a very popular time for people to travel to see the family or friends and the sheer volume of traffic overwhelmed the motorways.

The Problem

A first type of person would simply accept that too much traffic equals congestion - the defeatist. I would like to think that the kinds of people who manage these things however accept that there is a solution - the big problem, and the basis of my argument, is that of the remaining people, most lack any real innovation. They basically believe that money = solutions. Their solutions are expensive to install, expensive to maintain and will life expire probably long before they were supposed to by which point there is no come-back other than to spend even more money.

Then there are the innovators!

A Good Example

Let's rewind to the 1930s and the removal of the tram lines in Halifax, Yorkshire and the resultant problem that car drivers could no longer see the road well at night. It turns out that the silver rails were useful for navigation in the days when street lighting was not universally installed. If most people at the time were asked for a solution, some would have said there is nothing you can do, most would have probably said, "let's install a load of electric lighting" - a non-innovative (but possibly acceptable) solution based on what has already been invented. Percy Shaw however, inspired by seeing lights reflect in the eyes of a cat, produced a relatively cheap and clever device - the "Cat's Eye", which was a simple pair of reflective studs embedded into the road surface and using the car's own lights to reflect where the centre of the road was. These clever devices are still installed largely unchanged from the original (although some idiots have tried to improve them with LEDs and solar power etc. not realising their error) and this goes to show that innovation improves things, most other times the solutions just make things more expensive.

A Bad Example

Let us return to the UK motorways. There are various motorways that at certain times are very heavily used, mostly around London and Birmingham. The M5/M6 junction, particularly, get's very busy as two sets of traffic attempt to merge (just before an exit which probably makes it all a bit worse). What have the Department for Transport done here and elsewhere? They have produced "managed motorways", a new flavour of the month and a horrifically expensive way to try and improve the throughput of the motorways at these busy spots. I would be interested to see how much they help things but let us consider the additional expense.

Basic Motorway

A carriageway, some emergency phones, an occasional overhead LED sign, some occasional hazard warnings lights and either street lighting or cats eyes. Fairly easy to maintain.

Managed Motorway

The hard shoulder is converted into another lane that is used when required to increase throughput. Because of this, the lane markings are now confusing - you have a solid line separating lane 1 and the hard shoulder but you sometimes want people to drive in it - you do this by having loads of overhead signs telling people to use the hard shoulder (in English - so foreigners beware!) you then get slight mess at exits since the traffic might be exiting from the hard shoulder or lane 1 depending on whether the hard shoulder is in use or not. There are a load more overhead gantries with variable speed limit signs - about 5 of these weren't working when I came through (not great for a system that is less than a year old). Because the hard shoulder is now used for another lane, there are additional refuge areas for broken down vehicles, which had to be built. Of course, you cannot assume that a broken down vehicle will reach that far so if someone breaks down, they should be seen in the hundreds of additional cameras that are now fitted above the hard shoulder along its length so that it can be closed again to traffic. All of this requires speed sensors, cameras and more importantly, a control room staffed at various times of day or night so that all this technology can be used as well as it can.

What's the problem? Non-innovative people have thrown money and technology at a problem and have probably made a limited difference to the problem. How can you measure success? Should you even be allowed to spend money if there are no objective success criteria? There are a couple of reasons why this particular scheme doesn't work as well as I'm sure the salesmen said it would. Firstly, we are conditioned not to use the hard shoulder - ever. People are fined for stopping there for any reason other than emergency (and understandably so) so why do you think that putting a sign in English above it will make people comfortable driving into it. I drove down a mile or 2 section where not one vehicle was in the hard shoulder despite the signs telling us to do it. Secondly, The complexity is worrying since it requires a whole plethora of additional signs that drivers are supposed to read and respond to while driving. This is always dangerous but seems to have been missed on the design of these managed motorways. Since most people gravitate to the right of the motorway, there will be loads more undertaking, especially where the hard shoulder is for an exit and someone is shooting up the inside just as another vehicle decides to exit here also. People do not obey the variable speed limits and like most road laws are so rarely enforced, it is almost a joke. If people ignore the speed limits, the throughput cannot be increased by slowing everyone down a bit. You can tell these limits don't work well because when it gets busy, the limit is almost always higher than the speed you can actually drive at.

The worst section was going southbound where the M5 and M6 diverge. Historically, it was a simple 1 + 1 + 1 scheme. Left lane for M5, middle lane for M5 or M6 and right-hand lane for M6. It's obvious, it's instinctive and it worked here and elsewhere. Now we have the wretched temporary hard shoulder, the overhead sign (which is printed so it can't change) now says you have 1 lane only for the M5 and 2 for the M6 so everyone for the M5 is in lane 1. Guess what? When they switch on the hard shoulder, you now have two lanes for the M5 but as mentioned previously, people don't quite get the idea of driving in the hard shoulder so they don't. 2 lanes of traffic now squashed into 1 with a whole load of new signs that people are trying to read. I knew that what they had ended up with is worse than before. I have driven that junction many times and it's simply worse. They added money and no innovation and ended up marginally better in some places (at what cost?) but worse in others.

The Solution

OK, it's easy to criticise a specific example but I think the fact that most people are average and lack not only innovation but even any training or encouragement to be innovative in their solutions. What is the solution?

The answers are specific to each problem but I think the principles probably work across the range of different industries.

1) What is the problem and why? If we really understand this, the first question is how can we avoid the problem in the first place? We accept that roads get busy at certain times but why is this so? Regimented work hours - probably inherited from the Victorians, school runs all over the place, seasonal travel. There are actually ways to deal with these issues - good alternative transport, schemes to car-share, schemes to spread out the traffic either cooperatively or with some system that drivers could use to ensure that they can make their journey at a time they know will be clear (rather than guessing).
2) If you have to manage the problem - this is the worst scenario and to be avoided - how are you going to garner ideas? Do you assume that all the most innovative ideas will come from the government departments or local councils? Do you actually have the desire to source ideas from other countries or people outside the immediate industry? There are things they do in Japan that would work in this country but for reasons that are beyond me, they are not done. Do you encourage your people to think creatively and give them training in how they might do that? Do you have a philosophy that actively discourages ideas that are not in the mainstream of thinking and do you have good reason for this other than just the fear of risk - even at the immense amounts of money these things cost to get wrong?
3) So you have a solution. Are you going to be able to measure it's success objectively? Are you going to hold onto the idea for pride reasons even if it isn't actually value for money? Can you set a timeframe when you will know if it works? Just because it worked in one scenario, do you know how to measure its effectiveness elsewhere rather than just rolling it out? What is the cost of reversing the work if it doesn't actually work out as you thought? Are you going to allow people to critique the solution or have you already decided what you are going to do?

Maybe there is no innovative solution - or at least not one that would probably work so the last important question is, "do we have to do anything?". If you cannot find a solution that seems to make provable benefit to the scenario, are you better off leaving it alone? Let people see that there is a problem so they avoid it themselves rather than you pretending you're helping and encouraging even more people to make the problem even worse?

Software

Since this is a software blog, I guess I should try and anchor some of these points into the software world, although I think most of the principles are the same. You mostly likely have a customer who effectively wants a solution. That customer might even be you if you are a service provider but the whole design process and your solution is in danger of doing what the roads people have done. Their assumption is that the solution is more of the same and yours might be likewise. 

You give the customer a web site with a shopping cart but is that really what they want or need? You give them a framework hacked around to suit - is this scalable and maintainable? Do you understand their business need enough to think creatively about architecture and site layout or are you just going to work on the basis that the better the system, the more it will cost?

Are you going to ask the correct questions of your customer? "How often do you need to change things?"; "How often will the design or branding change?"; "How often do your business rules change?"; "How long is this site likely to be used for?"; "What is the worst case scenario for number of users total and simultaneous?"; "Are you looking for cool or traditional?"