The Triangle Problem
A lot of people who do not work with software wonder why software has defects in it. To the end-user, something like Microsoft Word looks simple enough, why is it that in a certain scenario it locks up or crashes?
It would be easy to say that people who write software understand why and are able to avoid these situations with the correct processes but sadly this is not usually true at all. Even very able and competent engineers make mistakes and, of course, we expect the less competent engineers to make mistakes.
I came across the following example in a computer magazine once and apparently it is a classic testing question:
q) You need to write a function that takes 3 inputs and determines wether they form an equilateral, an isoceles or a scalene triangle or displays an error if the inputs cannot form a triangle.
Spend a few moments and see how many tests you could think of to determine the type of triangle.
Ideally, the functionality for this would be specified in a very robust way by somebody who is both a good spec writer and has done the necessary research to ensure all bases are covered! Back in the real world, you would be very lucky to get a spec for this level of functionality and even if you did, you would still have to interpret it.
When I first heard the problem I immediately thought that all values would form a triangle thinking of the phrase "any three points make a triangle" (which of course they don't if they are on a straight line).
After some hints from a colleague I started thinking about inputs that are negative or zero and but still forgot about what happens if 1 side is much longer than the others and therefore the triangle can't close. I didn't think about what happens if the 3 inputs are not specified correctly, such as 23E04 or what happens if the input is mistyped like 23..56 or if the inputs are specified to too much precision or whether the inputs are guaranteed to be within the limits of the specification. I think I managed about 6 tests out of a potential of 15 and I thought I was pretty clued up!
|Being the engineer, I then got really concerned that this was the real-life experience of many software engineers. When you write code you are assumed to be able to guess what error checking etc is required in your code and implement it. Of course, most of the error combinations would probably never surface but what if they do in a way that is hiddden from the end-user. Something gets corrupted and crashes?
The real question is, what can I do about it? Well you can have a committee to discuss what tests you need to perform but this is unlikely to reveal much more than you would by yourself (big quiz teams don't always win first prize!) so what else can you do?
You can write Unit Tests as you go along. You can test functions using test cases that give you very good overall coverage of the code. You can spend more time on functions that use logic rather than functions that simply call a string of other functions sequentially. You can keep the test cases as a reminder for when you are testing newer code. You can share tests with other developers so that over time, you gradually increase the types of error conditions that you test for. It is an underused mechanism for reliable code.
You can develop shared controls. For instance, you could design a web control that only allows valid numbers to be typed in so that your functions are never passed malformed data and don't fall over. You can even create simple classes like strings etc that can perform earlier testing on data before it has a chance to upset the belly of your code.
At the end of the day, the issues need to be analyzed and something needs to be put in place to keep the closed-loop quality control doing its job properly. You need to learn if your counter-measures are working and either modify them or drop them if they are not. You need to listen to other people and what they use. You need to take your craft seriously.
It would be easy to say that people who write software understand why and are able to avoid these situations with the correct processes but sadly this is not usually true at all. Even very able and competent engineers make mistakes and, of course, we expect the less competent engineers to make mistakes.
I came across the following example in a computer magazine once and apparently it is a classic testing question:
q) You need to write a function that takes 3 inputs and determines wether they form an equilateral, an isoceles or a scalene triangle or displays an error if the inputs cannot form a triangle.
Spend a few moments and see how many tests you could think of to determine the type of triangle.
Ideally, the functionality for this would be specified in a very robust way by somebody who is both a good spec writer and has done the necessary research to ensure all bases are covered! Back in the real world, you would be very lucky to get a spec for this level of functionality and even if you did, you would still have to interpret it.
When I first heard the problem I immediately thought that all values would form a triangle thinking of the phrase "any three points make a triangle" (which of course they don't if they are on a straight line).
After some hints from a colleague I started thinking about inputs that are negative or zero and but still forgot about what happens if 1 side is much longer than the others and therefore the triangle can't close. I didn't think about what happens if the 3 inputs are not specified correctly, such as 23E04 or what happens if the input is mistyped like 23..56 or if the inputs are specified to too much precision or whether the inputs are guaranteed to be within the limits of the specification. I think I managed about 6 tests out of a potential of 15 and I thought I was pretty clued up!
|Being the engineer, I then got really concerned that this was the real-life experience of many software engineers. When you write code you are assumed to be able to guess what error checking etc is required in your code and implement it. Of course, most of the error combinations would probably never surface but what if they do in a way that is hiddden from the end-user. Something gets corrupted and crashes?
The real question is, what can I do about it? Well you can have a committee to discuss what tests you need to perform but this is unlikely to reveal much more than you would by yourself (big quiz teams don't always win first prize!) so what else can you do?
You can write Unit Tests as you go along. You can test functions using test cases that give you very good overall coverage of the code. You can spend more time on functions that use logic rather than functions that simply call a string of other functions sequentially. You can keep the test cases as a reminder for when you are testing newer code. You can share tests with other developers so that over time, you gradually increase the types of error conditions that you test for. It is an underused mechanism for reliable code.
You can develop shared controls. For instance, you could design a web control that only allows valid numbers to be typed in so that your functions are never passed malformed data and don't fall over. You can even create simple classes like strings etc that can perform earlier testing on data before it has a chance to upset the belly of your code.
At the end of the day, the issues need to be analyzed and something needs to be put in place to keep the closed-loop quality control doing its job properly. You need to learn if your counter-measures are working and either modify them or drop them if they are not. You need to listen to other people and what they use. You need to take your craft seriously.