Building Test Scripts
How do you know your code is reliable? That it can handle all situations in a known and/or designed way. For a simple test application, you might be happy for exceptions to be thrown which stop the application from running and which allow you to identify what went wrong but what about serious business or mission critical applications? It is not enough to hope they are OK. Do you think you are a good programmer? You cannot rely on that to make your code robust because you simply cannot guarantee that you will not miss something. Another pair or eyes is helpful but again cannot provide enough reliability.
One of my current favourite ideas is to build unit tests in software that instanatiate your objects and call functions on them to test the expected results. This might sound trivial or ineffective but it is surprising how effective they are since they make you think about what the function should do. You will refactor if you cannot think of a simple test because you won't want to write a 500 line unit test function. The trick is to consider what should and could happen and how the system should cope.
A classic scenario is data input fields. Your user needs to type, i.e. a price for an item which will be saved and then used. How many times should this data be checked in the system? You should assume that all external data is potentially tainted so anything coming into the system from the user OR the database should be checked and dealt with.
The most simple case, you might trust the user to type the correct thing in with no real checking. They accidentally type -100 instead of 100. What happens now? In most systems, the data would be strictly valid (but incorrect) and will cause all calculations to be broken. Things like total = cost x quantity will compute without crashing but will generate an incorrect value. OK, you get clever and ensure the user can only type 0-9 and a '.' character. So what happens now when they type 100.12.23? Your system might not notice but this time something will crash (most likely when the number is parsed into a numeric data type). You should actually use regular expressions for most validation so you can be very specific about valid data and can give the regex a grilling with a unit test to make sure it allows all valid numbers and disallows anything incorrect.
Here are some other things you might need to remember:
1) Number ranges: Are you allowed negative numbers? Numbers with decimal places? Do you need to retain leading zeros? Are the values re-displayed (if applicable) in the same format they were typed in? If they are currency, what happens if someone types the currency symbol into the input field? Do you allow commas/periods to separate numbers into thousands? Do your numbers need to consider the user locale and display differently? Will they need rounding (especially if you have divided them by something)? If so, when are they rounded? What happens if you need to compare these rounded figures with other figures? Will they equate to each other or will you need to check merely that they are within e.g. 0.001 of each other? What happens if the user types a massive number that is too large for your number type? Do you restrict this in some way?
2) Strings: What characters are allowed? Will you encode or remove illegal characters or tell the user that they are not allowed? How long can the string be? Do you know what the database will permit? How are you going to ensure that a user will not type in a string that is too long for the database? Do you catch the potential database errors that might result? How do you avoid people injecting SQL into user input? Do you need to upper case or lower case anything? Do you need to spell check anything?
3) Aggregates: There are plenty of chances for error in situations where you are summing a number of items or performing some other calculation. Do you know that each item that should be part of the calculation *is* part of it? If you are updating a page, does it always update for every item that can be changed by the user? Does the sum need rounding? Should the numbers be rounded before or after they are added? Are there any round-trip issues where a number is perhaps multipled, rounded and then changed when attempting to back-calculate the unit price?
There are many things to think about so take your time and let your managers know that there is always a trade-off between quality and the amount of time given for design and testing.
One of my current favourite ideas is to build unit tests in software that instanatiate your objects and call functions on them to test the expected results. This might sound trivial or ineffective but it is surprising how effective they are since they make you think about what the function should do. You will refactor if you cannot think of a simple test because you won't want to write a 500 line unit test function. The trick is to consider what should and could happen and how the system should cope.
A classic scenario is data input fields. Your user needs to type, i.e. a price for an item which will be saved and then used. How many times should this data be checked in the system? You should assume that all external data is potentially tainted so anything coming into the system from the user OR the database should be checked and dealt with.
The most simple case, you might trust the user to type the correct thing in with no real checking. They accidentally type -100 instead of 100. What happens now? In most systems, the data would be strictly valid (but incorrect) and will cause all calculations to be broken. Things like total = cost x quantity will compute without crashing but will generate an incorrect value. OK, you get clever and ensure the user can only type 0-9 and a '.' character. So what happens now when they type 100.12.23? Your system might not notice but this time something will crash (most likely when the number is parsed into a numeric data type). You should actually use regular expressions for most validation so you can be very specific about valid data and can give the regex a grilling with a unit test to make sure it allows all valid numbers and disallows anything incorrect.
Here are some other things you might need to remember:
1) Number ranges: Are you allowed negative numbers? Numbers with decimal places? Do you need to retain leading zeros? Are the values re-displayed (if applicable) in the same format they were typed in? If they are currency, what happens if someone types the currency symbol into the input field? Do you allow commas/periods to separate numbers into thousands? Do your numbers need to consider the user locale and display differently? Will they need rounding (especially if you have divided them by something)? If so, when are they rounded? What happens if you need to compare these rounded figures with other figures? Will they equate to each other or will you need to check merely that they are within e.g. 0.001 of each other? What happens if the user types a massive number that is too large for your number type? Do you restrict this in some way?
2) Strings: What characters are allowed? Will you encode or remove illegal characters or tell the user that they are not allowed? How long can the string be? Do you know what the database will permit? How are you going to ensure that a user will not type in a string that is too long for the database? Do you catch the potential database errors that might result? How do you avoid people injecting SQL into user input? Do you need to upper case or lower case anything? Do you need to spell check anything?
3) Aggregates: There are plenty of chances for error in situations where you are summing a number of items or performing some other calculation. Do you know that each item that should be part of the calculation *is* part of it? If you are updating a page, does it always update for every item that can be changed by the user? Does the sum need rounding? Should the numbers be rounded before or after they are added? Are there any round-trip issues where a number is perhaps multipled, rounded and then changed when attempting to back-calculate the unit price?
There are many things to think about so take your time and let your managers know that there is always a trade-off between quality and the amount of time given for design and testing.