Tuesday, March 13, 2007

Tips for automated Testing.

  1. Never write calls that directly interact with the product you are automating, write an interface layer. This way when the product changes you will only have to change a limited number of lines rather then the hundreds that it would be if you have a line clicking a specific link in every script.
  2. Automate the running of the scripts as much as possible. Make it so all you have to do is click a button or edit a file this way re-batching the scripts is not that big of a deal.
  3. Never have a literal string in a function call. Even though you pushed every call that actually interacts with the program that you are testing. You will still have the same string called from several locations. When this changes all you have to do is change one string.
  4. Never have a navigation function that validates a page. Navigation functions get called more than any other set of functions in a test framework. Putting validation code in them will slow down every script that uses the function. It is much better to have a separate validation function you can call when you really want the page to be validated.

  5. When a string literal can be more than one value check for all values and act accordingly, don't just do all possible actions at one time. Even though it seems easier to do this it will take longer and you spend more time handling error conditions.

  6. When dealing with errors handle them as soon as they appear. If it is an expected error clear it and go on your way. If it is not expected clear the error and do your best to clean up the script work area. Cleaning up all the data and the program before exiting the script. This way you don't kill the following scripts chances of completing successfully.

Tuesday, December 12, 2006

Boundary Value Testing

Boundary value testing is an extension of equivalence class testing. But rather than testing just one of the values in the class you test the upper and lower values of the class as well as the value above and below that value. Since most programs have a significant behavior change at boundaries it is always good to make sure that you have the correct behavior at the boundaries. This is more thorough than equivalence class testing and will find a lot of the off-by-one errors in the product.

Some steps to setup Boundary value tests.

  1. Identify the equivalence classes.
  2. Identify the boundaries of each equivalence class.
  3. For each boundary, write one test case for one unit below the boundary, on the boundary, and one unit above the boundary.
  4. The one unit above and below may create duplicate tests. If this is the case just run the test once, and note the results.

Some of the positive aspects of this type of testing are:

  1. You focus on problematic areas in the system.
  2. You cover both valid and invalid cases.
  3. You have a relatively small test set.
  4. You can use this style of testing with non-functional requirements like performance and capacity.

There is only one problem with Boundary Testing is that in some cases it is hard to come up with appropriate boundary values.

Boundary value testing is rather simple and is one of the few testing strategies that has more good things to say about it then bad.