Boundary Value Testing

Boundary value testing is an extension of equivalence class testing. But rather than testing just one of the values in the class you test the upper and lower values of the class as well as the value above and below that value. Since most programs have a significant behavior change at boundaries it is always good to make sure that you have the correct behavior at the boundaries. This is more thorough than equivalence class testing and will find a lot of the off-by-one errors in the product.

Some steps to setup Boundary value tests.

1. Identify the equivalence classes.
2. Identify the boundaries of each equivalence class.
3. For each boundary, write one test case for one unit below the boundary, on the boundary, and one unit above the boundary.
4. The one unit above and below may create duplicate tests. If this is the case just run the test once, and note the results.

Some of the positive aspects of this type of testing are:

1. You focus on problematic areas in the system.
2. You cover both valid and invalid cases.
3. You have a relatively small test set.
4. You can use this style of testing with non-functional requirements like performance and capacity.

There is only one problem with Boundary Testing is that in some cases it is hard to come up with appropriate boundary values.

Boundary value testing is rather simple and is one of the few testing strategies that has more good things to say about it then bad.

Equivalence Class Testing

Today I thought I would talk about Equivalence Class Testing.

Equivalence Class Testing is when you create sets of input that have equivalent expected behavior. You do this until all possible inputs show up in one of the groupings. Now when you test one item from each of the inputs you should be covering all the possible inputs in that grouping.

Some types of inputs lend themselves really well to this type of testing – the first is a range of values. You break it down into values within the valid range, invalid values above the valid range and invalid values below the valid range. Lets have an example to better illustrate what I am trying to describe. We are going to test the appointment-making part of the program. An appointment has four fields:

1. Time
2. Day
3. Month
4. Year

Lets look at the month input field. Depending on how it is implemented it could take valid values from 1-12 or it could take the months in a string format. Lets look at the input of values from 1-12 and what equivalence classes world be necessary to test this input.

• valid range of 1-12
• invalid range below the valid range <1
• invalid range above the valid range >12

If the input was a string you would end up with two sets. One that had the valid inputs in it and one that had every thing else you could think to put in it.

So along with the month input you would also have a day, year, and time input. The program is written in such a way that they all have to be tested at the same time. So you list all the groupings for all the other inputs fields and test them at the same time. You will notice that you can test most of the valid test cases at the same time, but you will have to test the error cases one field at a time so you can make sure the error is handled appropriately.

This kind of testing will allow you to keep the number of test cases small, and you could use this type of testing at any level of testing from unit all the way up to acceptance testing. One of the limiting factors of this type of testing is that it assumes identical behavior for all items in a grouping. This is best suited to testing programs where inputs are within ranges or values in sets.

Watching the file Access

This week I stumbled upon a bug that was quite difficult to reproduce. It had to do with file locking and two separate processes competing for the file. I thought that this might be the case but the developers were adamant that it was not. So, where I work, if a developer says you are wrong, you are wrong until proven otherwise. Since I knew I was right – or at least thought I was strongly enough to take the time to prove it – I started to look for a way to see what is happing to a file.

I needed to be able to see which process was opening what files at what time. So I went over to the IT team at my company and asked if they had such a beast. They proved yet again that they can anticipate my every desire and had such a program installed on the computer I wanted to use it on. So I spent about 20 minutes setting up the scenario and logging the results with the program and had the information to prove the developer wrong. It felt so good.

The nifty little program that has now been added to my list of must-have programs can be found at. http://www.sysinternals.com/Utilities/Filemon.html It takes a little work to get it to do what you want but everything worth having does. I would recommend it to all testers as well as developers

7 ways that input can break a web page

Since this blog is supposed to be about testing I thought I would share some of the things I have learned about breaking a web site. The following are 7 areas I always test on a new input field in a product. All of them are things that a developer should have known better than to do but every product I have worked with gets at least one of them wrong.

Query String special charters

If you have a forum that pass values using get rather then post. The & can cause several problems. If a man was to enter in their name and their wife’s name in to a field named fName. An Example of this would be Tom & Jennie. The information passed on the query string would then look like …&fName=Tom+&+Jennie&lName… in the query string. The fName attribute would only get Tom in this case and depending on what system was parsing the backend lName might be broken as well.

An unencoded = and ? also have the potential to cause the same problem but are much less prevalent in usage. Still they should be tested but are not as critical as the &.

Another charters that can cause a problem when using a query string rather then a post on a web forum is the +. The + sign is a marker for a literal space in the string. If you try to pass it unencoded in a query sting it will go away. Which can cause some problems but it doesn’t usually break the processing of the rest of the forum like the other 3 have the potential to do.

Terminating html where it shouldn’t be.

One of the more entertaining things you can do to a web page is to break the page by closing html tags before they are supposed to. The way you do this is by passing in a >, ‘>, or “> depending on how values are encapsulated in the tag. This error usually shows up in cases where the user data is redisplayed to the user in the same format. A good example of this is when you forget to enter a value in a forum and the page is redisplayed to you. When it happens you will see > displayed in the html after the input field. Another case is when the data you pass in is stored on the next page as a hidden input field. This is one of those things that doesn’t kill the use ability of the page as much as it makes you look stupid when it happens.

Cross site scripting

When I was a wee little brat well truth be told last year. I took great joy in posting java script and images to the main page of a friend’s web site through the title of the post I was making. I also made the entire page blink using the blink tag. These things are all lumped under cross site scripting. The annoyances that can be caused by this can range from the simple annoyances like what I was to doing to very dangerous things like a person collection your users login and password. The solution to this problem is actually the same one you would use to fix Terminating html you either don’t allow <> in your input fields or you encode them so that the browser will not think of them as html.

Over loading the buffer

First thing you might ask is what buffer are you talking about? Simple answer to this is yes. I am talking about any buffer you can think of. Some times the database that stores the data limits the number of chars you can pass into it. Other times it is the buffer of the web server for processing the query string. This buffer is very large and takes a bit of effort to over load but always is fun to watch the results after words. The way you test this is you pass in the maximum number of chanters you can into the field. Good developers usually limit this but some times they forget.

SQL commands through input.

Most times text is entered on a web page it passes through a database. So one of the things you need to watch for is if you enter a SQL command in the text field it doesn’t get executed in the product. This is a test that is a little harder to test. One of the ways you can test this is by putting SQL string terminators in after some text then a SQL command. I will discuss this more in the next section.

The mighty ‘ and “ how they can mess up a page.

Two charters that can cause more trouble then any other for web sites. They reason for this is they can reek havoc from the html level all the way down to the database. One of the is always used to denote test with. We mentioned them in conjunction with the > in the section of terminating html. You can also place them with other text and sometimes watch the text after it disappear when it is displayed in an input field again. A unecnoded ‘ is one of easiest ways to cause a SQL statement to get executed when it wasn’t supposed to be.

HTML escapes and how they can make all other problems possible.

Now we get to talk about HTML escape charters. These are things like &nbsp; and &#44; and the like. These can be used to display text that should not be displayed in on a page. In some cases I have seen them used to do cross site scripting. The program decoded them before it displayed the html on the page and it executed the JavaScript commands. This is one of those places where you have to decide if it is ok or not to allow them in the product. Some places don’t think it is worth it, and have JavaScript that will always force a space after the & before posing it back to the server.

Well I hope this helps you in you endeavors to create better software. If not at least it might have showed you a way you can mess with a web master out there and have a little fun.

I spend a lot of time testing software, and writing automated tests to test software. I figure I should put some of the things I have learned on line just for the fun of it. Waite and watch one of these days you will see code and my thoughts about testing appear on this very page.