Proper Black Box Testing Case Design – Equivalence Partitioning

In today’s IT world, the lines between developers and QA Engineers are being blurred. With the emergence of Agile, Test Driven Development, Continuous Integration, and many other methodologies, software testing is becoming even more critical.

To support daily releases, multiple Operating Systems, and multiple browsers, the Development team (QA and Software Engineers) needs the capability to create test cases faster while increasing test coverage. Therefore, learning how to design test cases in the most efficient manner is becoming a critical skill in the IT industry.

There were a few occasions when not knowing these skills has come back to bite me in the wallet. I actually missed out on a few amazing opportunities because my skills weren’t up to par. However, one of my favorite quotes is, “Losers make excuses and winners make progress”.

Therefore, rather than sulking in pity, I take my weaknesses and improve upon them. Turning them into strengths. I spent countless hours of research, reading, and practice to improve. Today, as a result of my failures, I have more knowledge and power to deliver higher quality software.

I want to teach you the best way to design test cases, so that when your dream job comes along, you are prepared to show off your testing skills and impress them.

The ultimate goal with all of these testing techniques is to make you more efficient at writing tests. I will teach you how to write less test cases, but still have higher testing coverage.

Make sure that you actually do the exercises that I provide and then apply these skills to your work. Research shows that when you apply something that you’ve learned immediately, the retention rate drastically improves.

This post will be the first of several in this series that will focus on test case design using Black Box Testing techniques. In this post you will learn:

1. What a well designed test case is.
2. All about Black Box Testing.
3. Equivalence Partitioning for test case design.

I love to add quotes because they are usually created by wise individuals for the purpose of inspiration. I want to inspire you before we begin our lesson.

The design of tests for software and other engineering products can be as challenging as the initial design of the product itself. Yet … software engineers often treat testing as an afterthought, developing test cases that ‘feel right’ but have little assurance of being complete. Recalling the objectives of testing, we must design tests that have the highest likelihood of finding the most errors with a minimum amount of time and effort.

(Copeland)

Key Point: To be most effective and efficient, test cases must be designed, not just slapped together.

What is a well designed Test Case?

Let us take a moment to lay the groundwork for better testing practices. I will briefly introduce a few important concepts. Afterwards, we will dive so deep into black box testing that you will be able to test any object in the world.

An awesome test case is composed of three parts:

1. Inputs
2. Outputs
3. Order of execution

Inputs – This is the data entered into a system

Outputs – This is the data that is displayed, written, or sent

Order of Execution – The order of the steps for the execution of the test case

You can either create cascading test cases or independent test cases.

Cascading test cases are those in which each test case can flow into the next one. For example:

1. Open the browser and navigate to the home page
2. Login to the site
3. Change membership level

Each of these steps can be a single test case.

The advantage is that these test cases are smaller because there is less repetition of steps.

The disadvantage is that when the “Login” fails, you cannot test the “Change membership level” step because it is dependent on the former.

Independent test cases are as simple as the name implies. They allow you to test the functionality, independent of other functionality. In the previous example, “Login to the site” can be tested, regardless of whether the home page is present or not.

The advantage of this is that these test cases are independent and may be run in any order.

The disadvantage is that they become overly long to write and execute. We need to execute repetitive steps in each test case to get to a specific place that we want to test rather than chaining multiple test cases together.

In automation, this is the preferred method that you should follow and the method that I preach. It allows for independent testing of components, even if all the other components around the application are broken.

Types of Software Testing

There are several types of testing methodologies:

Black box testing (BBT)

This is a technique in which testing is solely based on the requirements designated for software design. Black box testing requires no knowledge of internal paths, structure, or implementation of software under test. This type of testing is what we are going to cover thoroughly over the next several weeks.

White box testing (WBT)

The antithesis of black box. This type of testing actually tests based on the understanding of the system. This type of testing requires a detailed knowledge of programming. Unit testing is an example of white box because you actually have to know the code to write tests for it.

Gray box testing (GBT)

Again, the name is self explanatory. You check inside of the box to understand how it has been implemented. Afterwards, you close the box and design even better black box tests (Copeland).

Black Box Testing

What is black box testing in software testing?

Black Box Testing techniques refer to the fact that a developer cannot look into the system under test (SUT) by looking at internal paths, structure or implementation. The developer must think of the system as a black box, about which they know nothing. You can only test based on what you see and the requirements presented to you.

At what development levels can black box testing occur?

Black box testing can occur at all levels – unit, integration, system and acceptance. As we move from each level, we are actually forced into the black box testing because at some point, the SUT has too many potential use paths.

What are the disadvantages of Black Box Testing?

A developer can never be certain about how much of the application was actually tested. The amount of inputs required to test all possible scenarios is too big and you can only choose a subset. Therefore, the goal is to efficiently design a subset of the most effective test cases to test the system.

What are the advantages of Black Box Testing?

Formal BBT techniques are still far more efficient than random test case creation. Following black box techniques will allow you to create better test cases, faster. Thus, using these tests makes you a very valuable asset to your company.

What are Black Box Testing techniques?

1. Equivalence partitioning
2. Boundary value analysis
3. Decision Tables

What is Equivalence Partitioning in software testing?

Equivalence class partitioning (EP) is a very widely used method to decrease the number of possible test cases that are required to test a system. The idea behind EP is to split all of the possible inputs into something known as equivalence classes. These equivalence classes represent a range of inputs that work exactly the same way in the system.

Consider the following situation:

Test Scenario

A text box allows the following integers to be entered:

1 – 5 Success

How would you test this text box? Should we enter every single digit from 1 – 5 to make sure that the system works as expected? If we did, it would give us at least five test cases for the following system. However, such a testing technique requires way too much time and is too boring.

In software development, we always run into the problem of not having enough time to test. Therefore, we need to test efficiently.

Lucky for you, most developers would implement this system in an efficient manner that allows you to test faster. They would use their programming logic to write something like this:

Take some time to think about this implementation…

You will see that we do not need to test every single integer, because everything between 1 – 5 inclusive will be treated exactly the same. There is no other logic in the code to specially handle 2, 3, 4, and so on. Hence, you can handle the testing of this system with only three test cases instead of five by creating the equivalence classes that I spoke about above.

The equivalence class is a set of data that is treated the same by the module and any data within this class is equivalent.

You can expect that:

1. If one test case from an equivalence class can detect a defect, then all the other test cases in that same equivalence class will also detect the same defect.
2. If one test case does not detect a defect, no other test case in the equivalence class will detect a defect (Copeland).

EP can be applied at any level of testing, like unit or integration. And it can be applied early in your test case design.

What are the advantages of equivalence partitioning?

EP is a great technique for reducing the number of test cases required to test a system, while allowing for a greater testing coverage of the system.

What are the steps to designing Equivalence Class test cases?

1. Identify the equivalence classes
2. Create a test case for each equivalence class

Practice exercises for equivalence partitioning testing

Simple Scenario

Let’s assume that there is a textbox that accepts a range of integers from 1 to 5. We can split this up into three equivalence classes, assuming that the developer implemented the code in the most logical manner.

1. The text box does not allow values less than 1. All values below one will behave the same way. Therefore, you only need to test one of the values. This group is the equivalence class.
2. The text box allows all integers from 1 – 5 inclusive. It doesn’t matter if you enter a 1, 2, 3, 4, 5. All of these values will behave exactly the same and should be accepted. Again, this is another equivalence partition, a valid equivalence class.
3. The text box does NOT allow any integer greater than 5. Same concept with this range of values. They will all behave the same, no need to try them all. Just pick one value, test it and you are done.

More Complex Example

Sadly, not all of our testing is going to be as simple as the scenario above. I wish that it was – our jobs would be cake. But then, anyone would be able to do our jobs, and that’s not what I want. So let’s tackle a more complex example.

Test Scenario

Our system has an input field that asks a user to input an integer value that represents the number of the month. After entering this integer, the system will output the name of the month and a cute bunny with a tattoo of that month on the side.

Using equivalence partitioning, try and figure out the minimal number of test cases required to test the system.

Remember, that the best QA Engineers cover all of the functionality of a component with the fewest number of test cases. Obviously, this is the most efficient approach, since there is no duplication of effort and waste of time.

Solution

Three test cases are all that is required to test all of the inputs for this specific scenario. However, this is not at the functional testing level. We would need more test cases to make sure the functionality is fully working.

Complex Example

Test Scenario

Consider a software module that is intended to accept the name of a grocery item and a list of the different sizes the item comes in, specified in ounces. The specifications state that the item name is to be alphabetic characters 2 to 15 characters in length. Each size may be a value in the range of 1 to 48, whole numbers only. The sizes are to be entered in ascending order (smaller sizes first). A maximum of five sizes may be entered for each item. The item name is to be entered first, followed by a comma, and finally, by a list of sizes. A comma will be used to separate each size. Spaces (blanks) are to be ignored anywhere in the input.

Think this through before you look at the answer. Use your brain, it’s a powerful thing.

Solution

Found Here

Conclusions

Be sure to actually put all this knowledge into practice so that your retention rate is better. You will be able to see that this technique is actually really useful in helping you to come up with possible test cases.

In the next tutorial, you will learn Boundary Value testing, which is an expansion on everything that we learned in this post.

Talk to you then. Comments are always welcome here.

Citations

• Copeland, Lee. A Practitioner’s Guide to Software Test Design

Great Resources

• Site that teaches a bunch of ISTQB techniques
• Testing Overview and Black-Box Testing Techniques
• Software Testing