Exploratory Testing vs. Test Case Testing

Software testing comes in many forms, but the goal is synonymous: to ensure release quality by identifying bugs and flaws quickly, so developers can fix them before they appear in costly environments.

One approach often used by engineering-driven companies is test case based testing, executed either by automated script, or human testers.  Another is exploratory testing -- where there’s no test script to follow -- and must be performed by real humans.

Test case testing is great for finding out if your software is working as designed. Most mature software teams eventually build automation frameworks around test cases or have crowdtesters execute these test scripts. But relying on this type of testing alone can be dangerous. When you create a test case, you provide a program or a person with steps to follow e.g. proceed from point A to B to C. However, humans are unpredictable and don’t always operate linearly. So what happens in those cases? If you only conduct test case based testing, bugs can go undetected, reach your real-world users, and affect your bottom line. With exploratory testing, humans act like real users and are encouraged to jump from point A to C and back to B. This allows them to discover and find potential flaws you couldn’t imagine existed -- often gaps left by scripted test cases.

While worthy of a full read if you have questions about what exactly exploratory testing is, Guru99’s blog breaks down some of the primary differences between scripted test cases and exploratory testing. Take a look:

So what are some of the benefits of exploratory testing? What is Exploratory Testing in Software Testing (A Complete Guide) provides us with a few:

• It promotes creative thinking that can uncover hidden defects
• It requires minimal documentation while emphasizing learning
• It avoids duplicate work; different testers will explore different avenues
• It’s also useful when you need to audit another tester’s work

But exploratory testing isn’t without some drawbacks. Primarily, exploratory testing demands skilled testers who:

• Are excellent at communication
• Take thorough notes on their findings
• Can use their experiences and creativity to check common problem areas
• Will effectively answer follow-up questions developers might have
• Will think like your potential users and beyond

Bottom line: your exploratory tests are only as good as your exploratory testers. You need testers who don’t slow you down, and who can be trusted to deliver results promptly. That’s why it’s beneficial to go with a trusted QA service like test IO who employs thousands of vetted testers who are available on-demand to execute your testing needs.

So when should you go with scripted test cases and when should you perform exploratory tests? Let’s look at a few examples. Test case testing is great for checking the basics, and for features and functions that don’t change often. You can write and run a test script for almost any of the core functions of your product. For instance, for a mobile gaming app, can the user open the app, create a user profile, navigate through a set of instructions, and then play? These are basic functions that don’t allow for any creativity or spontaneity on the user’s part, so if you just need to know if this process works or not a test case might be the best solution.

But what if using your product is not quite so straightforward? eCommerce websites or apps are a good example of products that users might not approach linearly. Let’s consider the checkout process. You could develop a traditional test case to verify that a customer can make a purchase. It starts with a user opening the app or going to your site, selecting merchandise, going to their cart, logging in or continuing as a guest, and then completing the point of sale. And you could certainly stop testing there. Or you could perform exploratory testing instead of or in addition to this. Why? Because of the unpredictability of humans.

What if they lose connection?

What if they close out the app and come back?

What happens with their cart?

What if the user adds an item to their cart, goes to complete the sale, but then decides they want to shop more?

Will they be able to easily search for more items?

If a user clicks on an item to take a closer look and then they want to go back to browsing, will it take them right back to where they were or will they have to start from the top again?

These are all aspects that can impact user experience, and these are just a few scenarios that your real users could encounter. And while exploratory testing is generally unscripted, it doesn’t have to be completely without parameters. Tell the testers you want them to reach the point of sale and then to provide you with feedback. With just that guideline, they can now freely explore all aspects of your app or website, create test scenarios on the fly and find potential problems in these operations that you didn't even know you could have.

There are benefits and drawbacks to both approaches to testing. Test case testing has limitations because you’re only checking for flaws you think might exist, and you have to dedicate significant time to maintain those test cases as your software evolves. You also won’t get feedback about the usability of your product before you release. But exploratory testing requires creative testers who can use their cognitive abilities and experiences to find bugs quickly and efficiently.

In short, whether you’re a developer, manager, or in QA, you have to consider which approach best suits your testing needs along the way, and it doesn’t have to be an either/or. By performing test case testing along the way and bringing in exploratory testers to double-check your work and bridge the gaps inherent in scripted tests, you can have confidence that you are releasing a product that will perform as expected and, most importantly, people will love.