What is exploratory testing?
Exploratory testing is an approach to software testing that emphasizes the engagement of the tester, the unscriptedness of testing, and the tester’s freedom to go down paths they think are important. Empowered to choose what they focus on and how they evaluate the software, these testers can pursue tests in areas that have potential for problems or that they think (based on issues in other areas) should be checked out.
Test design, exact testing steps, and expected outcomes for exploratory testing are not determined in advance. Thus, while scripted testers follow a set of test cases, and the paths in the software they are checking have been considered and laid out for them in advance, exploratory testers get their results without following a predetermined route.
This approach to testing values the testers’ experience, intelligence, creativity, learning, and even intuition by locating test design, test execution, and test insights in one person at one time. Exploratory testers are freed from the encumbrance of documenting their steps in advance, which gives them the chance to incorporate what they learn, as they learn it, directly into the next test.
Of course, this doesn’t mean the testers are doing whatever they like; exploratory testing isn’t random clicking or pushing of buttons. Yet, by removing strictures of scripted testing, exploratory testing allows for a more holistic approach to test design, execution, and analysis.
Exploratory Testing Advantages
Exploratory testing evolves during the process as testers are empowered in the moment of testing to explore scenarios related to the devices and platforms they’re using. This enables testers to look for all sorts of potential problems and issues related to the software’s platform. For example, an app functioning as expected on the Chrome mobile browser
on iOS might not do as well on an Android device (even of the same browser). And that’s just one glimpse of all the possible platform problems the product might encounter running on all of the different mobile or web apps, smartwatch software, or anything else.
An exploratory tester can and does take into account the issues of each browser, operating system version, hardware configuration, and other salient factors. This type of testing excels at discovering platform-based problems.
Other advantages of exploratory testing include:Reduced test prep time
— Since this testing methodology requires less documentation, the tester is able to start experimenting sooner.Simultaneous learning
— Skilled testers can utilize their personal skills, experience, and knowledge to the testing process and learn about system behavior while testing. This means they can design, execute, and even improve testing scenarios in real-time, seeing how scenarios or steps connect with an agility that can’t be replicated by automated testing.Dynamic feedback
— With testers freed from the traditional test-case based approach, they are able to focus their efforts on key areas or problem areas as needed. This can give developers much needed input on product quality or possible issues on an ongoing and more immediate basis.Finds critical bugs
— Scripted tests are limited to some degree by confirmation bias. The developers will design cases based on what they think might go wrong. On the other hand, exploratory testers think differently and explore more freely to discover difficult-to-detect defects.
Exploratory Testing Techniques
Exploratory testing supports the release of superior quality software. How? By getting the testers and the software out of the self-contained bubble of a testing lab. Free of the confines of the controlled testing environment, exploratory testers can make sure that apps and websites work in real-world conditions — and in the way real users would want to use them.
Confirming that the app or software runs smoothly inside your office doesn’t guarantee it will work all of the time for everyone. Your users may be using your product at home, at work, on the road, with spotty Internet, in rural or urban areas, on mobile, laptop, desktop or even Internet of Things (IoT) devices… the possibilities are endless.
That’s why exploratory testing techniques focus on identifying:Product purpose
— what is the fundamental thing the product is meant to do?Primary functions
— what are all of the primary functions the product does?Areas of potential instability
— which functions are most likely to fail, crash, violate stability standards, be unreliable, or have negative side effects?
The actual testing of each function and recording problems then follows. The aim is not to spend a lot of time examining every possible pathway until defects are detected. A diligent tester will take a deeper dive in identifying issues otherwise difficult to find. Based on their familiarity with all that can go wrong during the course of software development, these testers focus on how users might interrupt, corrupt, crash, bend, or break the system. Nevertheless, exploratory testing is anything but making wild and uneducated guesses. As James Bach wrote in an often-cited paper on exploratory testing
, the prime directive of this methodology is to “be thoughtful and methodical.”
Bach was one of the pioneers, with Cem Kaner, of the exploratory testing concept in the 1990s. They saw early on the value in putting software to the test even while the testers were getting acquainted with the product and of giving testers the freedom needed to come up with test cases that developers and traditional testing practices might not think about doing. As a result, when exploratory testing methods complement other techniques, testing quality improves. This, in turn, means higher quality products are released and better user experience that leads to software success.
Exploratory Testing in Agile
A fast-paced agile environment is founded on reacting and adapting quickly. Exploratory testing fits right in with agile methodologies. Exhaustive automated testing is difficult to accomplish within the timeline of compressed agile sprints. This doesn’t mean exploratory should replace automated testing — the one can effectively support the other — but exploratory testing is more often feasible when it’s too early to employ test automation.
In the exploratory testing methodology, instead of requiring complete documentation and written, scripted test scenarios, the expert tester can dig into the product in development and gain a general idea of its functionality. The tester is able to design and test at the same time. This recognizes his or her ability to hit upon the right tests to do at the right time.
This is a creative kind of testing that can better keep pace with the demands of an agile process. This methodology embraces continuous refining and improvements. The tester develops a mission, designs tests to accomplish that mission, and does the testing. If the testing questions go unanswered, the tester can tweak the process until the tests return results offering insight into the original objective.
Exploratory testers are also a good fit in the agile team, which is typically lean and mean (at least in terms of meeting process deadlines). After all, this type of tester is able to quickly switch directions, think creatively, and pivot as needed when requirements or priorities change.
Identifying the Best Exploratory Testers
Just as this article notes that exploratory testing isn’t about clicking willy-nilly, it’s also not about finding just anybody to take your software for a spin. What makes certain people great exploratory testers? There are several attributes test IO seeks out:Inquiring
— Exploratory testing doesn’t pre-determine what’s to be tested or how, so great exploratory testers are inquisitive and interested. They want to find out everything. They want to know what’s behind this door (or button) or and what’s hiding in this box (or feature). They need to be driven to learn more since they’re on their own when it comes to checking the different parts of the website
or application and investigating its nooks and crannies.Flexible
— Great exploratory testers are competent test designers who can roll with the punches: if an investigation of a specific part of the software turns up an error, they need to be able to improvise new tests that look for related problems and issues. An exploratory tester who is good at coming up with impromptu additional tests can also determine the scope of a bug, or at least circumscribe the impact of it, which makes the task of fixing it easier for the developer.Observant
— Inquisitiveness isn’t enough on its own. Exploratory testers should not only be curious, but also able to notice any aberrations and document them. An excellent exploratory test requires the testers to observe settings, environmental factors, and internal mindsets as well.Holistic
— Exploratory testing needs to consider and record anything that may influence the testing. This could a range of factors such as: wifi versus cellular, signal strength, battery level, or even ambient noise. The great exploratory tester embraces the need for additional investigation and fully considers all paths that should be investigated instead of settling for one scripted solution.Excellent Communicator
— Someone can be intelligent and inquisitive and awesome at finding defects, but it does little good if they can’t communicate their test findings to the developers. Without a separate test designer sitting down with the product manager to pre-determine individual testing steps and expected outcomes, testers are responsible for explaining the idea behind a given test, the circumstances surrounding it, and then the outcomes that result.
The best exploratory testers know when to use words, when to use screenshots, and when to take videos. They take into account the point of view of the people who will be using their work: developers, QA professionals, and product managers and offer the key information those people need to do their work.Focused
— Great exploratory testers also know their aim is not to be comprehensive, covering every scenario. They are to find defects and issues that wouldn’t be found through typical testing. At the same time, while they are looking at edge cases and parts of the system away from the normal usage pattern, they are focusing on the areas that are less likely to be well tested otherwise.
Exploratory Testing with test IO
If exploratory testing unleashes the full potential of each and every tester, it makes sense to turn to test IO’s high potential testing specialists
Our experienced and engaged crowdtesters can ensure the kind of coverage required for high-quality software and let you benefit from the wisdom of the crowd.
With test IO
, your software gets the benefit of exploratory testing and the unfettered examination of an experienced software tester, counterbalanced by the security of having a crowd to find all the important bugs. We can quickly put dozens or hundreds of testers to work testing your software, app, or website based on the exploratory parameters you outline.
Enjoy real-word tests from a diverse group of global professional testers using the varied devices, operating systems, and browsers you want to test. Learn more!