Verify Bug Fixes with Bug Fix Confirmation

In this blog, we will discuss the history of test IO’s bug fix confirmation, why development teams need it, and how to integrate into your team’s workflows.

Like all great inventions, bug fix confirmation was borne out of necessity.  

Several years ago, as the sole Product Manager among a small team of developers and engineers for test IO, Alex Leonhard was responsible for the quality of our products. Like test IO’s clients, once the team finished building a product, they released a test to the crowd. The crowd, being the studious perfectionists that they are, would then return detailed results of their tests, which included lots of bugs that needed to be fixed—and a lot more work for Alex and the team. 

The developers would then fix the bugs, but then, as the Product Manager on a team that included no QA engineers, Alex found himself in the position that many of his fellow peers in product management are familiar with—spending hours manually checking that each bug was, in fact, fixed. When he inevitably found some that weren’t fixed, Alex would go back to the team, and again repeat the process ad nauseum until finally the team had the perfect product. 

As the company grew—and with it, Alex’s responsibilities—Alex realized they needed a faster, more efficient way to do this. They needed to free up their time to focus on high-level activities that drive the business forward. They had thousands of the brightest minds in testing at their disposal 24 hours per day, seven days a week – there had to be a way to leverage that.  

That’s why Alex created Bug Fix Confirmation, a tool that allows developers to harness the power of the crowd to confirm if bugs were, in fact, fixed.  

Within 30 minutes, test IO’s professional software testers will confirm if issues have been fixed on real-world devices and in real-world conditions. With test IO’s bug fix confirmation add-on for JIRA, teams can receive detailed results without ever having to leave their testing environment, further decreasing the time spent on bug fix confirmation.  

If your team doesn’t work in JIRA, however, you can also receive notifications through test IO’s interface or via email. Our integrations team is currently expanding the number of bug trackers that bug fix confirmation can integrate with.  

No matter if your team is made up of 500 developers or five developers, your team can free itself up to work on higher-value tasks that ultimately prove to be a bigger return on investment in your team’s time.  

Once your team has fixed all the bugs, it’s time to check that all of the changes have been made appropriately. Here’s how to request a bug fix confirmation: 

1. Request a bug fix confirmation:
   • Select the test environment or upload a new build.  
   • Specify the device that should be used by the tester to confirm your bug fix.
2. Our testers try to find the bug again 
   • The bug fix confirmation is assigned to a professional tester. 
   • The tester repeats the steps that led to the bug, using a real device. 
   • They attach a screencast showing the bug (or its absence) and add a comment.
3. Receive results fast 
   • You will receive an email notification about the result of your bug fix confirmation.  
   • You will receive one of four possible results for each of the bugs tested: 
     • Pass: Your bug was fixed! There’s no need to focus on this bug any further. 
     • Fail: Your bug was not fixed. Testers will provide as much detail as possible about the problems they encountered in the bug report. Once you have fixed the bug again, you can again request a bug fix confirmation once ready. 
     • Blocked: In this case, the tester could not identify if the bug was fixed or not. There are a number of reasons this can happen: the tester did not have the username or password necessary to see the feature, the feature was eliminated when another bug was fixed, or something else entirely. Testers will provide as much detail as possible about why they couldn’t confirm the bug fix in their report. 
     • Expired: If test IO was not able to find a tester who meets all of your requirements within 24 hours, we will email you that your test has expired. This typically happens when a test is requested on a not-widely-available device or software version.  
4. Repeat the process as necessary!
   • Continue to request bug fix confirmations until all of your product’s bugs have been fixed and your product is working as intended.  

By integrating test IO’s bug fix confirmation into your workflows, your team can save itself time (and headaches) spent on manually testing bug fixes that our team is all too familiar with. Leverage the crowd to verify bug fixes through bug fix confirmation, and minimize your team’s time spent on these lower-value tasks so they can perform the higher-value tasks that ultimately drive your business’s success.   

If you’d like to see how this works in person please reach out for a demo, here.