Segmenting and Inviting Testers

August 30, 2017
Powen Shiah
Powen Shiah

We're continually improving and re-engineering how testers get selected and asked to join a test cycle. Our goal is to make sure you get the best coverage, fast turn-around, and the right testers for your particular product, and that invitations to tests are distributed fairly to testers.

Tester Segmentation

How do we do group testers and figure out which ones fit for which tests? To start off, we can classify testers based on many characteristics: geography, experience, devices, when their last test was, and other criteria. We use these segments or dynamic lists of testers to power our invitations. 

Primarily, we place segments into testers based on their capabilities. Based on data like the types of bugs reported, the acceptance rate, and tests participated in, our machine learning system determines whether a tester is better at rapid tests or focus tests, desktop or mobile, or any number of other preponderances.


The test IO platform invites testers in waves. We do this to make sure that the test cycle has enough of the right devices or other requested characteristics. These waves of invitations are also set up to fill uncommon devices first. For example, if a test cycle includes a relatively rare device/OS combination like iPhone 5 running iOS 8, we'll send out invitations first to testers who have this device listed. If they join this test cycle, they can only submit bugs on that device.

We also use invitations to make sure that in addition to our experienced testers, testers new to the test IO platform ("greenhorns") get invited to a broad range of tests. We do this to so our newer testers have opportunities to gain more experience. To help test IO's crowdtesters improve their software testing skills, greenhorns also get extra feedback from our experienced test leads.

Read More

April 1, 2020
How test IO helps amplify distributed teams

With remote work capability now a necessity, companies must have the tools in place to enable distributed teams to perform at the highest level. test IO is one of those tools.

March 12, 2020
Tester Spotlight - Alex Che

We’re able to do what we do because of our incredible and diverse community of testers. Meet Alex, a Policeman now QA Engineer who gained the real-world QA experience he needed to change careers testing with test IO.

March 10, 2020
A Benefit of Crowdtesting - Time Compression

When budgeting for crowdtesting in the coming years, it's important to know all the value it adds. Here a little more on one of those values, time compression.

Ship Faster, Sleep Better

Get a Demo