Segmenting and Inviting Testers

August 30, 2017
Powen Shiah
Powen Shiah

We're continually improving and re-engineering how testers get selected and asked to join a test cycle. Our goal is to make sure you get the best coverage, fast turn-around, and the right testers for your particular product, and that invitations to tests are distributed fairly to testers.

Tester Segmentation

How do we do group testers and figure out which ones fit for which tests? To start off, we can classify testers based on many characteristics: geography, experience, devices, when their last test was, and other criteria. We use these segments or dynamic lists of testers to power our invitations. 

Primarily, we place segments into testers based on their capabilities. Based on data like the types of bugs reported, the acceptance rate, and tests participated in, our machine learning system determines whether a tester is better at rapid tests or focus tests, desktop or mobile, or any number of other preponderances.

Invitations

The test IO platform invites testers in waves. We do this to make sure that the test cycle has enough of the right devices or other requested characteristics. These waves of invitations are also set up to fill uncommon devices first. For example, if a test cycle includes a relatively rare device/OS combination like iPhone 5 running iOS 8, we'll send out invitations first to testers who have this device listed. If they join this test cycle, they can only submit bugs on that device.

We also use invitations to make sure that in addition to our experienced testers, testers new to the test IO platform ("greenhorns") get invited to a broad range of tests. We do this to so our newer testers have opportunities to gain more experience. To help test IO's crowdtesters improve their software testing skills, greenhorns also get extra feedback from our experienced test leads.

Read More

February 21, 2019
Announcing the New and Enhanced Tester Academy

Introducing test IO's new Tester Academy: additions and enhancements!

February 19, 2019
Code Coverage vs. Test Coverage

What is code coverage? Test coverage? How do they relate (and why it matters)?

February 12, 2019
Tester Spotlight - February

Our February Tester Spotlight focuses on Hans, one of our incredible US-based Team Leads.

Ship Faster, Sleep Better

Get a Demo
Follow us on:
twitterfacebooklinkedin