Crowdsourced Testing and Your QA Team

Free all the people: Your most skilled QA team members are a precious resource. With crowd testers behind them running tests, they can focus their efforts on working closely with developers on making products more testable, developing automated tests and frameworks, and debugging and analysis. If you’re a tester and you’ve ever said, “I wish I could collaborate more with developers, do some pair testing, and spend more time mentoring my team, but I’m too busy filling out this test case template” you should think about having the crowd behind you!

Test all the devices: 10 years ago cross-browser testing was hard, but today’s mobile and IOT landscape poses a mammoth challenges for QA teams. What works well on one device may be flaky on another, and conditions like cell coverage and even the carrier may affect your product’s performance. Crowd testers work with real devices, in the real world, so you know how your product will perform when your customers see it. If you’re tired of hearing, “It works on my machine,” or “It works in Chrome,” or “It works in the emulator!” then you should think about crowdtesting.

Flex up when you need it: Sometimes it’s crunch time and you don’t have enough skilled testers to cover the big new features you need to ship. Regression testing in agile sprints is challenging when code is changing fast. With the crowd behind you, you will never bottleneck on a lack of skilled testers, so velocity will increase. So if you hate Christmas, you should think about crowdtesting.

Usability feedback with fresh eyes: Even the best testers can fall into a rut with a product they know well. Crowd testers come to your app without familiarity biases, so their exploratory testing finds bugs and discovers usability issues that your team might pass over. Also, your product team is more likely to listen to external feedback than their own colleagues (I know, unfair!) because they look more like your customers. So if you think something “smells funny” in the user experience, the crowd is your friend: one person’s feedback is opinion; a crowd’s feedback is data.

Code locally, test globally: You can convene crowd testers from specific countries, with specific linguistic and demographic profiles, so there’s no more worrying about how your product works in Spanish or on a phone with Russian locale settings. We all know that “foreign countries” often get no love (or at least much less love) from your team, but once you’ve got a crowd, you’ll know what you’re missing.

Maybe you get nervous when you release software. These days, there’s nowhere to hide. People can trash you on the app store, and everyone knows.

Customers’ ratings on app stores are subjective because customers are people. Opinions matter. That’s why our testers can give you a “release readiness” score that aggregates testers’ impressions of your product’s readiness to ship. Since our testers don’t have the emotional stake in your product that you do, their assessment is much closer to your customers’. It’s like getting a private app store rating.

Soliciting human feedback before you ship and fixing the major emotional pain points is the single biggest thing you can do to improve customers’ perception of your product. You know if you test software for a living that you’ll never have bug-free software, but sometimes little red flags can distort your entire impression of an application.

The crowd can help you stay out of the red zone.