Crowdsource QA Testing: Insight as a Service?

September 9, 2016

Exploratory testing tends to attract people who like investigating problems, solving puzzles, and discovering patterns. It’s intellectually demanding work, and it requires judgment. Like most demanding work, some people are better at it than others, and you get better at it with practice, training, and feedback.

In other words, it’s not clickwork. You wouldn’t ask untrained people from Mechanical Turk to do it -- or if you did, you wouldn’t get very good results. And you can’t simply automate it, though like most modern work, you can make it more efficient with good tools.

Unfortunately -- perhaps because crowdtesting also includes executing scripted tests, which can be done by people without skills and automated -- I’m not sure that everyone knows that crowdtesting is not clickwork.

That’s one reason I wrote this article for TechCrunch. Maybe we need to play around with the language here to make it more human-friendly. “Crowdtesting” is an odd word, because it suggests that the work is done by a “crowd,” (like the wave at a ballgame) when really it’s done by individuals. Sometimes we use the phrase “QA as a service,” or “testing as a service.” But somehow these phrases aren’t human enough either. It’s really more like “judgment as a service” or “insight as a service.” But who buys that?

Anyway, since we’re in a business where human judgment matters, we want our customers and testers to understand that we’re investing in testers’ training and development, and that we also encourage customers to communicate with testers directly.

Some of the work we’re doing in this area isn’t visible to everyone yet as we continue to refine it. But as we roll out more features that depend upon human judgment, training and feedback become increasingly important. So if you’re a tester, I think you’ll be seeing some improvements from us in the coming months. And if you rely on testers’ judgment about your product, you’ll benefit too.

Read More

January 17, 2020
QA Squads: a new offering from test IO, amplified by EPAM

Going beyond software  Customers come to test IO for many different reasons. Sometimes, an internal product or QA team needs a force multiplier for real-world testing – to extend the existing QA team’s processes and activities beyond their internal team. Other times, it’s crisis mode – perhaps QA leadership has left the company, or there is a critical product release upcoming that […]

November 25, 2019
iOS testing: TestFlight or Resigner

Here's our guide to which method you should use you to distribute your IOS app to the crowd.

November 15, 2019
Exploratory Testing vs. Test Case Testing

Exploratory testing emphasizes creativity and learning. Test Case testing emphasizes planning and execution. Which one is right for you?

Ship Faster, Sleep Better

Get a Demo