When I first started working for test IO, I spent most of my time simply trying to explain what crowdsourced testing is. I was immediately met with remarks like, “Oh, so you’re like tech support in India or China!” I struggled to explain that there is indeed a difference between crowdsourced testing and outsourced testing.
As the new CEO of test IO, I spend a lot of time thinking about testers, so I might as well think out loud. A tester’s point of view on crowdtesting depends a lot on what you mean by “crowdtesting.” Speaking broadly of functional testing, there are two types of crowdtests: one where testers follow a script and are paid for completing the script and checking off whether it worked; and another where they’re given a charter and set of things to test, but not told precisely how to test them. At test IO, we call the first type of testing “test cases” and the second type “exploratory tests.” (We’re being sloppy here with the second definition, but I’ll cover that in a later post).
Welcome to part 3 of our occasional series on weird software bugs that people find even when automated tests don’t.
I wrote an article for CMSWire with the self-consciously clickbaity concept that crowdtesting is like sex, for product quality. Rather than summarizing the sex analogy here on our blog, I’ll let you read it in its original glory. To atone for the clickbaity-ness of that article, let me summarize it in engineer-friendly terms here. Software […]
Welcome to part 2 of our occasional series on weird software bugs that people find even when automated tests don’t.
There’s a nice post from yesterday by Michael Feathers (author of the classic Working Effectively with Legacy Code) about characterization testing - the process of writing code that characterizes the actual behavior of that code. He makes the important point that, “When a system goes into production ... it becomes its own specification.” In crowd testing, […]
Jan: Why do you think every company isn’t doing crowdsourced testing already? Phil: One reason is that I think people do have this perception that it’s a lot of people with no skills, clicking through predefined test plans with no skill or human insight -- basically automation with people, or Turk-work. That’s clearly not true […]
People sometimes ask us, “What kind of bugs do your testers find even when there’s a lot of test automation already?” Today we inaugurate an occasional series that highlights interesting bugs (and types of bugs) that our testers find. Don’t worry, customers, we won’t share your data. The truth is, a lot of the things we find before release are out there in production software every day! We can use those.
Martin Reinbach has been with test IO for over two years and works with our customers in the DACH region. He’s worked with dozens of customers, sharing his expertise on crowdsourced testing, agile development, and software projects generally. I asked Martin to give new customers some insight into what to expect from crowdsourced testing.
We talk a lot to QA managers about crowdtesting. We talk to some of them (our customers) who’ve embraced it, and others (sales prospects, people in the street) who may be new to the whole thing. I sometimes wish we could get those groups together to share notes. Since this is our blog, we can! Here’s what test IO customers tell us about why combining in-house QA with crowdsourced testing works for them.
