Crowdsourcing vs. Outsourcing: A Testing Smackdown

It turns out I’m not alone. Confusion arose from the first reference to crowdsourcing in 2006, and while others have explained the difference between outsourcing and crowdsourcing generally, I want to talk specifically about their role in agile testing and continuous development.

Breadth vs. Depth

• Outsourcing - If you're looking for people to write your automated tests, implement or maintain load testing scripts, employ specific technologies related to your product's technology stack, you're probably looking to outsource. In general, outsourcing can connect you with teams possessing specific expertise.
• Crowdsourcing - If you’re interested in running an exploratory test prior to releasing a new feature or performing a regression test to guarantee your platform’s functionality, crowdsourcing is a perfect fit. By tapping into the crowd, you are able to quickly get results around the clock across a full-spectrum of devices and browsers under real-world conditions. This breadth of coverage is often difficult to perform in-house (or through outsourcing) because of limited resources (money, time, personnel, etc.).

Management Overhead

• Outsourcing - As others have noted, success with outsourcing involves considerable work upfront -- selecting a suitable project manager, creating step-by-step playbooks, educating the outsourced team about internal processes, coordinating conference calls (often across timezones). And once you start, you’ve got a new team, so you’re managing people with all that entails: egos, team dynamics, communication overhead, career paths, and employee performance.
• Crowdsourcing - Crowdsourced testing with a platform like test IO provides a layer of abstraction that removes managerial overhead. Our platform provides a nice UI for creating and managing tests, and we take over the challenge of coordinating, evaluating, and paying the testers, allowing engineering and QA teams to focus their efforts on continuous delivery of software. Our goal is to make running a crowdtest as easy as running an automated test.

Independence

• Outsourcing - We often see companies outsourcing the work of creating automated functional tests, with the outsourced team creating test scripts for each test case in a test plan. This is a good project for an outsourced testing team because you can describe what you want pretty easily, measure the team’s progress, and give them the ongoing task of keeping these often brittle tests up-to-date. While you get what your contract stipulates, you’re diminishing risk but also creativity.
• Crowdsourcing - Because the crowdsourcing model is less rigid in its expected outcome, the crowd engages with a piece of software independently. An independent collection of diverse minds helps companies to think about their product in new ways and often uncovers unforeseen issues. Moreover, since crowdtesters are not “part of the team” they can’t fall prey to the cognitive biases that even an outsource team develops, so they can provide a realistic assessment of how customers will evaluate the software when it’s released.

Outsourcing helps companies accomplish specialized tasks that would be too difficult or otherwise costly to complete in-house. Crowdsourcing is a low-overhead and low-risk way to scale your QA testing force while introducing a diverse set of experiences and perspectives that more accurately mirrors your customer base. Each offers benefits to a development organization, but they are, in fact, quite different.