Crowdsourcing vs. Outsourcing: A Testing Smackdown

September 14, 2016
Aqueelah
Aqueelah

It turns out I’m not alone. Confusion arose from the first reference to crowdsourcing in 2006, and while others have explained the difference between outsourcing and crowdsourcing generally, I want to talk specifically about their role in agile testing and continuous development.

Breadth vs. Depth

  • Outsourcing - If you're looking for people to write your automated tests, implement or maintain load testing scripts, employ specific technologies related to your product's technology stack, you're probably looking to outsource. In general, outsourcing can connect you with teams possessing specific expertise.
  • Crowdsourcing - If you’re interested in running an exploratory test prior to releasing a new feature or performing a regression test to guarantee your platform’s functionality, crowdsourcing is a perfect fit. By tapping into the crowd, you are able to quickly get results around the clock across a full-spectrum of devices and browsers under real-world conditions. This breadth of coverage is often difficult to perform in-house (or through outsourcing) because of limited resources (money, time, personnel, etc.).

Management Overhead

  • Outsourcing - As others have noted, success with outsourcing involves considerable work upfront -- selecting a suitable project manager, creating step-by-step playbooks, educating the outsourced team about internal processes, coordinating conference calls (often across timezones). And once you start, you’ve got a new team, so you’re managing people with all that entails: egos, team dynamics, communication overhead, career paths, and employee performance.
  • Crowdsourcing - Crowdsourced testing with a platform like test IO provides a layer of abstraction that removes managerial overhead. Our platform provides a nice UI for creating and managing tests, and we take over the challenge of coordinating, evaluating, and paying the testers, allowing engineering and QA teams to focus their efforts on continuous delivery of software. Our goal is to make running a crowdtest as easy as running an automated test.

Independence

  • Outsourcing - We often see companies outsourcing the work of creating automated functional tests, with the outsourced team creating test scripts for each test case in a test plan. This is a good project for an outsourced testing team because you can describe what you want pretty easily, measure the team’s progress, and give them the ongoing task of keeping these often brittle tests up-to-date. While you get what your contract stipulates, you’re diminishing risk but also creativity.
  • Crowdsourcing - Because the crowdsourcing model is less rigid in its expected outcome, the crowd engages with a piece of software independently. An independent collection of diverse minds helps companies to think about their product in new ways and often uncovers unforeseen issues. Moreover, since crowdtesters are not “part of the team” they can’t fall prey to the cognitive biases that even an outsource team develops, so they can provide a realistic assessment of how customers will evaluate the software when it’s released.

Outsourcing helps companies accomplish specialized tasks that would be too difficult or otherwise costly to complete in-house. Crowdsourcing is a low-overhead and low-risk way to scale your QA testing force while introducing a diverse set of experiences and perspectives that more accurately mirrors your customer base. Each offers benefits to a development organization, but they are, in fact, quite different.

Read More

November 25, 2019
iOS testing: TestFlight or Resigner

Here's our guide to which method you should use you to distribute your IOS app to the crowd.

November 15, 2019
Exploratory Testing vs. Test Case Testing

Exploratory testing emphasizes creativity and learning. Test Case testing emphasizes planning and execution. Which one is right for you?

July 19, 2019
Reducing Risk in Agile Development

How can you reduce risk when implementing agile development in your SDLC?

Ship Faster, Sleep Better

Get a Demo
twitterfacebooklinkedin