When Testing Discovers the Spec

August 9, 2016
Phil
Phil

There’s a nice post from yesterday by Michael Feathers (author of the classic Working Effectively with Legacy Code) about characterization testing - the process of writing code that characterizes the actual behavior of that code. He makes the important point that, “When a system goes into production ... it becomes its own specification.”

In crowd testing, we see this point play out on the macro level. Since our testers bring their own testing heuristics [PDF] with them, they approach software in ways that its developers never envisioned. So sometimes customers will come to us with a production app they think is solid and we discover hundreds of “bugs.”

It then falls to the customer to determine which of these are “bugs” and which aren’t; a decision of no small moment to the testers, who get paid on the basis of the bugs they find. But Feathers’s article makes me realize that in some cases, our testers are effectively discovering the spec for the customer.

They found something the customer didn’t know about, which the moment the “not-a-bug” button is pressed our app becomes forever more a “feature.” You’re welcome.

Read More

June 13, 2019
The Important of Manual-Stage Testing

Incorporating manual testing into your development pipeline, without bottlenecks.

June 7, 2019
Regression Testing: Where Does Crowdtesting Fit In?

What is regression testing, and how does crowdtesting benefit it?

May 30, 2019
HUM Nutrition: A Test a Day Keeps the Bugs Away

test IO helps HUM Nutrition provide customers with life-changing products.

Ship Faster, Sleep Better

Get a Demo
testIO Logo Footer
test IO Inc.
1111 Broadway, Floor 3 
Oakland, CA 94607
USA

test IO GmbH
Sonnenallee 223 a
12059 Berlin
Deutschland
twitterfacebooklinkedin