When Testing Discovers the Spec

August 9, 2016
Phil
Phil

There’s a nice post from yesterday by Michael Feathers (author of the classic Working Effectively with Legacy Code) about characterization testing - the process of writing code that characterizes the actual behavior of that code. He makes the important point that, “When a system goes into production ... it becomes its own specification.”

In crowd testing, we see this point play out on the macro level. Since our testers bring their own testing heuristics [PDF] with them, they approach software in ways that its developers never envisioned. So sometimes customers will come to us with a production app they think is solid and we discover hundreds of “bugs.”

It then falls to the customer to determine which of these are “bugs” and which aren’t; a decision of no small moment to the testers, who get paid on the basis of the bugs they find. But Feathers’s article makes me realize that in some cases, our testers are effectively discovering the spec for the customer.

They found something the customer didn’t know about, which the moment the “not-a-bug” button is pressed our app becomes forever more a “feature.” You’re welcome.

Read More

November 25, 2019
iOS testing: TestFlight or Resigner

Here's our guide to which method you should use you to distribute your IOS app to the crowd.

November 15, 2019
Exploratory Testing vs. Test Case Testing

Exploratory testing emphasizes creativity and learning. Test Case testing emphasizes planning and execution. Which one is right for you?

July 19, 2019
Reducing Risk in Agile Development

How can you reduce risk when implementing agile development in your SDLC?

Ship Faster, Sleep Better

Get a Demo
twitterfacebooklinkedin