When Testing Discovers the Spec

August 9, 2016
Phil
Phil

There’s a nice post from yesterday by Michael Feathers (author of the classic Working Effectively with Legacy Code) about characterization testing - the process of writing code that characterizes the actual behavior of that code. He makes the important point that, “When a system goes into production ... it becomes its own specification.”

In crowd testing, we see this point play out on the macro level. Since our testers bring their own testing heuristics [PDF] with them, they approach software in ways that its developers never envisioned. So sometimes customers will come to us with a production app they think is solid and we discover hundreds of “bugs.”

It then falls to the customer to determine which of these are “bugs” and which aren’t; a decision of no small moment to the testers, who get paid on the basis of the bugs they find. But Feathers’s article makes me realize that in some cases, our testers are effectively discovering the spec for the customer.

They found something the customer didn’t know about, which the moment the “not-a-bug” button is pressed our app becomes forever more a “feature.” You’re welcome.

Read More

April 18, 2019
The Downfall of Manual Testing

Is manual testing being left in the dust?

April 15, 2019
Customer Conversation: Finding Your QA Fit

Looking to learn more about setting internal quality standards and finding impactful bugs? Hear what our veteran customers had to say!

April 12, 2019
How to Convince Your Boss that Thorough Software Testing is Worth the Money

Thorough testing isn't free, but it can definitely be worth the cost.

Ship Faster, Sleep Better

Get a Demo
Follow us on:
twitterfacebooklinkedin