When Testing Discovers the Spec

August 9, 2016
Phil
Phil

There’s a nice post from yesterday by Michael Feathers (author of the classic Working Effectively with Legacy Code) about characterization testing - the process of writing code that characterizes the actual behavior of that code. He makes the important point that, “When a system goes into production ... it becomes its own specification.”

In crowd testing, we see this point play out on the macro level. Since our testers bring their own testing heuristics [PDF] with them, they approach software in ways that its developers never envisioned. So sometimes customers will come to us with a production app they think is solid and we discover hundreds of “bugs.”

It then falls to the customer to determine which of these are “bugs” and which aren’t; a decision of no small moment to the testers, who get paid on the basis of the bugs they find. But Feathers’s article makes me realize that in some cases, our testers are effectively discovering the spec for the customer.

They found something the customer didn’t know about, which the moment the “not-a-bug” button is pressed our app becomes forever more a “feature.” You’re welcome.

Read More

February 21, 2019
Announcing the New and Enhanced Tester Academy

Introducing test IO's new Tester Academy: additions and enhancements!

February 19, 2019
Code Coverage vs. Test Coverage

What is code coverage? Test coverage? How do they relate (and why it matters)?

February 12, 2019
Tester Spotlight - February

Our February Tester Spotlight focuses on Hans, one of our incredible US-based Team Leads.

Ship Faster, Sleep Better

Get a Demo
Follow us on:
twitterfacebooklinkedin