Continuous Testing Manifesto

November 17, 2015

Continuous testing...

  • combines automated testing and testing with people to ensure full test coverage in real life situations for your products
  • makes the most of computers' precision and humans’ creativity to find bugs before they can cause damage
  • turns testing with people from the major drag just before shipping into an essential part of development, just like automated testing
  • can be used to solicit tester feedback to improve and shape the product from the beginning of and throughout the development process

Testing with People

While automated tests are awesome and an integral element of the agile software development process, they are only part of the story. The majority of software is written by people and used by people - and thus should also be tested by people.

Computers are precise and logical, which makes them perfect for automated testing - but even those tests are written by people and often can't cover all eventualities.

Humans may not be so precise, but they outdo computers in many other ways: creativity, context-awareness, outside-the-box thinking and – a clear advantage in this case – being somewhat unpredictable. Combining automated testing and human testing is what continuous testing is really all about: ensuring full, real-life test coverage for your products.

The Development Process

While automated testing has found its way into the processes of many development teams, testing with real people in the real world often remains a time-consuming, necessary evil just before the product launch.

Officially, testers are supposed to uncover bugs, but unofficially, everybody secretly hopes they will not come up with anything major that would delay the scheduled launch. The reality is, however, the earlier a bug is found, the less damage it can do and the fewer costs it give rise to.

Continuous testing uses this insight by integrating testing at the very beginning and keeping them involved throughout the development process. This prevents bugs from disrupting the development plan and keeps expenses down.

Continuous Testing and Continuous Integration

A continuous testing scenario implies real, working continuous integration. As soon as code is pushed to the CI server and the automated tests run green, it gets deployed to the staging server where testers can screen it for bugs.

Detected bugs are submitted to developers, who can then fix them right away, and ideally write the corresponding automated tests so the same bug doesn’t turn up again.

This turns human-driven testing  from a major impediment to shipping the product to just another part of development, like automated testing.

Improving the Product

By implementing continuous testing, tester feedback is available early on to refine and improve the product, not just as the end when it might be too late. all the way through the development process.

While testers are looking for bugs, at the same time they can provide crucial feedback on usability and features, continuously complementing market research and public beta testing.

Read More

January 17, 2020
QA Squads: a new offering from test IO, amplified by EPAM

Going beyond software  Customers come to test IO for many different reasons. Sometimes, an internal product or QA team needs a force multiplier for real-world testing – to extend the existing QA team’s processes and activities beyond their internal team. Other times, it’s crisis mode – perhaps QA leadership has left the company, or there is a critical product release upcoming that […]

November 25, 2019
iOS testing: TestFlight or Resigner

Here's our guide to which method you should use you to distribute your IOS app to the crowd.

November 15, 2019
Exploratory Testing vs. Test Case Testing

Exploratory testing emphasizes creativity and learning. Test Case testing emphasizes planning and execution. Which one is right for you?

Ship Faster, Sleep Better

Get a Demo