Continuous Testing Manifesto
Build – Test – Learn… and Repeat!
While automated tests are awesome and an integral element of the agile software development process, they are only part of the story. The majority of software is written by people and used by people – and thus should also be tested by people.
Computers are precise and logical, which makes them perfect for automated testing – but even those tests are written by people and often can’t cover all eventualities.
Humans may not be so precise, but they outdo computers in many other ways: creativity, context-awareness, outside-the-box thinking and – a clear advantage in this case – being somewhat unpredictable. Combining automated testing and human testing is what continuous testing is really all about: ensuring full, real-life test coverage for your products.
While automated testing has found its way into the processes of many development teams, testing with real people in the real world often remains a time-consuming, necessary evil just before the product launch.
Officially, testers are supposed to uncover bugs, but unofficially, everybody secretly hopes they will not come up with anything major that would delay the scheduled launch. The reality is, however, the earlier a bug is found, the less damage it can do and the fewer costs it give rise to.
Continuous testing uses this insight by integrating testing at the very beginning and keeping them involved throughout the development process. This prevents bugs from disrupting the development plan and keeps expenses down.
A continuous testing scenario implies real, working continuous integration. As soon as code is pushed to the CI server and the automated tests run green, it gets deployed to the staging server where testers can screen it for bugs.
Detected bugs are submitted to developers, who can then fix them right away, and ideally write the corresponding automated tests so the same bug doesn’t turn up again.
This turns human-driven testing from a major impediment to shipping the product to just another part of development, like automated testing.
By implementing continuous testing, tester feedback is available early on to refine and improve the product, not just as the end when it might be too late. all the way through the development process.