These best practices can help you get the most from your functional testing.
Commonly used in the realm of programming, the concept of an immutable object simply refers to an object whose state cannot be changed after its creation. Many programming languages support immutability, thereby forcing a variable or object to be read-only, in which the contents can only be viewed or read, but not altered.
This concept of immutability has recently been transferred into the world of DevOps, specifically in the form of immutable infrastructure. Just as with immutable objects in programming, immutable infrastructure means that we're taking the various components that makeup the application infrastructure (web server, database, CDN, etc), and preventing them from being altered in any way once created. This contrasts with a "normal" infrastructure, in which a new deployment to production would require changes to existing components, all the while the software must remain live.
With automated immutable infrastructure, instead of dealing with changes to live components, all the components are entirely replaced when a new build is deployed. Typically, this is achieved using a common image that was previously built during early development, and can therefore easily be validated and tested prior to using it in production. That image is then used to create an entirely new immutable infrastructure, on top of which the new production deployment is placed.
There are a few clear advantages to implementing immutable infrastructure automation:
While it may be an irrelevant concern for many development projects, managing proper regulatory compliance can be critical for some organizations. Since most developers lack a background in law, maintaining proper regulatory compliance can be a challenge, if and when it is necessary.
Compliance is required in a variety of fields, from government and finance to pharmaceuticals and trade. The standards of compliance vary from one regulation to the next, but most regulations, as with HIPAA and PCI, heavily emphasize confidentiality and privacy.
In spite of the complications associated with regulatory compliance during software development, as with many other aspects of DevOps, it's possible to gain a bit of assistance through automation. Some organizations are now finding that they are able to comply with industry regulations through automated DevOps practices, in large part by properly focusing on and maintaining a clear (digital) paper trail.
Through common automated DevOps practices, it's now much easier for organizations to point to consistent, proven, and verifiable records of compliance. By using tools like automated testing and deployment, there now exists historical evidence that test cases directly related to regulatory compliance are passing, and automated deployment following those successful tests ensures that compliance is pushed to production.
Automated provisioning of servers is a fairly common DevOps practice on which to host development or production applications, yet one often overlooked method is the automatic creation of development environments. Similar to the concept of storing images for use in immutable deployments we explored earlier, generating a baseline image as a "standard" developer environment for your current project is a massive timesaver.
When a new developer joins the team, there's no need to spend hours installing the proper apps, making the appropriate configuration adjustments, or providing the correct access rights to the developer's machine. With an automated development environment image ready to go, a copy of the standard developer environment can be copied to the new developer's machine, and all the necessary access rights, applications, and configurations will be included right out of the gate. Best of all, this image can be used on a local machine or even remotely, via a virtual machine.
There are a number of existing tools that can be used for the creation and management of automated developer environments, but a few popular choices include Vagrant and Chef. Each tool provides its own benefits, but they're all generally designed to be user-friendly, by providing a simple API and using a common language or syntax, and are able to build most any configuration into a new development image.
Even though most of the automated tasks associated with DevOps tend to focus on managing code once it’s written, there's something to be said for the practice of automatic code reviews. In the simplest terms, an automatic code review aims to evaluate existing code, using robust services and applications, by checking for proper style, complexity, security, duplication, and coverage... automatically. There are a number tools that can be used for automated code review, including Code Climate, Codacy, and Codebeat, but every service will largely aim to perform the same basic role within your DevOps.
Automated code review provides many of the same benefits of the practice known as pair programming, which places two developers at the same desk and has them work in tandem on active development. Rather than requiring another human to physically sit with each developer, an automated code review service aims to provide many of those same benefits, by verifying code quality and coverage with simple interfaces, all while being compatible with existing automated DevOps practices such as continuous integration and continuous deployment.
While the concept may seem contradictory at first, another powerful tool that can be automated as part of your DevOps process is automated human feedback through crowdtesting. In cannot be overstated how critical feedback is during the entire software development life cycle. Whether that feedback comes from other developers, users, customers, or testers, it is imperative that the team is able to gather, analyze, and adapt to that feedback as effectively as possible. Adapting to incoming feedback is particular important for projects following a form of agile development, where iteration and adaptation is a staple of the methodology.
Crowdtesting can provide your project and organization with unique insight into what may otherwise be uncommon scenarios or unforeseen test cases. It is common for automated testing solutions -- whether for unit testing, functional testing, or otherwise -- to remain somewhat blind to potential issues or bugs that professional human testers are experienced and trained to seek out. Moreover, even if all automated tests pass, that isn't a guarantee that the software is fully-functional and ready for production. This is a great reason to slip crowdtesting into the mix, so human testers can perform testing and provide curated feedback.
The implementation of automated crowdtested feedback will vary from one tool to the next. Services like test IO provide an API to trigger tests based on your own conditional requirements, so feedback can be gathered automatically, alongside your other automated DevOps best practices.
Ultimately, automated human feedback can grant your organization detailed insight on the quality and readiness of your application for future builds or even production release. Through exploratory testing and automated feedback loops via crowdtesting, a multitude of testers from all around the world can provide real world testing feedback by simulating the experiences of actual users, to better ensure your software is ready for launch.
These best practices can help you get the most from your functional testing.
Exploratory testing has unique advantages, like better collaboration and less preparation. Learn more here.
Regression testing is important to QA but can be undervalued, especially compared to other testing.