Biggest Challenges in Software Quality Assurance

We speak with a lot of prospective customers. Typically, these conversations will paint a detailed picture of the current trends and challenges in development and software quality assurance (QA). This allows us to more thoroughly understand the challenges that teams in target industries struggle with; and, the better we understand our market audience, the better we can hone our product to fit their needs.

To get a better idea of some of these common challenges and industry trends, I asked my team to help summarize what we’ve heard and are hearing from prospective customers.

• QA is a bottleneck: we need testing and we need it now.
• Device fragmentation: we need to test across all devices and browsers.
• Developers’ time: our developers divert their time for testing and have less time for coding.
• Limited resources: internal gaps in resources lead to suffering quality; and, poor quality leads to financial loss, especially with revenue-impeding bugs.
• High competition (especially within e-commerce): our competitors are just one click away, so our website and mobile app conversion must work.
• Assurance: we fear missing bugs and the chance of releasing them unintentionally.
• Poor customer service: our brand has a reputation to uphold, and we need to ensure a high quality across our product lineup.

The above in mind, the most notable complaints we hear are that teams lack the time to test adequately, they lack the personnel to test comprehensively, and they don’t yet have automation in place or assume it to be the catch all for issues.

When it comes to the time it takes to test, device and browser coverage, developers taking up QA efforts, and the small number of hours in a day all add up. If a team needs to test for all of the newest web and mobile devices and browsers, the team is either continually stuck with the cost of updating their device farm, or they’re relying on simulation software to provide accurate representations of the customer experience. Not only is the former expensive, it's also time consuming. On the other hand, the latter is more scalable but also a less accurate depiction of customer experience.

Additionally, QA can seriously become a bottleneck for engineering teams when developers are stuck combing through a product for bugs as opposed to focusing on the next build. Developers should being looking ahead at what's next, not stuck critiquing their own creations. Lastly, even with a fully scaled QA process or in-house team, there are only so many hours in the day and only so many hours that can occupy the time of dedicated QA resources. Maybe you need testing overnight, over the weekend, or over the holidays? These duties are simply difficult to impart upon an in-house team.

Moreover, one could argue for a connection between the comprehensiveness of testing and the amount of personnel available to provide this testing. Scalability is a critical factor in software testing, as the number of testers per test cycle can impact both the time it takes to receive results as well as the quality and overall completeness of findings. Even if a team does have a robust personnel count dedicated to QA, how often are tests running? Eventually, any team will hit capacity, and there will be a backup of scheduled tests. One way of providing against this is with an unlimited testing model, something that is only possible with crowdtesting in particular.

Many teams don’t have automation in place yet and assume it to be the answer to everything. They may also lack the time or resources to get it running efficiently. Many teams don’t have the time to manage automation and subsequent documentation when they’re trying to move quickly. Every time a minor change is made, a given test case might be affected. This often necessitates that someone be hands-on-deck for all subsequent changes. Even issues as simple as broken links (sometimes due to changes in content or functionality on the website), can be hard to keep up with without a large in-house QA team keeping up-to-date on all, even minor, product changes.

If conversations with our prospective customers have taught us anything, it’s that if we want to see real improvement in the QA process, we need to affect the common challenges facing most teams. We need decrease the time required to test effectively, minimize the amount of internal employees needed to test completely, and properly support the development of automation processes. No QA process is perfect, but changes in QA technology -- crowdtesting for one -- are beginning to address the above challenges, allowing teams to scale and move much more quickly, with fewer restraints on personnel, finances, and time.