by Glenn Gruber, AVP Travel Technologies
I always liked this military term. For those who have been living in a bunker for the past 10 years, "force multiplier" refers to a factor that dramatically increases the combat-effectiveness of a given military force.
In the world of software testing, a strong and well executed test automation program can act as a force multiplier for not only your QA organization, but the entire R&D team. With an aggressive test automation program you can:
- Reduce test cycles by at least 40%
- Slash testing budgets by removing the time and effort from manual testing
- Vastly expand code coverage
But where automation becomes a real force mutlipler is in catching more bugs before the code is released into production! Industry data shows that the cost of remediation can be 100x higher post-release than at the beginning of the product development cycle. By using automation to find the bugs earlier in the cycle you can greatly reduce re-work (not a value-added activity in the first place) and sustaining engineering costs. On top of that you can cut support volumes and costs by up to 75% (Gartner), not to mention saving yourself the damage to your brand and CSAT by having bad product in the field.
So it all sounds great, right? Yet when we engage with clients and prospects we find that while some have some level of automation, it’s often quite low, with less than 25% of all test cases automated. And of course many others haven’t implemented automation at all. Why is this? I’ve got a couple of ideas:
- Lack of Automation Expertise — This is of course the most obvious reason why people don’t use it aggressively. But that shouldn’t mean that you should eschew test automation as a strategy. There are plenty of companies who have the right expertise and automation frameworks to help.
- Wrong Automation Strategy — This of course pre-supposes that there’s a strategy in the first place and not a decision to just start automating cases. This problem manifests itself through wrong selection of test cases, inadequate maintenance considerations, and insufficient ROI models which lead to inefficient test automation with low ROI yield.
- High Upfront Automation Costs — That is if you’re starting from scratch. Highly maintainable scripts require careful planning and the development of reusable frameworks & components. If you don’t have these handy, this increases upfront investments and reduces time to value. Additionally, you need to drive towards lights-out execution to really ratchet up the benefits, but the ability to do that needs to be designed in at the beginning.
- No Attention to Script Maintenance — This is what usually kills an automation project. Teams begin an automation project, but then can’t keep up with the script maintenance. So the scripts get out of date, the benefits begin to melt away and the project seems like a failure and is abandoned. Common approaches such as "Record & Replay" yield fragile automation scripts and don’t work. You need to access re-usable libraries and modular, self-sufficient scripts.
So what’s holding you back from implementing test automation?
Of course if I’m all wet, let me know that too!
Glenn Gruber is AVP of Market Development for Travel Technologies at Ness. He can be contacted at: firstname.lastname@example.org. This post originally appeared, in a slightly different form, on Glenn's blog, Software Industry Insights.
Image Source: Velo Steve.
"Software people don't want to be measured," a friend recently told me. He added that, when it comes to software development, up till not, "efficiency wasn't talked about."
Indeed, when it is talked about, it is is often in the context of "the wrong way do go about things." For example, in this blog post on Agile Development, Mendelt Siebenga writes, "Efficiency is easy to measure and easy to manage. That's why it gets a lot of attention but by focusing exclusively on efficiency effectiveness suffers."
He goes on to argue that sacrificing efficiency in the name of effectiveness "will often turn out to be the right choice." Now, while I agree that no amount of efficiency will make ineffective practices effective, I don't believe that we can afford to choose between one or the other.
On the contrary, I believe that our methods, Agile or otherwise, need to be BOTH effective AND efficient. Moreover, I think that a premium falls on efficiency for several reasons.
1. Speed Matters
Competition in the software industry continues to be fierce and thanks in particular to innovations in the SaaS space, speed to market is becoming a major factor. Given the rapidity with which software developers can get their applications to customers via the "as-a-service" model, time lost during the development process, either due to ineffeciency or ineffectiveness (registered by defects in the code, for example), is essentially revenue lost.
2. In the Cloud, You Pay for What You Use
While speed of deployment is certainly one advantage offered by the cloud, people are even more enamored by the cost savings it offers. Paying for compute time as a kind of utility allows you to synch your computing needs and the money you are paying to meet them, thus eliminating the days of major capital investment and the scramble to make it back via optimized usage.
However, as Alex Handy pointed out recently, "Bad Code=High Bills." Handy goes on to say that, because code which runs inefficiently and thus takes up more CPU time actually costs you more money than properly optimized code, "The cloud can give you a direct line from your code to your dollars."
Of course, that line ultimately leads back to how efficiently, or not, you produced the code in the first place.
3. You Can Work More Efficiently, So Why Not?
The biggest reason to think about efficiency in software development, I believe, is the simple fact that we can actually do things more efficiently. Take test automation as an example. Technical methodologies available and in use right now can bring about vast reductions in testing time without compromising quality. Similar examples can be found at every stage and level of the development eco-system.
The real question becomes: If there is a way to work more efficiently, is there any reason not to?
Image source: Royal New Zealand Navy.
Satish Taribalalu, Program Director, Ness Software Product Lab
As the head of Ness's "Testing Council," I've seen the power of test automation first hand and have actively worked to gather best practices from across our many engagements.
Along the way, I've seen testing scenarios which used to take 15 days streamlined to just one night. And I've seen script-based automation deployed to enable test-driven development and improve overall product quality.
Based on what we've learned thus far, I recommend the following approach to automation.
1. Choose the Right Products for Automation
Automation can save time, but implementation takes time too, so you might not realize the overall time savings in the present release. You have to wait for the next one. For this reason, before you start automating, you need to ask about the product roadmap and where you are on it. If the product won't be around long enough for you to see any benefit from automation, don't do it.
2. Choose the Right Automation Methodology
Automation methodologies have evolved over time from the traditional "record and play" method to more contemporary, script-based and data-driven methods.
These methods are more efficient and produce greater time savings because they let you build in reusable templates and scripts. Not only that, they are easier for your functional and business teams to use.
Applying a framework focused on reusability and ease of use to your automation projects will not only give you benefits in the longer run, but will also lead to a faster adoption of automation.
3. Choose the Right Roadmap Strategy
Automation can't work in a silo. Not only does it need to support the development work, and even be integrated into it through a "test-driven development" model, but it also needs to satisfy the concerns of business owners, project managers, and product managers.
That is, even though the move to automation is usually a technical decision, not a business decision, it still needs to be justified in business terms. When doing that, you have to show that short term pain will bring a long term gain:
"To get automation up and running will require a two month investment and may increase costs slightly for the first two cycles, but, it will improve speed and reduce costs on the next 10 cycles."
On the production side, automation needs buy-in from both functional test teams and development teams. Involve these teams upfront and make them part of the automation journey.
Remember, as part of the normal software development process, testing can be very time-consuming, and that creates a lot of pressure. Many are tempted to save time by cutting corners on testing (by doing less of it, primarily). When you test less, though, quality suffers. That is a bad outcome.
When you automate testing in the right way, on the other hand, you save time, which then allows you to be more thorough in your testing approach, and that leads to the outcome everyone wants: a better quality product.Image Credit: