Home » English Contents

Laying a Foundation for Software Test Automation

16/12/2011 | a cura di Redazione Data Manager Online

by Randy Rice

Over the years, I have had the pleasure of helping many people in a variety of organizations implement software test tools. In that time I have observed what works and what doesn't work.

Most people agree on two big challenges of test automation: 1) unrealistic expectations and 2) getting management support for test automation. Of course, there are other challenges, but these are big ones.

On one project in particular I experienced first-hand how one of my clients skillfully handled both of these challenges in one effort.

A Little Background

First, let me explain a little about the business domain. This domain is highly complex both in terms of technology and business application - the financial sector. This organization was dealing with four different technology platforms and already had test automation in place for three of them.

I was hired to assess the level of test process maturity and the readiness to introduce test automation on the platform with no automation in place. To their credit, this client chose to get the process assessed before we embarked on a tool search.

One final piece of information is that the test automation in place was pretty dysfunctional. The automation on one platform has not been maintained along with the application, so it was unusable. There was no integration between the tools and the technology transfer between the consultants that created the automation and the client staff wasn't working, either.

Why is This Important?

Place yourself in the role of a senior VP in this organization for a moment. One of your test managers comes to you and says that to keep up with the rapid pace of delivery (over 500 releases in the main system in one year!), some type of automation is needed and it will probably cost over $250,000.

You're no fool. You know that the company already owns three tools and one isn't being used (and can't be used for the proposed situation). The two that are being used are being used very well.

So you fold your arms and ask the test manager to make his case.

Now, place yourself in the role of the test manager. How would you make your case?

Here's What My Client Did

1) He didn't promise quick results

Instead, he promised an effort that would start small and grow well. He told senior management that it would take 12 to 18 months to lay a good foundation for test automation. However, my client understood the value of early successes. We had some of those quick wins and they helped build momentum and buy time for continued effort.

2) He didn't oversell the benefits

Instead, he gave the reasons why the tool was a necessity instead of a convenience.

3) He got the process and skill sets in place first

Instead of placing the focus on a tool and what the tool could do, my client concentrated on understanding how the tool would be applied. He also knew that the tool would only be as successful as the people who used it.

Why Did This Work?

1) My client had credibility and management trusted him.

This is why credibility is probably the most important attribute a tester or teat manager can possess. Credibility equals trust.

2) My client had the courage to make an unpopular message.

It wasn't easy for my client to set expectations low, but he knew that high expectations were a set-up for disappointment.

3) There were at least three past examples in the company of how diving immediately into using a tool had failed or deliverer less than good results.

This gave some rationale for trying a different approach - start small and grow. Starting with small and simple test automation projects allows you to learn the lessons of test automation without the pressure of also dealing with high complexity. This approach also lets you publicize your success and gain acceptance from people who may be skeptical about the value of test automation.

4) My client backed up his approach with good information and frequent updates to his management

My client was always forthcoming and timely with updates to management. This is also why they found him to be highly credible. He gave the bad news along with the good news, but found ways to present solutions with the problems.

5) The company invested in their people to build skills.

The team hadn't been trained in years. We laid some basic foundations in test design and understanding what makes a good test. We also trained about designing tests with automation in mind. We trained the team in how to work well together to design good tests and then automate those tests.

6) We did a proof-of-concept (POC) instead of a quick evaluation.

This was very important and was a key success factor. In a POC, the tool vendor often comes to your company and works with your people and systems to apply the tool. There is typically a cost involved, but most vendors are motivated to reduce the standard consulting costs in favor of making the sale of their tools. Most vendors would much rather have the opportunity to showcase their tools in positive as opposed to a potential customer struggling with the tool, only to give up. In our first POC, the vendor's consultant actually completed the automation of one application! It was a win-win-win. The vendor sold tools, my client looked great to his management, and I looked great for suggesting the idea.

7) We were able to show some quick wins to build momentum.

These early successes allowed us to learn lessons with lower risk. The momentum from those early wins let us even faster. By going a little slower at first with automation, we got faster quicker.

You Can Do This, Too

You may be thinking that you can't imagine your management accepting this approach. Believe me, I understand your skepticism. However, someone must manage the expectations of management and the rest of the organization.

I don't have a formula or process to build this kind of foundation, but I do have some principles:

1) Be open and honest with your management.

When there's bad news, get it out fast while there's still time to make a reasoned response. In my experience, bad news usually does not get better with age.

2) Keep the lines of communication open.

You may need to remind your management that you are building a foundation or framework.

3) Go for the quick wins and publicize them.

These successes don't have to be big. Look for situations where time was saved, testing was done better, and/or people are freed for other work.

4) Get the process and skill sets down first.

Before you get on stage and start playing the instrument, learn a song first. I know that's not about testing, but who would take this approach to playing music? It's the same principle with testing. The process and skills guide your decisions about what, when and how to test.

The tool will likely force changes to your process - and that's fine. Tools, people and processes usually grow together.

5) Be flexible.

Be willing to try new approaches, even in the process of deploying the tool. Listen to the team and try their ideas.

6) Learn from your mistakes (and those of others).

I like to learn from others' mistakes, since that's less painful than learning from my own. However, we all make mistakes, so we should learn from them. Unless you have a time and place to reflect and discuss these lessons learned, you probably won't do this. A great book on the topic is "Project Retrospectives" by Norm Kerth.

Summary

I hope you'll take this case study to heart and try this approach, especially if you have tried implementing test automation in the past, but have not fully realized your goals. This approach overcomes two of the biggest challenges in test automation - unrealistic expectations and getting management support.

You don't have to be super-human to do something like what my client did. However, you do have to educate management and work with them to understand the issues, risks and benefits of test automation.

If you can keep expectations in line and keep management on your side, you will find that in a year or so, you will be in a place where the elite minority reside - in the successful implementation of test automation.

-------------------

Randy Rice e' un autorevole esperto di fama internazionale nei settori del Software Testing e del Software Quality. E' un Certified Software Quality Analyst, Certified Software Tester, Certified Software Test Manager e ASTQB Certified Tester, ha lavorato con molte organizzazioni in tutto il mondo per migliorare la qualità dei loro sistemi informativi e per ottimizzare i loro processi di Testing. E' membro dell'American Software Testing Qualifications Board ed è editore di The Software Quality Advisor. E' co-autore con William E. Perry dei libri "Surviving the Top Ten Challenger of Software Testing" e "Testing Dirty Systems" pubblicati da Dorset House Publishing. E' stato Chairman del Quality Assurance Institute's International Software Testing Conference, membro fondatore del programma di certificazione CSTE (Certified Software Test Engineer) e ha fatto parte del board of directors di ASTQB (American Software Testing Qualifications Board). Nel 1990 ha fondato la Rice Consulting Service.

Randy Rice presenterà a Roma per Technology Transfer nella primavera 2012 “Testing di sistemi Legacy complessi e non documentati” e “Software Test Automation”.

 

Nessun voto finora

Tags: