DRAFT: What’s the Role of a Test Manager in Agile Organizations?

This blog posting is a draft. It represents a work in progress, and is posted with the hope of gaining peer review for improvement.

Many people have asked me—or challenged me—about the role of a test manager in agile organizations. A basic premise of agile being the self-lead team, who needs a test manager? Many agile engineers even bristle at the thought.

Reality shows, however, that not all teams or organizations benefit from a complete lack of management. Smaller engineering organizations can often do without, but larger organizations perform better with some amount of centralized management. A team of ten engineers, with no role specialization, has no need for a test manager. A team of fifty engineers, which maintains a differentiation between test-oriented engineers and development-oriented engineers for a valid business reason will certainly benefit from a test manager (and, incidentally, a development manager).

What do I think is the role of a test manager in such an organization? I think the test manager contributes in five higher-order functions: interfacing with senior management; providing vision and leadership; fostering cross-group collaboration; leading specialties, tools and reporting; and providing organization management. That’s a lot of Dilbert-esque lingo! What do I really mean? Read on for details…

Interface With Senior Managers

Larger organizations, such as the team I led at Circuit City, have deep management structures. It’s simply a fact of life. And those managers generally manage cross-discipline organizations and benefit from help with a manager between them and each discipline. In Circuit’s MST project, I managed over one hundred engineers, with thirteen concurrent workstreams. My role was to interface between my manager and the test organization, communicating messages and centralized decisions in a way that addressed my teams’ needs, interests, and concerns.

Test managers also act as a buffer, insulating the team from senior management’s see-saw decision-making process. Sometimes just shielding testers from the questions senior management can ask is performing a huge favor—questions like “do we have the right number of testers” can really cause concern in a test team; the test manager can research and answer that question often without churning up any worried or troubled feelings.

When senior managers arrive at a strategic plan, the test manager’s job includes integrating that plan into the test organization. A great example would be an organization which has decided to migrate from waterfall to agile! The test manager’s role will change dramatically, moving from ‘manager’ to ‘coach,’ assisting the test team in understanding their role and helping to set engineering goals in terms of process adoption.

Another role test managers play in this larger organization is simply pushing back on bad ideas. Individual contributors are often heads-down (especially on agile projects), in the day-to-day work. If senior management stumbles upon a ridiculous idea, individual team members often lack the cycles (and sometimes the experience) to push back on that decision. The role of a test manager often includes helping senior management understand the possible side effects of a decision.

In some organizations, testing or quality assurance are not often viewed as critical. These organizations may lack experience, or may have experience in smaller projects than the one at hand. A test manager’s role includes the responsibility to advocate for quality across the entire company. Sometimes this means helping customers understand the value of the ‘extra cost’ of a few test engineers on a project. Other times, it’s helping executive management see the need for experienced testers. Sometimes the job includes preventing releases from happening when core quality issues beyond simple functionality could cost the company in terms of money or bad PR.

Provide Vision and Leadership

Agile teams are (rightfully) quite busy in the day-to-day and may not be cognizant of everything going on in the company. The test manager’s responsibility in an agile organization can include driving vision. With an understanding of company growth plans, the test manager can be looking ahead to understand how to double or triple the size of the test organization. The test manager can also be thinking about strategic redirection – for instance, perhaps the test organization depends on developers for all automated testing. The test manager can set in place organization-wide plans to train testers, helping them become better automation engineers. The test manager needs to be the person asking “Why not?” and helping to drive the team forward toward that goal.

Sometimes agile teams fall behind due to circumstances beyond their control. The agile approach is to take those hits, learn, and move on. But sometimes there’s value in getting help. For instance, if a member of an agile team becomes ill during a critical period, a strong test manager can step in and ‘pinch hit’ for that tester while he or she is out. This isn’t advocating that the test manager is simply a ‘reserve’ resource. Teams which underestimate or over-extend themselves need the experience which comes from missing goals. But there are times when a test manager can contribute at an individual level.

Another leadership role is to help teams through transition. The change from waterfall to agile is a fantastic example of this – as a leader, the test manager needs to play a critical, positive role in helping testers work through the difficult migration process. The test manager provides structure and understanding, offers insight, and encourages team members as they face challenges. Above all, the test manager needs to be supportive and reassuring during the struggles which come with change. This leadership will help retain top talent, which might otherwise leave the organization due to the uncertainty and stress of change.

Finally, the test manager can represent the test organization and the company as a whole. A good test manager can be a key asset in customer retention. By contacting external customers, being aware of situations they’re facing, and communicating the status of fixes and changes, the test manager can help the customer through any rough spots in project implementation. Another role the test manager can play is sitting as a member of industry or standards organizations, making sure quality has a voice at that table and providing expertise in the subject.

Foster Cross-Group Collaboration

A critical role played by any management in an Agile project is eliminating road blocks and keeping the team moving. The test manager can play a pivotal part in this by interacting with other groups, with the business side of projects, and with the customer. Sometimes it takes a management-level title to ‘influence,’ even within the same company. More important, some negotiations and even simply driving closure can involve a lot of time, talking, and walking. By remaining proactively involved in projects, the test manager can help remove impediments, all the while allowing the team to remain heads down.

Also, very often in a larger organization, the test manager is the first level team member with corporate purchasing privileges. Therefore, the test manager can often take care of licensing and other purchases and fees for supplies the team needs to carry on their work.

In larger engineering organizations, the test manager works closely with development and program manager/project manager counterparts. They collaborate on cross-discipline objectives as well as on budgets and scheduling. They are often the point-people for driving mitigation for lessons learned which surfaced in retrospectives.

Another role the test manager plays in a larger organization is introducing engineers around the company. A successful test manager is very familiar with projects and personnel across the entire organization; when one employee is looking for help, that test manager can make appropriate introductions.

Finally, the test manager can play a scrum master role in a scrum of scrums, helping to lead the meeting.

Specialties, Tools, and Reporting

Generally test organizations are split into teams in  a matrixed structure. Most testers fall easily into a project team, where they’ll be busy until that project completes. There are, however, occasions where testers are hired who act in more of a ‘Center of Excellence” function. For instance, performance testing, automation and tools, or security-oriented testers. These testers are shared across the organization, brought in as experts to play a specific role on a release. Organizationally it’s often most efficient for these testers to report to the test manager, and the test manager manages their assignments.

In a large Agile organization, there is often tension between the Agile maxim “do what works'” and organizational efforts to standardize, especially on tools. Standardization sometimes aids in keeping costs down (when expensive commercial tools are used), especially for tools like load testing tools. Standardization also aids in maintaining resource ‘portability’ (the ability for a tester to move from team to team, without needing to learn new tools). A test manager needs to be very sensitive to the needs (perceived and real) of a team, but can often be instrumental in maintaining consistent toolsets across the organization.

Agile is notable, among other things, for a distinct lack of metrics. Status reporting is as important to senior management as it is a burden to agile teams. The test manager can play a critical role in pulling together status out of several test teams, presenting it to management in a format they’re used to. It seems a bit mundane, but in spite of the gains made moving to agile, many senior managers still yearn for detailed status reports.

Related to reports is the idea of common metrics. Just as the Agile team needs to ‘do what works,’ the Agile organization also needs to do what works. Sometimes that means a bit of a burden on the Agile team. With engineering expertise as well as an understanding of project management, a test manager can help Agile teams arrive at metrics which are easily generated (often times in an automated manner, if unit testing or defect tracking systems are in place). There are metrics that work – they vary from company to company, but they work—and the test manager is a key player in defining these metrics and automating the gathering process.

Finally, the test manager can offload the ‘compliance’ burden from the Agile team, reporting up to senior management on the progress against various company initiatives. Again, maybe not exciting to the test manager, but critical to the company’s improved effectiveness and efficiency (well, we like to hope each initiative is critical).

Organizational Leadership

Even in the Agile organization, things like performance management have to happen. The test manager carries a significant administrative burden driving this process. This is a good thing – a good test manager not only helps his or her teammates grow in skills and experience, they also advocate with senior management for promotion and compensation increases.

As a test manager, my experience has always been that my teams are more effective when I screen candidate for open positions, and then pull team members in on formal interview loops. As a test manager, I have had responsibility for opening job postings, authoring job descriptions, and sourcing resumes for open roles. When the test manager takes on this administrative burden, it allows the Agile team to focus on their work. When decent candidates are sourced, the team can stand down briefly to perform interviews.

Closely related to recruiting is the role the test manager takes on in hiring and onboarding new hires. The manager shuffles new hires through the hiring process, helps in signing up for benefits and such, and makes sure the new hire settles in well. A key role I have seen too many test managers overlook is introducing the new hire around the company. Finally, the new hire will achieve their highest potential as quickly as possible only if and as the manager assists them in setting goals for impact, development, and growth.

A test manager’s work is to drive his or her team to higher levels of effectiveness and efficiency. Working in concert with members of the Agile teams, test managers can accomplish this in part by researching and procuring appropriate training. Whether it’s bringing in experts in Agile testing or arranging for on-site training in automation skills, training is critical for improving what a test organization can accomplish. Procuring that training can be a time-consuming task. Also the test manager arranges for training which fits broad, cross-organization needs rather than focused on specific teams.

Finally, the test manager is critical in helping with employee retention. A good manager is aware of the interests and professional goals of each employee in the organization, and makes the effort to provide that employee with opportunities in sync with the employees wishes. With a broad, cross-organizational focus, the manager can also help employees find new roles when, without that assistance, the employee might have left the company.

Help the Agile Process Work

A closing concept for the test manager’s role in Agile is the idea that he or she is a make-or-break participant in the transition to Agile. It can take several months or even a year to successfully transition to an Agile process, and that transition can be painful—especially for testers! The test manager helps coach teams through the transition, aiding and lending support and encouragement, fostering courage, and assisting in power transformation. This is not a tops-down role, but a one-on-one, coaching role helping team members with courage, patience, and confidence.

Technorati Tags: quality assurance,qa,testing,software testing,agile