Software Testing: Is Your Quality Assurance group a Thermostat, or a Thermometer?

I frequently read a daily thought on families and how to improve your family. Yesterday’s thought dovetailed neatly into something thinking I had on QA organizations: are you a thermostat, or a thermometer?

A thermostat is a device which regulates the temperature in your home. It controls the atmosphere. A thermometer is a device which reflects the temperature. To be a thermometer is to be reactive; all it does is indicate the temperature. To be a thermostat means to proactively set the temperature.

In the family, a ‘thermostat’ parent doesn’t react to grumpiness or other negative emotions, but calmly sets the tone in spite of what’s going on around her or him. A ‘thermometer’ parent, on the other hand, gets ‘set off’ easily when children bicker, whine, or complain. Obviously my goal as a husband and father is to be a thermostat, although there are ‘thermometer-like’ days!

In a recent thread on the Yahoo Agile Testing Group (about continuous release), I noticed more than one poster saying something like “I tell management about an issue, and let them decide.”  This reactive approach to QA has been a pet-peeve of mine. It smacks of cop-out to me, and I’ve seen it develop into a victim mentality among many testers—myself included.

In my tenure at Microsoft (where I’m ‘serving’ my second stint, with a combined tenure exceeding 12 years), the testing organization has been consistently active in pushing for quality. A few things differentiate Microsoft from other organizations: first, the engineers wear the pants (that’s changing, slowly). In fact, nothing ships if the program manager, development manager, and test manager sign off on it. And they won’t sign off unless their teams sign off. Secondly, any individual has the right to stop a project, even the newest tester in an organization. Third, while the management structure is massively deep (I’m some six or seven layers below Steve Ballmer, current CEO and President), any individual can get the ear of their VP, Senior VP, and CEO.

So the culture I’m most familiar with is one wherein testing is an active participant in product development. The testing organization moves beyond a ‘show and tell’ relationship with management to an active participant in business strategy. The testing organization is part of thermostat, and rarely plays a thermometer role.

I have worked in organizations where this is not the case, and I’ve seen a lot of evidence that this often isn’t the case. In many IT and product software organizations, testing plays a thermometer role. We ‘take the temperature’ of a piece of software, and let management decide when to ship. In fact I have been told by management, when I’ve tried to be a thermostat, to back off and let them make the decisions. In one case, I have even refused to sign off on a risky release (security-related issue on a piece of beta software), telling the release organization they were welcome to release but they’d have to convince my manager to sign off because I wasn’t going to.

The Testing Thermostat

Thermostats regulate the temperature. In my “Quality 2.0” vision, testing organizations participate in regulating quality. The test organization moves from just taking the project’s temperature, to a role where they are helping to set quality. To do this, testing teams need to:

• Include experienced, respected engineers. At one time in my career, I worked with Christian Hargraves, probably one of the most experienced engineers I’ve ever met. Christian led and maintains the Jameleon project. He’s a sought after expert in the Salt Lake IT market, and has worked for the LDS Church as what I would call a Test Architect. Every developer in the organization knew Christian knew his stuff, and if Christian recommended something, that had weight. Qualified, respected engineers who are part of a test organization can help contribute to quality process and practices across the organization. They can influence core development teams to push quality into their process, from code reviews to adopting best practices.
• Advocate actively about projects. Testers need to be active participants in release decisions, influencing senior management. In many instances, testers are shareholders. In every instance, testers share the benefits of a good release by enjoying continued employment, and in ‘worst case’ scenarios, testers go down in flames along with the rest of the company. In other words, our employment is just as dependent on a good release as any CEO’s. If all we do is report quality, without influencing decision, we are placing our career in the hands of other people. I do not accept this point of view!
• Drive quality upstream: we all know it’s significantly cheaper to fix a bug in the planning phase, early implementation, or even release validation phases as opposed to post-release. Testers need to work to push themselves and their influence upstream—the earlier we can look at code, the better.
• Be involved in planning activities. Take an active role in scheduling. Drive feedback about previous releases (especially lessons learned) into current project planning. If the company has a history of underestimating test duration, speak up about it. Track and present data which shows a more realistic approach. Work patiently but consistently to drive reality into schedules.
• Hold dependencies accountable. If the development team isn’t dropping code early enough for full testing, don’t stand for that! Your reputation as a tester, not to mention your continued employment, depends on a solid release. If the company runs with firm release dates, but development is dumping code in your lap a week before release, you’re going to end up working 90+ hours that final week, and still releasing crappy code! Negotiate and set milestones, and hold development accountable for reaching those milestones. Two things will result: first, your test organization will start to work a consistent, sane amount of hours. Secondly, quality WILL go up because you’ll be looking at code earlier.
• Be involved in establishing release criteria. This is critical—across your business, you need to have consensus on what constitutes release-readiness. If you leave this up to marketing, they’ll say a set of customer stories need to be developed. Leave it to development and they’ll say that, plus possibly that all unit tests need to pass. As QA engineers, we have a unique perspective in that we understand the value of positive AND negative testing. We can help influence the establishing of release criteria which truly reflect product quality. Setting this criteria early, well before a project is nearing a release deadline, removes the emotion and sets a rational basis for declaring ‘Ship Ready’.

These are just a few things a test organization can do to transition from a thermometer to a thermostat. What’s missing? Does your test team take a more proactive role somewhere else? Let’s hear it!

Technorati Tags: quality assurance,qa,software testing,quality