Recruiting for Testing Skills

I’m a test manager, so ultimately my interviewing is to find candidates who can test – who can find bugs, prevent bugs, and write tools to help in those processes. Testers have a special mentality—seriously, I’ve interviewed (and worked with) plenty of solid development engineers who couldn’t test their way out of a paper bag. It’s a special person who can code AND test. The good news is, finding a person with a ‘break it’ mentality is pretty easy. Probably the easiest of all the characteristics I interview for.

The best tool I’ve found for this is to simply throw the candidate a test question. It can be something theoretical, or it can have a real-world application. One of my favorites is to test a function which receives three integers representing the three sides of a triangle, and returns whether the triangle is equilateral, isosceles, a triangle, or invalid. This question works really well; it presents the candidate with a real world situation (who remembers what an isosceles triangle is anymore?), it has a ton of effective cases, and it can be easily moved into additional problems such as becoming a web service or the likes.

What’s the right answer? Well obviously candidates are going to have to catch the positive cases of each type. But from there the sky’s the limit—I’m not going to list all the cases here but suffice it to say that I was once interviewed by a test manager and given this question. We went on for a good 30 minutes and could have continued. And that’s one of the key points I’m looking for – does the candidate throw out a series of cases over a few minutes and give up, or does she keep going? A real tester should be able to run on and on and on with new cases, if your question lends itself to this. That’s why the question is so important.

So I’m looking for my candidates to just keep generating test cases. I’m looking to make sure they cover the basics – positive cases, boundary cases, etc. I’m going to keep an eye out for candidates who aren’t equivalence-classing their cases. A candidate who tells me 1,1,1 2,2,2 3,3,3 and 5,5,5 isn’t getting anywhere; there’s no difference in any of these cases (well, maybe 1,1,1). So rambling along with repetitive cases is no help. But going on and on and on with cases is definitely a key.

Another thing to look for – is the candidate organized? I’ve interviewed with a number of leads and managers who claim no candidate is sufficient who isn’t organized in their approach. I disagree with this somewhat—a candidate who, through free associate of thought, jumps from performance to security to positive test cases isn’t necessarily incompetent. As long as candidates 1) cover all those case types and 2) make sure they get the required coverage, that’s all I’m looking for. I happen to be rather random in my own test case generation. I may use an outline, but I’m plugging cases in all over the outline rather than working top to bottom. It’s simply how I think. Don’t mistake the trigger of association for disorganization.

Another thing I’m looking for is the candidate’s ability to think like a user. In testing a problem like the triangle evaluation, are they coming up with good use cases and showing customer empathy? I consider myself a technical guy and I’m always looking for good perf bugs or bad data manipulation or the likes. But the most recent bug I wrote up (just yesterday) had to do with how difficult it is to perform a certain workflow procedure in our line of business application. Nah—it’s not engineering. It’s about solving the problem the right way for the customer. In our organization, the IT department has made huge progress but still has room to go in winning over customers. Many groups are still working their own solutions rather than using our expertise. Fixing weird useage scenario bugs like this has an amazing positive effect on the department’s reputation for being client-centric.

So don’t just focus on whether the candidate got the obvious cases. Look deeper and evaluation for thoroughness, for customer empathy, and for the ability to derive great cases out of what seems like a straightforward and obvious tool. Your department will benefit greatly from this kind of test resource!