Interviewing testers, or anybody really, is a challenging task and it can be expensive to get it wrong. On the other hand, being interviewed, and showing who you are and what you can do is not straight forward either. The interview is generally led by the recruiting company, what they think they are looking for and what questions they think will reveal any valuable insight regarding the candidate’s fit for this context. Pattern recognition tasks, logic puzzles, and other relatively narrow “tests” still seems fairly popular among those few who don’t just go down the “how would you automate filling in this form” route, and It’s all relatively contained so you can time box everything in 10-20 minute sections. Add to this the desire, usually on both parts, to keep things short and “efficient” and it’s easy to start sliding from what you think is “a well considered empirical evidence based decision making process” to something more related to “lottery”.
The process we had at eBay here in London (or more precisely in Richmond) a couple of years back was one of the better solutions I have experienced on either side of the table. One of it’s strengths, I think, was that it allowed people to show what they considered their best side(s) rather than trying to tick a limited, specific set of (unknown) boxes on somebody else’s evaluation sheet. Depending on the candidate we discussed topics from UX to automation, from accessibility to infrastructure, test techniques, exploration, and all sorts of things in-between. Nothing is perfect of course and we did tweak it a bit over the years, but I think most people agreed that we got pretty good results. We got some good feedback, even from people who didn’t get an offer in the end. I, for one, think that counts for something.
“What are good results then?” would be a good question here so let’s start this off by diverging down that path for a moment; The most obvious part would probably be that as a recruiting company you want to find and successfully hire “the right one”. The problem is that there are relatively few absolutes in this world and in many cases you might not fully know what you need, what you’re currently missing, or where your blind spots are. In that case judging your candidates based on on your favourite pattern recognition puzzle might not be all that great, and making sure the candidate can list some arbitrary list of tools-of-the-month might not really solve your problem either.
Some strongly believe “hire for cultural fit” is the magic bullet. Others laugh at these people due to the risk of building echo chambers and thought bubbles. “Hire not for cultural fit, but for cultural ‘add'” they say. I’d say it’s probably not a bad idea to try to hold both of those ideas in your head. Rock-Star-Ninja-Prima-donas who are technically excellent but can’t work with other people will certainly add “something” to your culture, but that doesn’t mean it’s good. This goes both ways too. You want the candidate to feel welcome during the interview process and show that this is about learning about each other. It’s easy for the recruiting side to focus on whether this is the right candidate for them. But the company needs to be the right match for the candidate as well, and if the latter is not the case then neither is the first.
Note: as mentioned earlier there are very few absolutes in the world. Given that people seems to jump jobs about every two years here in London anyway the need to find the perfect company is probably not that strong for many candidates. As long as they get some useful experience and decent pay many people probably don’t really care much about what company they work for, or what their ethics are like (e.g. Facebook doesn’t seem to have any difficulty recruiting). Still, a bit of cultural fit, combined with a different perspective and some new blood for a bit of a cultural add is probably a good mix.
The challenge is then evaluating technical chops, cultural fit and add, collaboration skills, interest and willingness to learn and share knowledge, organisational manoeuvring finesse, domain and/or transferable knowledge, all within the tiny window of a couple of interview sessions.
So, in short; the candidate should feel they had a fair opportunity to show or tell who they are and what they can bring, as well as being given a feel for what the company and it’s culture is like. It is totally reasonable to frame the interview based on what the company (thinks it) needs but one of the marvellous things about testing, and testers, are that in the search to uncover the unknown unknowns you don’t always know what you need… and if you only use your testers to assert what you already know, I’d say you’re doing it wrong anyway (but that particular rabbit hole is for another post).
In the next “episode” we’ll take a closer look on how we approached this challenge out in Richmond.
+1 Testing 
4 thoughts on “One Approach to Interviewing Testers: Episode One”