So, now that we have thoroughly and unequivocally defined what “good results” mean (yeah, right) in Episode One, let’s return to what we did out in Richmond to achieve just that. Hopefully this approach gave our candidates the opportunity to express what they felt was their strong skills and let them “sell” themselves in a best possible way, as well as getting a feel for what our organisation had to offer in return.
As most other places, I guess, most candidates came to our attention either by applying directly or through one of our small group of trusted recruiters. Reading through a high number of CVs in a short time can be quite daunting and rather mind numbing to be honest. You try your best to be fair and reasonable without spending too much time, but you’re also limited to work with what you are presented. Looking at my own CV I probably shouldn’t be slagging other peoples CVs off too much, but there is a lot of very unimaginative stuff out there to say the least. If the most exiting things you can say about your work experience and responsibilities is a bullet list of “Writing test cases, execute test scripts, raising Jira tickets, triage Jira tickets, verify bug fixes, and closing Jira tickets” then it can be challenging for the one reading it to become very exited about having you join the team to be honest.
Things that would make candidates stand out a little was if there were anything indicating some interest in learning and developing as a tester, or helping others do so. Did they mention anything like mentoring other testers, Exploratory Testing (or other approaches or techniques at all)?
Any mention of “context”, “learning”, or “thinking” would probably give you bonus points.
After finding an interesting looking CV we would follow up with a half an hour (maybe 40 mins) phone chat with the candidate. Other than the regular “Who are we, and who are you?” intro we would normally be picking up on clues from the CV to see if the candidate was just playing buzzword bingo. Did they actually know anything about the things they mentioned on their CV? Could they explain their interpretation of concept X? Words and concepts might mean slightly different things to different people and that’s fine. It can lead to interesting discussions and bring an unexpected perspective to the table. However, if you claim to be a Grand Master Exploratory Testing Wizard with a strong track record of mentoring others in the Art of ET, and then tell me it’s all about randomly poking things… well, let’s just say your bonus point might be retracted.
The next stage would be a bit of homework.
Note: Yeah, I know. Nobody likes homework and there are some problems and challenges associated with it. I’ll get back to that a little later. For now; we asked for it. Deal with it.
For the “Home Assignment” we would share a small codebase for a Sudoku puzzle REST-service written in Java with instructions how to run it and what endpoints were available. We would then ask the candidate to test or evaluate this “product” in whatever way they saw fit. This means they could:
- Look at the code
- Look at the unit tests
- Run the app and functionally test it
- Focus on the API
- Perform a risk analysis
- Describe how they would test for performance risks, and why.
- Do some security testing or analysis
- …or any other approach or technique they felt would be useful
Either or any approach would be good and we asked to please don’t spend more than an hour to 90 minutes after setting it up. We tried to make it clear that we would not be looking for a comprehensive, exhaustive test session, but rather an example of thinking and area of expertise. The candidate was asked to send us their test notes for the session for us to review. This part made it clear at one point that we had some different views internally on what exactly we meant by “test notes” 🙂
Finally. Based on the information gathered so far we would hopefully be able to decide whether a candidate would likely be what we were looking for or not and they would be invited to our offices for what we called a “Sprint In A Day” (S.I.A.D.). The S.I.A.D. would include some hands-on testing, (contribution to) planning and prioritisation, pairing with developers, a chat with the other testers, and a short interview focusing on behavioural, organisational, and agile topics. The point was to give the candidate some idea about how we were working, what kind of contribution was expected of them, and for us to see how they reacted and get an idea of what they could bring. As you might guess, this last stage was quite “expensive” with regards to both our people taking time out from their daily work as well as the candidate having to take a day off. Due to this we really wanted to make sure we had a good feeling before making this decision.
Stay tuned for Episode Three where we dig into the details of the Grand Finale that was the “S.I.A.D”.