One of the awesome things about the business we’re in is that you can ask people to actually do the thing you’re hiring them to do IN THE INTERVIEW! You can’t really ask a banker to do some banking during an interview, but it’s relatively easy to set up a computer with a coding problem and ask a potential hire to work through it. Having recently gone through a job search, I though I’d discuss my thoughts on coding problems as part of the interview process.

The first time I ever wrote code for an interview was at ThoughtWorks. It was a 3 day at home coding problem. There were always 3 different questions and each could be knocked out in a few hours. The candidate could choose which one they wanted to solve and upon submission, the solution was looked at by at least 2 different developers. The great part about this is that we could filter out a lot of candidates before bringing them in to the intensive day long interview. The downside of a take home test is that you never really know if they got some help. Which is why in this type of interview it’s important to go over their solution in person.

One coding interview I did was 2 developers and I working on a Rails plugin. It seemed like a good idea, but I don’t create Rails sites from scratch every day so there was a fair bit of fumbling around hooking up the database and trying to remember what generators to use and so on. When we finally got to the writing a plugin part I was a bit flustered and short on time. We made some progress but I don’t think either of us were satisfied with the outcome. This brings me to an important point: You will accomplish a lot less in an hour than you think you might. This is because the interviewee is using an unfamiliar machine, text editor, doesn’t have his or her custom bash aliases available, and is being watched. And questioned. All this will slow anyone down by 50%. So if it would take you 30 minutes to solve a given problem from scratch then it’s about right for an hour long pairing interview.

At Obtiva, Colin and I paired on some truly horrible code that he had inherited. Oh the stories we could tell you about that code base… But I’m not sure how I feel about these “work with me” interviews if they are going to last less than 3-4 hours. Actually, a day would be a better time allotment for this type of interview. The reason is that solving a problem in an existing code base requires a lot of backstory. If you only have an hour or two it’s hard to grok the domain and you may just be testing the candidate for familiarity with the subject matter instead of pure programming talent. It would be easy to miss a good candidate or hire someone who just happens to know the domain.

Speaking of domain, you should try as hard as possibly to eliminate it from your coding problem. I once did an interview with a firm where the challenge was to build a highly simplified version of something from their domain. The problem was that it really wasn’t simplified enough. After reading a page and a half describing the system my head was spinning with new terms and concepts (I wasn’t familiar at all with their field). They had helpfully started me out with some stub classes and some failing cucumber specs but when piled onto a shaky foundational understanding it actually hurt more than it helped. I had to figure out the domain, understand the class stubs (and any hints they may have been providing), describe what I was doing to the interviewer, understand what the cucumber tests were asking for, and navigate an unfamiliar workstation. It turned into a bit of a train wreck. Basically I did what I do when I don’t understand something: I went slow and tested everything. But I ran out of time before I could even get one of the cucumber scenarios passing.

You should try to structure the problem so that it has an easily achievable first step. This enables the interviewee to get his or her confidence up to try the trickier parts and also provides an easy hint you can give them if the are struggling. You might say: “Hey, why don’t you just try and get X working.” X being the easy part. You’ll get better results when people are relaxed.

At Backstop we have a couple of coding questions, but the central theme is that the domain is either something everyone is totally familiar with or we can teach them in a few minutes before the coding starts. We also make sure that the candidate pairs with different programmers to see how they interact with different people. Now pairing while interviewing is not like regular pairing, of course. When I run the interview, I approach pairing with an interviewee like I’m a novice developer who is a domain expert. I’ll answer any questions they have about the domain but I don’t help much with the coding part. I always allow access to any documentation they want to look up and of course I have irb open and ready for experimentation.

For me the pairing interview is just a way to have an engaging conversation about code. If someone doesn’t solve the problem but does so in an interesting and thoughtful way then I consider it a success. This also means that an unreadable working solution is considered a failure.

Having actual programing be a part of your interview process will take a some thought, but all that time is well worth the effort. I can’t tell you how many times someone has sailed through the talking parts of the process only to crash and burn in the coding section. Hiring someone who is great at talking about code but terrible at actually coding is one of the worst things you can do.