Yes, I think this is a very reasonable approach. It will have a slight bias towards “talkers” - some people do tend to be shy, and as long as you are willing to spend the extra time to put them at their ease by guiding the conversation this gives a decent idea.
What I do for every interview is use the candidate’s CV as an ice breaker to get them talking and into a comfortable zone. Then I ask them to work together with me on a problem I’m currently working on. This is the “colleague simulator”. Why are we hiring this person? To be a colleague. What does a colleague do? Work together on problems with us. This gives insight into how well folks communicate, how quickly they pick up on the core aspects of a problem, how flexible they are, how respectful, how engaged. An interview is not a multiple choice test. I hear we have machines that can do THAT.
What do you do with people who haven’t been employed in the field before? Or do you just not interview such people?
Same thing, actually. Our field (bioinformatics) is fairly young and pretty interdisciplinary, so most of our candidates come from statistics, physics, software engg, bioinformatics proper (a few). Every problem that we touch has many aspects and folks will always find something to apply their prior knowledge to. Then part of the process is to introduce them to new ideas and see how they deal with that.
This is not a technical question, though it is a decent set of questions.
Do a work sample with an objective rubric and take feedback based on the performance of hires and non-hires.
It’s amazing that, “ask someone to do their job under normal circumstances and evaluate the result” isn’t the norm.
It isn’t really feasible unless your company has huge resources.
Many people have a job already and “normal circumstances” need a ramp-up time. Their performance will be influenced by that. Some developers are not good under stress (some excel under stress!). Some feel like this is an exam situation which a non-trivial part people have an unreasonable fear of. Also, Interviews conducted by untrained engineers tend to be biased towards their knowledge and interests, not the companies.
All this is incredibly hard to set up for small to medium companies, as it needs a dedicated hiring process, someone caring for it and trained interviewers.
I do enjoy having some technical part in an interview situation, but let’s not pretend it comes any close to the performance they will have on the desk later.
I guess it’s not amazing after all.
The problem, I think, is that it’s very difficult to create a work sample that fits any of the criteria you’ve listed: normal job, normal circumstances, and fairly evaluated.
The normal job criteria fails because every job is going to be a bit different, and in most cases a job is going to involve working with peers in an environment that you’ve had a hand in building and been able to spin up on. Because you’re in an interview situation the people you’re working with are evaluators, not peers, which changes the power dynamic. The environment is also going to be unfamiliar, and you won’t have been in a position where you’ve been able to spin up on the tech or culture being used. For more senior people this may present less of a problem because they should have a large enough breadth of experience and enough adaptability to be able to navigate unfamiliar environments and languages, but for junior and mid-level people and interview-length project (even on the order of a 1-2 week take-home project) isn’t going to be enough time.
The circumstances are also going to be different. The power differential noted above aside, there’s the problem that most people are already working a job when they go looking. The extra hours and severe context switch of going from day-job to interview-project are going to be a pretty significant deviation from the way a person would normally work.
Finally, the fair evaluation. I’m assuming when you say “evaluated” you are meaning some sort of fair and as-objective-as-possible evaluation. This is going to be hard because it’s going to strongly favor candidates who are most like the existing team. Leaving aside personal implicit bias regarding race, gender, age, etc. and assuming we’re talking about a black-box project, because of the factors above related to the timeframe, availability of peers, lack of contextual background, etc. you’re going to be strongly bias toward people who are already working in environments like yours, and with your technology stack. This can be a good thing if you need to scale up your team quickly, but at a rather severe cost to diversity of experience with different tech stacks, languages, backgrounds, etc. In the long-term you’ll be hobbling your team’s effectiveness by tending toward a monoculture of people who passed the interview thanks to a predisposition to thinking the way your team already things.
It’s also pretty reasonable to expect a 3-6 month “settling in” period for development work. You might know $TECHSTACK cold, but you don’t know our implementation at all, our idiom, and our quirks. Just chucking a real-ish problem at someone and expecting them to solve it probably isn’t going to be a good sample.
Honestly, the best approach is to do a few short interviews to establish: 1) they’re not a con artist, 2) they’re not a lunatic, then bring them on as contract-to-hire to see if they work out.
would work well for senior engineers, not as much for people whose work experience has been coding to requirements laid out by people higher up the chain.