I see a lot of (justifiable) hate for standard industry interviewing practices: whiteboard coding, reversing lists, take-home assignments, there’s a daily-blog’s-worth of interviewing horror.
Have you ever been on either side of an interview, or seen an interview practice, that was really stellar? That you thought really revealed either the team fit or engineering abilities of the candidate?
Personally, I don’t think I have, and I’d love to improve my skills as an interviewer beyond “find a better word problem some can solve in 45 minutes”.
The best interview I had was at the place I work right now. Absolutely no leetcoding. It was entirely verbal. 4 rounds:
That said, we also do take-home challenges for software engineering roles and I think that’s perfectly fine, as opposed to whiteboarding. A small task, something that can be solved in about 2–3 hours and not more; a call a few days after to discuss your solution.
I’ve seen some companies have a “work day” interview, where you work with the team you’re interviewing for, for a day—starting from the standup call, doing your assigned task, and a review at the end of the day. This is a great way to assess the candidate’s fit with the rest of the team—that’s what ultimately matters at the end, anyway.
I hate take home assignments with a passion. They’re so much worse than whiteboarding.
Either they’re so small they don’t show any more than a whiteboard session would, or they’re massively disrespectful of my time. And because the interviewer isn’t in the room with the candidate, it’s easy to do the latter: the time investment is one sided.
Many good candidates with jobs, families, and other commitments don’t have a day to be screened by every company they want to talk to.
Basically, I wish companies offered both options. I get really nervous during interviews and would rather just spend a few hours beforehand working on the assignment, so I can come to the interview confident and ready to talk about my solutions. As long as the scope is reasonable I don’t mind it too much. For other candidates who might not have time, there should definitely be a self-contained option.
I have gone through home assignments twice in my career. Both have asked for me to bill my time and paid it with a fair rate. I think that is the only correct way to do home assignments. Others should be rejected on the spot.
I spoke to a company the beginning of this year and they were doing an open source product. Their assignment was basically “pick one of these GitHub issues and send us a PR.” The issues were somewhat trivial - fixes in config file parsing [1], that mostly aimed at showing you could find your way around the code. It looked like half a day’s work. I rejected the company for other reasons and ended up not doing the work on that assignment, but I liked the setup nonetheless.
Those were not screens - I have gone through the screening steps beforehand. I agree that having day long screens is terrible. Unfortunately the alternative isn’t much better. I’ve went through “two hour” coding/algorithms screening tasks and, if you add the prep for that, you can easily land at half a day’s work. Yes, I hate HackerRank “exercises”.
[1] I know many people are reluctant to do “free” and “real” feature work for a company that they have not started actually working for, but in this case it was obviously not something that was a core product feature, and, as I already mentioned, was paid in full. If you are asked to do free work, run away and never look back.
If they’re asking you to complete real company-related tasks I agree. However, what if they’re asking you to do the type of meaningless problems you might find in an interview? Suppose we’re past the screening stage. Normally candidates aren’t paid for their time in a regular onsite interview, beyond travel accommodations, so I don’t see why it should be any different for a project done at home.
Normally an onsite interview costs the company engineer hours. A take home interview costs the company nothing. When interviewing costs nothing, the tendency is to throw shit at the wall and see what sticks. This isn’t theoretical – I’ve seen people say “we don’t have the bandwidth for interviewing, let’s do take-home assignments before we bring them in.”
The money isn’t enough to make a difference in my life – but its a signal that my time’s not being wasted.
They’re still an interview format I find to be an unpleasant time sink compared to whiteboarding, but that’s a personal preference. If a company is paying, it at least indicates they’re trying to be thoughtful.
That is not true at all in my experience. The company doesn’t have someone sitting in the room with you, so the time cost is invisible to you as a candidate, but someone has to evaluate your code after you submit it. I did a bunch of that at my last job and doing a thorough code review including coming up with followup questions typically took more of my time than an in-person interview slot would have. And we always had at least two reviewers for each submission.
Replace ‘thorough’ with ‘superficial’ for all but the few applications you like the most, and you’ve got the approach I’ve generally seen taken (or advocated) with take-home interviews.
There are certainly places where this isn’t the case. Paying for the interview is a way to convince me that you’re one of those places.
We do blind reviews. No resume. Just whether it’s product or platform, as those tests are different. I always read every word written and often take the time to figure out how much work would take to get it working. I approach it much like a random github project or PR. Is this something I can build off of? Can I drop it in and use it? Or is it mostly there and I can quickly fix a bug? On the other side of the spectrum, do I have to do most of the problem to get it working? Do I have trouble understanding how to even get started running or even reading the code?
I see you’re working on some open source projects. Out of curiosity, what proportion of interviewers ask you to walk them through your contributions?
It came up at most of the smaller companies.
However, the open source code I write is also not usually directly relevant to the work that I would be doing (intentionally, for a number of reasons, including being in a small niche, avoiding non-compete issues and burnout).
The better interviews I’ve done have been a mix of whiteboarding system architecture, and pairing on debugging and writing a project.
But the thing that really makes interviews fun is having a competing offer in hand. Highly recommend it as a stress reduction strategy.
This approach works well for me both as an interviewer and as an interviewee when coupled with a code walkthrough. E.g. Walk us through a recent project. What does it do? How’d you start it? What was the hardest part to write? What did you learn? What would you do differently? What were the most frustrating limitations of the tools you used? Many of these questions lead to follow-up questions and then settle into an illuminating, non-adversarial conversation. Open source or otherwise unrestricted code is preferable, but a take home assignment suffices as a substitute.
This was part of our interview process as well. After a brief phone interview we would have the candidate login to a remote system (this was all on clean VMs). You would do this alone. Then we’d call back and have the candidate walk through the code that they wrote, explaining the design choices made and their solution. This was for a web based software development shop so we had front end / back end specific tasks.
My favorite interview ever was a systems interview that didn’t go as planned. This was for an SRE position, and while I expected the interview to be a distributed systems discussion, the interviewer instead wanted to talk kernel internals.
I was not at all prepared for this, and admitted it up front. The interviewer said something along the lines of, “well, why don’t we see how it goes anyway?” He then proceeded to teach me a ton about how filesystem drivers work in Linux, in the form of leading me carefully through the interview question he was “asking” me. The interviewer was incredibly encouraging throughout, and we had a good discussion about why certain design decisions worked the way they did.
I ended the interview (a) convinced I had bombed it, but (b) having had an excellent time anyway and having learned a bunch of new things. I later learned the interviewer had recommended to hire me based on how our conversation had gone, though I didn’t end up taking the job for unrelated reasons having to do with relocation.
I’m unsure what actionable advice this can provide, except maybe:
When I’ve set up interview processes, one of the aims is that it should be a good experience for the person being interviewed even if ultimately you decide not to hire, or they decide not to work with you.
For one thing, it’s just nicer, but it also means that they leave with a good impression of your company, and may recommend you to others or join in future years.
It’s not trivial to do that, but it sounds like that company did a great job of it.
Reminds me of an engineering VP that interviewed me this year that opened it up with a simple “you are logged into an unknown system over SSH” hypothetical situation and then asked me how I’d discover certain parts of the environment: network addresses, running processes, etc. 30 minutes later we were geeking out on Linux linkers and code disassembly. The guy was very knowledgeable, curious about learning new things (of course, I shilled NixOS to him) and super methodical - he led the interview all the time and did not move his eyes off the goal - figuring out if I was a good fit for their team and product.
One of the best technical interviews I’ve had so far. Too bad they followed it with a low offer.
I was pretty satisfied with the (technical) format used at a former employer, though it was quite specialised (as was the job). I didn’t design it, merely tuned/maintained and held the interviews for a number of years (something around 1000+ candidates for the remote stage, and at least 100+ on-site).
Context: The ‘job’ was debugging other people’s code for the bugs they couldn’t solve within reasonable time/cost-frame, and bugs that nobody could be blamed for. No matter if this was in a compiler backend, image renderer or file system corruption recovery.
There were always a large number of applicants, and the cost for on-site interviews were considerable (compensating the candidate, logistics, politics, my time away from projects…) - but not as high as the consequences for hiring a poor fit. For this reason, there was a remote prescreening. Headhunter/consultancy “sales pitch”, CVs and so on were almost entirely ignored, and rarely did I ever look at one; didn’t want the bias, didn’t want the “intelligence” on a possible future coworker.
Remote part: The candidate got a generated/personalised PDF with challenges picked at random from a few pools. These were typically some optimised disassembly output of a known small algorithm or function, “just” name the algorithm/idiom/function. Others were C/C++ gotchas, or some former sanitised bad commit that needed a code review (or a bad code review itself) and so on. The criteria was that they should be hard to google/stack overflow/ask-a-friend, and still be indicative of the real test/work, and should take less than an hour to solve (a reply was expected within 24 hours). Candidates should be able to use this to self-assess, get as close a feel for the real tasks as possible, as to be able to opt out without losing face, suffer the stress of higher stakes rejection and so on.
On site: This was a full day with the basics covered (flight, hotel, meals, … etc). The candidate got written instructions grouped into three parts: terminology, core and “oh fsck”. She got to spend as much time as she liked to prepare in a quiet room with beverages etc. I didn’t care to see any written notes, just signal when ready and we’ll have a discussion. There were periodic checkups at about 30 min intervals to answer questions (also probe that the candidate wasn’t stressed out or fixating on something irrelevant) and so on.
In the discussion stage there was often someone from a different team involved sitting in, sometimes to assist in “translation”, sometimes for the “can I work with this person?”, and sometimes for my own safety (there were … incidents). It was more role-play than just “ok on question one the answer is…xyZzzzz”.
The terminology part served to figure out that we roughly had the same language for things; cultural differences could be quite pronounced in this regard, and it helped me to interpret the other parts. The intermediate “core” were sanitised and compacted/distilled versions of significant, non-textbook, bugs we had squashed in the past (~5 in total) each one at about a printed page or two. A successful candidate would solve maybe 2-3. Nobody managed all of it, which was something of a point.
The “oh fsck” was an example of the shallow end of more “systemic” problems, there were code snippets, partial crash disassembly and a bit of a story to it. Hints with points deduction in a sealed envelope and so on. The few with the intuition for that one was a near guaranteed fast-track.
Afterwards there was an (optional) debriefing where I explained the problems, some ways of reasoning that would work, and acceptable solutions. Alas it was also quite taxing, 2-3 candidates on-site at a time at the most and still mentally and socially fried at the end of the day.
In my experience, most of the “make sure they can actually code” tests can be easily replaced with simply asking the person to look at some code you’ve prepared and discuss it with you. Additionally just discuss the state of various things they say they know, and what they think of trends in that area, most importantly why they think that. You don’t want to screen for people who think the same shit you do, just that they understand the thing enough to have an opinion they can articulate one way or the other.
Best interview I ever had I failed. It was a pair coding session as a driver where I had to debug a UI error all the way down from the JavaScript calls to the database in the backend.
This really felt like it was a good representation of what I would have been doing as a general day to day.
Why’d you fail? Did you not have the skills you would have needed? I guess I’m really asking if you think the interviewer got a good assessment of you.
I had an interview like that once and I thought it was awful. The interviewer couldn’t decide whether he was pairing with me or examining me, and when I would ask questions instead of getting his feedback I’d get “well, what do you think?” or “how would you figure that out?” I mean…I’d ask a knowledgable coworker.
I think it needs a lot of training for the interviewer.
Well I am more of a backend dev, so I didn’t really take to using the web console quickly enough.
The place I remember that came really close to a perfect interview, we discussed the infrastructure of the company, what it is made of, how to improve it, what my experiences are regarding that. It came really close to what would be a kick-off meeting on a project. For various reasons on both ends I did not end up in that position, but that has both no school like tests and “things I’d actually do”, it also had essentially no waste of time (because it was kind of a kick off meeting). On top of that I spoke with both the manager and potential co-workers, all in a very face to face way (despite it being held remotely).
I also have been on the other side, interviewing people where I tried to replicate that. Sadly there was some out-sourced pre-screening, that essentially brought forth candidates, that expected to have standard questions asked and standard questions answered, school like. Also and here again I’d blame the external pre-screening we ended up with candidates, that were in the right realm, but their and our expectations about the working relationship didn’t really match, and it became really clear to both sides rapidly through these interviews.
In general - for whatever reason - the majority of good interviews were remote, instead of on-site (at least initially). It could be a fluke, but I like to imagine that it has something to do with respect of the other person’s time.
I’ve also heard about total opposites though, interviews were both sides were excited about working together and eventually there was some formality or strict company structure being a problem, ranging from “You have worked a decade, but we require a degree or you have to take a junior position” (after being offered a lead position during the technical interview) to “You can work completely remotely all the time, but you have to move to the city of the office” (with no other reason than company policy - nothing legal, etc.).
I think the most important way of getting good people is based on respect. Don’t waste people’s time, be open about your company (both good and bad parts), respect their time and don’t put them in uncomfortable situations. If you do that then what should make one think it would be different, during actual work. Treat them like they’d already be part of the company. I think this would give the best picture and show to both sides, whether there’s a match or not.
There’s nothing wrong with figuring out that it doesn’t make sense.
Also that way you have a bigger chance of finding people through referral. If a potential employee takes another job or simply isn’t a fit, they might send someone else your way, but if you coldly cut ties that’s never going to happen. And it does happen. So don’t just coldly cut ties or turn off respect, just because there is no match.
I’ve been using variations on The Monopoly Interview for about 15 years now, ever since reading about it in Reg Braithwaite’s blog. The biggest thing I’ve found in making this successful is to clearly establish up-front “This isn’t a test. This is a chance for you and I to have a conversation - I need to see how you think about a problem, and see we’re a good fit as teammates.”
A couple of changes I’ve made to the process laid out by Mr. Braithwaite:
I make it a collaborative exercise - “Our boss/client/stakeholder has requested we deliver on online, multiplayer version of Monopoly. You and I are going to spend the next 20 minutes mapping out our initial response to the request.” And then you ACTUALLY work WITH them to build out an answer.
I have filled multiple upper- and mid-management roles in the past 4 employers, so I frequently have to adapt the question to the role being interviewed for, as I am frequently interviewing in different disciplines. For example, for an SRE-type role, I often say “The client has built an online, multiplayer version of Monopoly and has engaged us to do the delivery of the application. Any day now, the client expects their good friend Justin Bieber to tweet out an endorsement; what do we need to know/do to prepare the application for this kind of flood of activity.” The question is amazingly flexible and so far I have been able to trivially adapt it to every role I’ve ever run an interview for - front-end engineering, back-end engineering, SRE, DevOps, QA, UI/UX/design, content marketing, management/mentoring, etc.
Be prepared with other board games for people who are not familiar with Monopoly; I’ve found that at least being familiar with the rules to Chess, Boggle, and Ludo (at least, the variant popularized in the US as the children’s game “Sorry”) to be enough to cover every interview I’ve ever run since starting to use Monopoly.
Not only have I personally found great success in the hires who have been through this interview process, everyone who has ever been through this interview process with me has reported finding it one of the best interviews they’ve ever had, and I know of at least a 14 companies who now use it as their base interview process, either because I introduced it at that company, or previous hires of mine who have moved on to other companies took it with them.
The best interview I had was also at my current workplace.
All in all, I really liked how it was done. There’s only a few minor things I would have changed, but we are constantly improving and refining the process to try and remove bias and make it better for all parties involved.
Small take home task combined with open-ended questions based on how the candidate designed and implemented the solution is, in my opinion, the best interview format.
It is an opportunity for both the interviewer and interviewee to learn and discuss pro and cons of a given design. If done right, both parties come out feeling that the time spent was valuable, even if it does not lead to an offer.
My favourite is similar - small job-relevant take home task, followed by in person refactoring of that task (‘the requirements changed’), then some open ended questions. It actually doesn’t take all that long compared to what some companies do, but I think it gets you to the knowledge you need quickly.
You have to sell people on the role though or they tend not to be prepared to do a take home task.
The one I’d objectively call the best was last year for a local startup (interview was still remote). It was a full day that would’ve normally taken place at the site and it was after a round of video chat iirc.
The task was a typical “read this requirements doc, discuss the architecture and solution with the team, and then implement a small part of the functionality and stub some of the rest”, I guess you could call it a microservice that was straight inside the domain of what you’d work on and using one of their apis (as a mock though) but contained enough to be easily grasped. They gave me 15 minutes to think about it, then it was a discussion with 2 of their engineers, bouncing off my ideas, providing 2-3 alternative paths where it would make sense (they said which one they preferred), including some light db schema modeling, API design, and so on. In the end I had another 20 or so minutes to prepare a short slide deck and give a 10 minute presentation to around 6-7 people from the company, detailing what we did that day and how the solution looked. This leads back to how it was a complete tie-in to their actual stuff running in production so the ones who weren’t involved before actually had a clue what I was talking about. Overall pretty good day and experience, sadly it didn’t work out although I’d gotten an offer.
The very best interview was where I made the appointment in a community IRC channel, sent a sample of “anything I’d coded and were comfortable to be judged”, showed up at the office, was offered a beer, had an hour long chat, and started my student job the next week (I’d stay with the company for nearly 7 years, 3 of them full time).
Thanks OP for posting, I’m interviewing people at work these weeks and this is a great way of getting insight into what people expect, like, feel uncomfortable with…
Personally I’ve only had/given around 20 interviews, and I don’t remember anyone in particular, so I guess I haven’t had a “wow” one yet.