The startup I work at has one of the best interview processes I’ve personally experienced. It went something like this. Hopefully this is helpful to you, I would think this style could be approached for any level or position.

24 hours before the interview, they sent me a fairly detailed brief of the project. They asked me to design, from start to finish, a service that would check the up status of a resource. Simple enough task.

The doc had a decent amount of information about the spec but i was given time to ask questions before starting the code interview.

Then, they just paired with me for a few hours while I tackled this problem like i normally would. I could look at docs, ask questions about what they wanted, read tutorials even. It was completely open ended.

There was no expectation of any specific form, they were mostly looking for how I approached the problem and solved it.

I normally use TDD so I started with a test harness, and they would ask things like why I chose that library, or why I wrote an interface the way I did. I was allowed to choose any language I wanted for the interview, and I chose Go.

By the end of the interview, I felt really confident that I had sufficiently displayed my abilities. I think that’s what people really get annoyed about with code interviews. Most of the time they’re not actually mad about not getting the job, they’re frustrated with feeling like they didn’t get the chance to adequately show their skills.

Hopefully this helps! I personally thought it was the best interview process I had endured.

This set of questions doesn’t cover everything, and it may cover things you explicitly don’t care about, but I think it does a great job of illustrating a process that can allow you to put together a set of questions that are useful, fair and have robust grading rubrics: https://jacobian.org/series/unpacking-interview-questions/. (I also think a couple of the questions are really good, and plan to steal them next time I’m interviewing.)

On the more technical end, I found this rant a useful reminder that there is no perfect interview process: https://software.rajivprab.com/2019/07/27/hiring-is-broken-and-yours-is-too/. Which is not to say you shouldn’t do the best job you can, or that they are all equally bad, but any process has limitations and weaknesses you should be aware of.

At my previous company we also had to build a lot of integrations and we also decided that’d make a good framework for interviewing.

Candidates got to choose between a technical phone screen with a live coding exercise using CoderPad and a take-home assignment with a 4-hour time limit. The live coding exercise was a more typical interview process, though we avoided dumb “regurgitate this algorithm” questions. To my surprise, ~95% of candidates chose the take-home problem.

For the take-home assignment, we repurposed some code from our test environment that emulated a typical HTTP-based remote service. We packaged it up as a public Docker image and wrote documentation for its API. The assignment was to write a client that would read instructions from an input file (provided by us), make the corresponding API requests, and write the results to an output file in a particular format. No restrictions on language or libraries or anything. The API was not massively complex but also not trivial; it required an authentication step that gave back a time-limited session token, so candidates had to decide how to deal with that.

To make it a more realistic exercise, we did one other thing: our API documentation was intentionally missing a few minor details. We were up front about it and told candidates to expect it. That wasn’t an attempt to be obnoxious, but rather a reflection of the sad fact that basically none of the real services we integrated with had accurate or complete documentation, so we both wanted to give people a taste of the real job and also evaluate candidates’ ability to figure things out in the absence of comprehensive docs.

When a candidate submitted a solution, we checked it into a git repo and turned it into a pull request that was then reviewed by at least two of our engineers. If we thought it showed promise, we scheduled an in-person interview round, where one of the sessions was the reviewing engineers going over the code with the candidate and asking questions. (Aside: It cracks me up when people complain that companies use take-home problems instead of phone screens so they don’t have to spend any time evaluating candidates. Reviewing those candidate PRs almost always took me longer than a phone screen would have, and took the time of two engineers instead of the one engineer required for a phone screen.)

The exercise worked really well! Candidates all gave us positive feedback on the experience (“hard but reasonable” was a typical comment we got back) and it predicted job performance much better than I would have expected.

For example, we were on the fence about one candidate because they’d gone down a rabbit hole trying to figure out a super elegant solution to a relatively minor aspect of parsing the API responses, and thus had failed to get a usable client finished within the time limit. But they were clearly very smart and the code they did write was clean, so we took a chance and made an offer… and six months later, they were on a PIP because they were constantly spending time trying to write the world’s most pristine elegant implementations of unimportant parts of the code and failing to finish the actual tasks they were supposed to be working on.

Well, good is relative and dependent on what your organization / business is looking for in a hire, and how you measure their success, etc. When I interview the questions I am asked tell me a lot about what they think my role is, what the “upper bound” for my role is, and so on (I recognize it’s not perfect, but it gives me some sense).

I would want to what kind of things you’re looking for in a hire – you’ve hinted at some technical details (i.e. building API integrations), but are there other qualities that matter? Interpersonal skills, diversity and inclusion, etc.?

Companies with good interview practices IMO start from there and work backwards to get the questions / process they need to achieve their target. The added benefit of this is that as an interviewee I have a really good sense of what they’re looking for, and I can more confidently decide if it’ll actually live up to my expectations.

The best ones I’ve seen do a good job replicating my real work experience (i.e. pair programming, working through a problem together that I might face on the job, etc. similar to what the other commenter echoed). The benefit of doing these live is that you can learn not just how I solve a problem but also the more interpersonal qualities that are sort of baked into that process.

To give you some anecdotes from the interviewee side…:

The current startup I work for basically had no interview… it was just “tell me about yourself” and then I got an offer after some more chit-chat, which was mildly terrifying. But I negotiated a 1-2 week contract phase to see if I’d like it, which sold me on it and I joined (+ they were happy with the work I did in that period).

The startup I was at before had the following process:

• HR screen
• a take home coding challenge which took me about 1-2 hours to complete, and their Senior Engineers reviewed it
• Behavior screen w/ senior eng to review my submission
• On site interview with 4 back to back (1 algo, 1 systems, 1 behavioral, 1 product)

It was pretty time-consuming and involved for applicants, which had the unfortunate effect of filtering out people who couldn’t dedicate so much time for a job application. I had a good sense of what I’d be doing, although it did seem to me that doing all of the on-site interviews on top of the take-home was a little redundant.

And another startup I interviewed at had the following:

• Screen with CTO (technical / behavioral)
• Remote algo interview with architect that was basically 4-5 algo / puzzle questions (head hurt like crazy after)
• Got the offer, but negotiated 2 week contract phase and didn’t really fall in love with their technical system, etc. so I passed.

This interview told me that they value brain teasers and creative algo problems, but it was just another startup with a similar ish “build a crud app”, or so I thought. The actual engineering implementation felt so unnecessarily “creative” (convoluted); in a sense it reflected what I felt during the interview.

I’m glad to see this discussion! I spend a lot of time interviewing candidates these days, and our process surely has room for improvement. Usually the things that matter most for my team are:

1. Gathering requirements, constructing a model from it, and verifying both (the former w/questions to interviewer, the latter w/estimation)
2. Writing code that you’d be comfortable sharing with a team (is it documented? tested? appropriately idiomatic / not?)

I also tend to agree with something a colleague shared earlier this week, https://auth0.com/blog/how-we-hire-engineers/ , that take-homes can be very, very useful. It’s just been really hard to ask anyone to spend time on one in the pandemic era – work/life separation is hard.

But…all this said, @pushcx : this seems to also be about business practices! Can you please clarify if this thread belongs on lobste.rs? I know in the past you’ve tended to delete such things / rename the story asking about the policy for it, etc.