1. 20
  1.  

  2. 25

    I think the moral here is “you don’t need to do coding tests if they have a good portfolio”, but I have also seen similar posts on this very site complaining about companies that judge you based on your portfolio because not everyone does side projects or work they can share.

    Maybe we need to consider that there are multiple classes of applicants that need different interview styles; in particular, the divide here seems to be between “day job” programmers who have no publicly visible work and “enthusiast” programmers who do. It would probably make sense to use a different process for each. (Although I would worry a bit about people who hire software ghost writers. If you think that’s far fetched, you haven’t done interviews.)

    1. 11

      It’s more complicated than that, even, because I have a portfolio, but it’s almost completely orthogonal to all my professional experience (the biggest single element of the former being a 3D engine in Rust, and the latter being backend/data processing software mostly in Java). I’m sure it still helps, but it probably shouldn’t—my ability to microoptimize linear algebra and my ability to wrangle large frameworks and macrooptimize data flow don’t really have much to do with each other.

      Ultimately, I think it’s the hiring manager’s job to tailor the process to the candidate. If they have a relevant portfolio, great, look at that. If they have an irrelevant portfolio, that can probably at least tell you whether they’re completely incompetent. If they’ve currently got a job, they’re less likely to put up with an extensive coding test. Etc. There probably are no hard-and-fast rules.

      1. [Comment removed by author]

        1. 1

          It really depends. For somebody who isn’t currently working a full-time job and has plenty of spare time but not a lot of experience, a take-home project is a great opportunity to showcase their skills in more depth than a mere phone screen ever will. Moreover, a properly designed take-home can actually be fun even for somebody who does have a full-time job writing software.

        2. 1

          I would worry a bit about people who hire software ghost writers. If you think that’s far fetched, you haven’t done interviews.

          You can fire someone for lying on their résumé. Not the ideal outcome obviously, but not a permanent dent to the company.

          1. 1

            If you use a take-home to screen people then it is a very good idea to follow it up with a 30 min discussion about the trade-offs made in the code and potential extensions to it, which will be extremely difficult to answer for somebody who did not actually write the code themselves.

        3. 12

          I disagree with this post. I’m also a professional and my time is valuable too. However, two of the three suggestions they made are taking significant time away from me and my team for evaluating a candidate that potentially can’t even write code to solve a simple task. Part of an interview process is to filter out people before it gets to that point, so we’re not wasting employees time.

          1. 10

            I think coding challenges are optimised for candidates who are looking for a job. I’ve been in that boat once, and when you’re actually looking for a job your “valuable time” is of course bet spent trying to get said job (by doing coding challenge or whatever else).

            Most of the time, though, I’m being recruited. I’m not going to do a coding challenge for a recruiter.

            1. 1

              Taking an entire day to work with them (unpaid) still strikes me as really weird.

              1. 1

                Think of it as a great way to find out if these are people you would want to work with every day before you actually have to do that.

            2. 8

              I disagree with this post. I’m also a professional and my time is valuable too.

              I have the same problem with it as a hiring manager– how do I screen out socially capable but inept developers– and I share the author’s opinion when I’m the candidate– this tells me nothing about why I want to work for you. Each side wants the other to make an unequal commitment so it amounts to a single round game with that candidate. As a candidate with a choice of games, I don’t want to play this one and it signals disregard for/trivialization of the candidate’s investment and work history. For the hiring side, this plays out multiple times and there is investment in posting, screening, reviewing, etc. so regardless of this round my total investment is higher but not visible.

              So what have I personally done? When I’m the candidate, I refuse to do the coding challenge and say, like the author, check my repos and talk to my references (unless the problem catches my interest, then I might). I have that luxury. When I’m the employer? Up front I explain how it works and what timeline they can expect as part of a 15-minute phone screen for basic info with someone technical. Then I arrange a round of 45-60 minute calls: a technical call with someone on the team and a social/technical call where I try to get them to talk with me about their work in detail and many of the non-code aspects of their job, habits, tools, designs, etc. They’ll probably have a call with my manager or another peer. Then, if I’m satisfied but not sure, I bring them in or have a video chat and they do a quick coding test. This wastes less of their time, makes my commitment visible, and seems to work but it is not a scalable process.

              1. 7

                I have a portfolio and some github projects. This is where most of my hiring emails come from. So when a company doesn’t spend the time to check that out, and they want me to implement some trivial thing that doesn’t generate value for them, I don’t have time for them either.

                I’ve had companies pay me to be a consultant for a week before giving me an offer, which was a nice way to learn about who they are. On the other hand, sometimes companies give me job offers now before I know anything about them, and I have to pump the brakes and talk to more of them before I feel comfortable going into something long-term.

                1. [Comment removed by author]

                  1. 6

                    They didn’t ask for the hiring managers time, they asked for developers time. Either way, these people have other job responsibilities too and their time is important to defend.

                  2. 1

                    …evaluating a candidate that potentially can’t even write code to solve a simple task.

                    In the post, they talk about how they have a blog, numerous GitHub repositories, etc. At that point it should be obvious they can code. The interview then should be more about “fit” or whatever, IMHO.

                    1. 5

                      They aren’t the only candidate we would interview and in my opinion, it is better to have a consistent process. If every candidate had a similar level of public presence to evaluate then maybe that would be different.

                      1. 7

                        So, again IMHO, at that point you’re basically throwing out someone with passion and talent due to bureaucracy. If I come to you with decades of experience/conference talks/published papers/lots of open source software to review/whatever…and you ask me to spend 30 minutes doing trivial work, you’re basically implying that I’m lying to you and/or that your company cares more about process than people.

                        Again, this is IMHO.

                        1. 7

                          I’m saying that you’re not the only person applying for the job and I need to treat everyone the same, so we’re not giving preferential treatment.

                          1. 3

                            I know, but…maybe you should give preferential treatment to people who are obviously better candidates. :)

                            1. 9

                              Some of the best engineers I know have zero public presence. Some of them are extremely humble and don’t like public flashiness. Some of them have families and maintain a strong work-life balance with non-tech stuff. Never assume those with a strong public presence are giving the world the whole picture. You still want to drill into the parts of their personality that they don’t highlight.

                              1. 4

                                Why does having a public portfolio make someone an obviously better candidate? What makes a candidate obviously better? Arbitrary social metrics? Ability to speak quickly about technical topics? Ability to talk bullshit without it sounding like bullshit?

                                How do you know a candidate is obviously better without having them go through the same process and pipeline?

                                1. 3

                                  How do you know a candidate is obviously better without having them go through the same process and pipeline?

                                  If the code in their GitHub account is as good or better than what would be tested by my coding test, why subject them to that? Ask harder questions, ask questions about the things that the coding test wouldn’t cover (including “soft” things that would judge a good fit), etc.

                                  Why does having a public portfolio make someone an obviously better candidate?

                                  Which surgeon would you rather have? The one nobody’s ever heard of, or the one who has published articles on the relevant surgical procedures, who goes to conferences to learn more about surgery, who obviously is passionate enough about medicine that they would study it even if they weren’t getting paid?

                            2. 8

                              There are, unfortunately, a lot of liars out there. I won’t say that industry hiring practices are anywhere near ideal, but as an interviewer it was astonishing how many people with great optics were incapable of coding. Someone would speak with great passion about all their projects and yada yada, and id be fairly convinced they could do the job, then I’d ask the most basic question imaginable. Splat.

                              I guess it helps if you choose to believe the test isn’t meant for you, but for all the other pretenders.

                              1. 7

                                Even more surprising to me is that people who can’t actually code are somehow able to sustain careers as developers. It started making a lot of sense to me why good developers are in such high demand after I had the opportunity to do some interviewing and found that a frustratingly large amount of applicants can’t actually code, even if they look great on paper and are currently employed as developers.

                                I think it’s incredibly risky to hire a developer without seeing code that they have written, be it from open source contributions or a coding test if necessary.

                                1. 3

                                  Onsite nerves can kick in. It sure as hell did for me. I hate white boarding and I lockup. Totally brain freeze. That said, if it’s a basic one like writing a loop…well, they somehow lied their way into the onsite. Thing is, a take home coding challenge can weed out those people pretty fast. If they do that and come in and fall flat on their face before the whiteboard I don’t totally discount them. Anyway, there’s no perfect solution. There is always the potential to hire someone that is great at coding interviews and then sucks at real world problems.

                                  1. 2

                                    This is exactly my company’s experience. Half of the candidates would just bomb out on the C++ test even though they have studied it at high school/college/university and worked with it at their job for 5-10 years. How?!? Because they were either doing Java, not C++, or they were actually managing a team that did C++ and never had to touch it themselves (Well since leaving school at least).

                                    1. 1

                                      What I don’t understand is why this is so hyper-specific to developers. You never hear UI designers talking about processes like this.

                                      1. 6

                                        Really? I’ve heard UI designers talk about it a lot.

                            3. 10

                              I’d like to offer a counter point - I prefer to spend 2 hours doing an interesting coding test than 30 min on the line with a recruiter who’s testing my “soft skills” (code word for all kinds of bias).

                              The interview, as always, is a two way street. The kind of problems you pose me and I pose you tell us about each other. For a person solving problems on a computer there are two exercises that I like to sit on either side of the table for, and they form part of a “colleague simulator”: Take a current, real problem. Take a part, a fun, challenging part. Do a conversational session where both parties work together to find a solution. Then let the person have their own space and code up a solution.

                              An ETL test is not going to hold my attention, but if you give me a small piece of a problem you are working on now and ask me to brainstorm a solution with you and then code it up, if I were looking, I’d do that because it tells me who you are and what you do everyday and if I’d like doing that.

                              (OK. This is the ideal world solution. In the past we’ve flipped the order - looked at the coding test and then discussed the code + extensions with promising candidates. That works too. If a candidate can’t be bothered to do the test, either we are not interesting enough, or they have an attitude. Either way, unlikely to be a good fit)

                              1. 8

                                Wowee do I ever disagree with this post. As someone who does a lot of interviews, I feel like an in person short coding exam is absolutely a critical part of evaluating a candidate.

                                I would assert that the author should check themselves. Their situation is not everyone else’s, and if most companies only ever hired people who kept blogs and had well fleshed out Github repositories, they wouldn’t be able to hire in any kind of reasonable time frame. Are these things pluses when evaluating a candidate? Sure - do they REPLACE a critical part of the evaluation process? Absolutely not.

                                1. 7

                                  My main gripe with coding tests is that they ask me for an investment of my time and resources so that they can gather information about me, but I’m getting nothing back.

                                  Sorry, but that’s incorrect - the return value is a job offer.

                                  I personally don’t mind coding tests, and think they’re a reasonable request. I’d opt for a coding test over whiteboard coding or goofy puzzles any day of the week. I also don’t think it’s fair to expect an interview team to dig through everybody’s GitHub and Bitbucket profiles.

                                  The author’s proposed alternatives (pair programming with the candidate, etc.) are a huge time waste for both the interviewers and the interviewee. If a coding assignment is too much work for the author, think of how much work it is to get up to speed with an unfamiliar code base well enough to contribute code while pair programming. And it’s completely ridiculous to expect an interview team to go through that hundreds of times while hiring.

                                  As an interesting data point, the company I work for used to send out a simple coding test, but after we were bought by a much larger company, their HR department made us stop. I believe their concern was that the criteria used to judge the assignments were too subjective. Some criteria (performance, scaling) could have objective measurements, but many (coding style, overall design, etc.) are inherently subjective.

                                  1. 3

                                    You don’t offer people a job based solely off a code test, so the return value is the opportunity to spend some time interviewing (not, as you claim, a job offer).

                                    I have no idea whether I want the job offer until I’ve spent some time with the people; a code test is time spent to earn the opportunity to find out if I’m interested.

                                    I would do that if I had to, but I don’t have to in this labor market.

                                    1. 1

                                      I have no idea whether I want the job offer until I’ve spent some time with the people; a code test is time spent to earn the opportunity to find out if I’m interested.

                                      And a coding test is a company’s way of seeing if they’re interested before they waste a bunch of time interviewing somebody who can’t write fizz-buzz or whatever.

                                      I also don’t think coding tests are as asymmetric as people make them out to be. There’s useful information to be gleaned about a company by the coding test they give, how it’s carried out, and how it’s judged.

                                      I do agree that some companies go overboard and have unreasonable coding tests that take too long or require too much effort on the interviewee’s part, but that’s a problem with their process and not necessarily coding tests in general.

                                      It really boils down to personal preference. I don’t bother with companies that make candidates interview more than 3 times, but I’m not going to write an article claiming nobody else should either.

                                      1. 1

                                        And a coding test is a company’s way of seeing if they’re interested before they waste a bunch of time interviewing somebody who can’t write fizz-buzz or whatever.

                                        Absolutely. The company would rather waste my time than theirs. I’m uninterested in that arrangement (they are welcome to either pay me for my time, or match the effort I put in).

                                        1. 1

                                          The problem I have with your argument is that reviewing a coding test isn’t zero effort on the employer’s part, and it’s usually even more work than just plain interviewing.

                                          Usually the applicant’s code gets sent to a group of 2-4 developers who each read through it, compile and run it to see if it meets the requirements, and then come up with some questions to ask about during the interview.

                                          IMO, it’s the best chance for a candidate to show off to the team they’ll work with that they actually know what they’re doing. The hiring people see dozens of solutions to the same problem, so creating a particularly elegant or high scoring solution is a chance to show that you’re particularly good at the thing they’re hiring you for.

                                          1. 1

                                            That’s a pretty good counter-argument, but from the outside I have no visibility of whether the employer is spending 5 minutes or 5 hours on the test (and I have seen both). Sometimes your test gets dropped, or reviewed 4 months later. Ordering matters - I’m spending my time before they spend theirs, and not every employer responds in a timely or thorough way.

                                  2. 6

                                    I remember when I first read this article and I kind of just blindly believed it. Also I hadn’t done many of these tests back then so I’ve kind of grown a bit and changed my views. I’ve done coding tests, and been on the other side and conducted them. I have no strong feelings about them what so ever anymore.

                                    Recently at my current place of employment, we decided to try something a little different than a coding test/exam/whatever. We devised a few arbitrary word problems and the candidate works with me, personally, to come up with a solution. There is no set solution. This gives us (especially me) an immediate sense of how well this candidate will work with our team. We both stand at the whiteboard, and I let them talk and diagram away and interject with my own ideas and see how they respond to them. We assess their ability to solve a problem and work as a team member. This is much more valuable to our small team than if they have memorized a bubble sort algorithm.

                                    1. 1

                                      This is an interesting idea. Would you be willing to share a couple example problems/solutions?

                                      1. 5

                                        Sure! Some preface about the company: we are an e-commerce retailer of a product we manufacture in house.

                                        “We would like to develop a system that allows customers to share their purchases on their social media (restricted to Facebook and Twitter for time constraints). Along with that functionality, we would like to keep track of which products in the purchase were shared, and which platform it was shared on.”

                                        Its a rather high-level question that doesn’t require a lot of in-depth discussion about platform-specific problems (versions of PHP for example) or intimate knowledge of our existing platform (Magento on a LEMP stack).

                                        1. 1

                                          Thank you!

                                    2. 5

                                      I prefer to solve these tests that answering boring questions about const-correctness or operator precedence, even if it may take several days to solve. Currently I’m finishing such assignment (tool loosely connected to their field) where company I’m applying to expects to deliver code + documentation + tests. Code @github may not necessarily reflects quality of your code, especially when you work for a company that does not show their code or take part in open-source movement.

                                      1. [Comment removed by author]

                                        1. 6

                                          The question is whether I want to work for an organization that is so lazy in their hiring practices that they’re handing out FizzBuzz to non entry level hires.

                                          Even as it seems like an insult to the intelligence, you’d evidently be shocked at how many complete frauds have glow-in-the-dark CVs, and the horrifying fact that FizzBuzz-level tests are actually a useful tool, and quite a quick one.

                                          Next time your company has a hiring round, I urge you to sit in. You’ll see why the poor person doing the hiring does this.

                                          (I’m a sysadmin not a coder, but we have the same thing in sysadmin. Twenty-year CVs where they clearly don’t know basics.)

                                          1. 3

                                            But are they, though?

                                            As a personal anecdote, I have to be in the right “mindset” for writing code, which is a very different mindset from “interviewing”, so much so that they collide. When I’m in “social mode”, talking about what I do, following social cues, etc. I absolutely cannot write code to save my life. When I’m in ‘coding mode’ I can write code for just about anything you can imagine but my social skills amount to grunts and forces smiles.

                                            Having been on both sides of the fence, I think there’s really a perception that there are frauds behind every CV and it’s up to YOU as the hiring manager to root them out. I believe that this is an adversarial position before the interaction even begins. I’ve found much more success in assuming people are capable and letting them rise or fall from that starting point.

                                            Further, this modality also doesn’t account for the fact that “interviewing” is a skill that is different than “coding”.. Solving large complex problems in relatively unbounded time (weeks, months) vs. on the spot 45 minute brain teasers/puzzles are two entirely different skillsets and don’t signal anything about the prospective person other than they’re practiced and your interview style.

                                            1. 3

                                              Solving large complex problems in relatively unbounded time (weeks, months) vs. on the spot 45 minute brain teasers/puzzles are two entirely different skillsets and don’t signal anything about the prospective person other than they’re practiced and your interview style.

                                              I wonder if this is analogous to doing arithmetic in one’s head versus doing mathematics.

                                              It’s pretty easy to practice arithmetic and get quick & good at it, just by mechanical drilling. That’s one way you get to be known as the guy who’s “good at maths” at school. And then you get lazy about it because you have a calculator, and there’s something more interesting to do than summing & multiplying numbers.

                                              Once on a coffee break, a coworker asked me something silly like how much is 1.5 * 0.5. It was sudden, unexpected, and shocking, and I just sort of froze. It probably took me at least five minutes to come up with the answer.

                                              Were that to be an interview question, I suppose one could extrapolate and conclude that I am hopelessly bad at mathematics. (To be honest I’m not a maths geek but I’m not that bad either, ha!)

                                              1. 2

                                                I’m talking specifically about 5-minute ones on the level of FizzBuzz, not 45-minute things. My cynicism comes from sad experience. Perhaps it’s the same 199 people, per Joel: https://www.joelonsoftware.com/2005/01/27/news-58/

                                                FizzBuzz isn’t a coding test, it’s a quick bozo filter.

                                              2. 1

                                                Next time your company has a hiring round, I urge you to sit in. You’ll see why the poor person doing the hiring does this.

                                                You know, I’ve been in a bunch of interviews at several companies, and I’ve never seen one of these frauds who are supposedly so ubiquitous. Maybe it’s because I haven’t worked at big or “name” companies. But I just haven’t seen people who couldn’t write FizzBuzz applying for developer jobs.

                                            2. 3

                                              I think coding tests can be useful when it’s a junior role and most of the candidates are recent graduates or have little commercial experience. It gives the candidate a chance to distinguish themselves and can make the subsequent interview less intimidating as they’ll be on familiar ground - discussing how they attacked the problem, why they chose one solution over another etc. Can be revealing.

                                              Though I agree for more senior hires, it can be almost insulting to ask someone who feels like their resume speaks for itself to spend 2-3 hours of their spare time proving they’re up to snuff. Probably for more senior positions, a detailed technical phone interview might be the way to go.

                                              1. 3

                                                Anecdotally, I interview great at companies that ask for take-home exercises, and therefore have gravitated towards companies that ask for (reasonable) take-home assignments. Hopefully, I haven’t been a false positive!

                                                Pair program with people from your team for an hour or two (Screenhero works great), so the candidate can learn from them as they learn from him/her.

                                                This sounds like hell. Pairing is a skill that takes a long time to develop, and expecting someone to just hop in and do it for an hour or two will only produce disastrous results.

                                                1. 3

                                                  Asking people to do a multi-hour take-home quiz is kind of shitty regardless of their experience level. I think it’s better to bring someone in and ask them to work through a few problems of increasing difficultly. That way both the candidate and the interviewer’s time are equally “wasted”. As for senior people being insulted by being asked simple question… Check your ego. There are lots of terrible developers out there. Do you really want those people getting through the interview process at the company you’re applying to?

                                                  1. 1

                                                    So, my key objection to take-home tests is that the employer is asking me to invest my time without investing theirs.

                                                    I don’t mind in-person tests, because the employer is matching my commitment - which shows they are willing to treat me as an equal.

                                                    1. 1

                                                      I’m torn. I do resent being asked to do significant amounts of work for free. But it’s far nicer for me to be able to solve a problem at my own pace, in my own home, with the tools I’m used to, than to try to do something on a whiteboard or with an unfamiliar dev environment during an on-site interview. (I did one awful remote interview where I had to write some functions that should have been trivial using a weird web-based multi-user editor I’d never heard of or seen before or since.)

                                                      One issue is that, in my experience, hiring managers often drastically underestimate how long their coding challenge is going to take. Maybe I’m just slow (though that would be contrary to the feedback I’ve gotten in all of my jobs), or too finicky (I’m not going to skimp on test coverage or documentation unless they are very clear up front that they don’t care about those), but several times now I’ve been asked to build little toy applications that “should take you a couple hours” that end up eating my entire weekend. That’s a really excessive amount of work to ask for from someone with years of documented experience in the field and a bunch of public Github repositories–especially before the real interview.