1. 37
  1.  

  2. 13

    The more I read pieces like this the more am convinced that no single “right” interview process exists. Always a perilous balance between respecting the candidate and watching for your company interests.

    1. 2

      YES! Exactly!

      i’m reading a lot of the responses here and thinking “How would this scale?” and I suspect the answer is it can’t, and that a lot of the much more time intensive techniques that people favor would work much better in start-ups and small companies.

      I also suspect from the tone of a lot of responses that a lot of people have been through a bunch of frustration at the hand of interviewers who weren’t actually very attuned to respecting them, their time, or their self confidence.

      1. 7

        It’s also very personal what each of us finds difficult or easy. At the same time, you have to say “no” to most candidates anyway, so some formal test is in order to even get a semblance of fairness.

        I had three candidates for second (and final) round of interviews recently. All three in the same age bracket and education level (Msc) and all had the same (and only) coding question: sort an array. I would first ask if they feel they can do it; all three said yes. Then I would hand them a computer and let them do it. No restriction for language, algorithm, generous time. Even allowed to use any help material online they need and no requirement to run it: reasonable pseudocode would do.

        The first one couldn’t do it, and you could see they didn’t have much experience iterating over an array. I have no doubt they felt miserable, even though I tried to avoid this exact situation by asking beforehand.

        The other had no problems with it, but looked at me incredulously - as if I asked them to touch their nose or do a squat.

        Third one was visibly nervous, but came up with insertion sort in a few minutes.

        Before that, we were talking shop and discussing applicants’ prior work. None had much trouble with that part. First candidate had the best grades.

        So in the end, I believe none felt too happy in this test. One probably thought the question was unfairly hard, the other must have felt I was insulting their intelligence, and for the third it was stressful even if they were entirely skilled to do it. On my part I wasn’t particularly happy doing this ether even if I think it was necessary.

        Interviewing just sucks!

        1. 2

          Interesting! Experience reports are great. If you don’t mind me asking, what led you to asking candidates to implement a sort routine? Was it something you wanted to evaluate personally, or was it part of an interview template that you had to implement as the interviewer?

          1. 2

            It’s a small enough problem, just few lines of code in most of languages unless you go fancy. It is also a problem that is taught in all CS101 curriculums and I believe it is impossible to get Masters in CS or SE without encountering it. Failing it signals that all other CS concepts are also out of grasp. It is proxy for a whole host of practical problems we deal with: you’d really struggle with most signal processing algorithms or fixing a bug in a spanning tree protocol implementation if sorting an array feels overwhelming.

            1. 2

              I think context is important here. I happen to have met @varjag so I know his company works on quite low-level hardware stuff. I don’t know if implementing an array sort would be appropriate for hiring a developer in another field of software. There are other relevant examples there.

              1. 1

                I know people do FizzBuzz on the interviews, although doing this to people with Master degrees felt silly. But looking back I suspect the result would be similar.

    2. 13

      Some of the psychology you cite is very pertinent and interesting but I really disagree with some of your conclusions.

      I think that if you as a candidate freeze up when asked to perform a basic example of what you’d be doing for your job, then that’s an interview skill you should consider building rather than advocating doing away with that class of interview question.

      I know a LOT of talented people who have great trouble with whiteboarding, and I get it. I REALLY do because I used to be in that category.

      But like any other fight / flight response where no actual mortal peril is involved, you can work through the anxiety and learn to process the situation differently, and once you do, it’s incredibly empowering!

      When I’m doing an interview and I can tell that my candidate is freaking out, I typically ask them to talk me through what they think a solution would flow like, without any of the syntax involved.

      And honestly, what job anywhere is free of this kind of pressure?

      [Full Disclosure: I work for a Major Cloud Company and our interview process is highly data driven and also includes some whiteboarding. I think this hiring strategy has been extremely successful, but obviously YMMV and different companies are different.]

      1. 15

        The problem is, its not only about anxiety. It could be, and is, any number of things, like active stressors (disease, divorce etc.), biochemical stuff (low sugar, hormones), meteorology (too hot/cold) etc. I remember, 10y ago that when I had an interview @booking.com, my head hurt so strong that I could hardly speak.

        And honestly, what job anywhere is free of this kind of pressure?

        Almost any job is without that kind of pressure. Its totally artificial.

        When you measure people this way, you have no option but to ask academic stuff. For example, local Microsoft tech interview I was part of were created by assistants of local faculty of Mathematics, and had questions such as what is the probability that frog will survive the road run and other crap like that. I finished the same math university 5 years before and I still couldn’t manage many questions. Not to mention, just finished students nailed it, with 0 practical experience (it was 0). So, this type of interview is biased toward fresh graduates and academy itself has entirely different goals and agenda far removed from usual everyday IT reality (since then I had number of professors working in my team and each time with disappointing results). Or, member the interview of author of brew for Google ?

        I used exclusively lets-speak-about-random-IT-stuff type of interview and I think it worked OK. I also ask for online profiles - Github, Gitlab, etc… there is no substitute for seeing (or not seeing!) code itself. There are few persons that were great in that while sucked on job a lot, but overall, the initial feeling after the conversation was at least 90% on the spot.

        1. 2
          And honestly, what job anywhere is free of this kind of pressure?
          

          Almost any job is without that kind of pressure. Its totally artificial.

          Couldn’t disagree more. I’ve worked in this industry for 30 years, admittedly only 5 or so of those years as an actual ‘software developer’ (mostly release engineer, devops, and before that highly code-centric sysadmin).

          I can’t think of a single job I’ve ever worked that didn’t have “OK. We need you to produce. NOW” moments.

          If you’ve worked jobs that don’t have that kind of pressure, I envy you, but I’m unsure whether or not your experience is one that many others share.

          1. 15

            Certainly many jobs require deploying a patch, developing a new schema, reacting to a fire in the moment on deadline, etc., but I think few require you to be able to converse about it to strangers who can fire you, in those circumstances. ;)

            My work is not complicated. My experience with whiteboard interviewing has been with insignificant companies that do not do hard engineering. When I froze up and couldn’t think of the name for modulo after having just used the operator, the interviewer decided I wasn’t a good fit… I think less technical or complicated jobs use whiteboard just as much as Google, and that’s frustrating.

            1. 11

              Most of them won’t then stare at you while you do your job on an unfamiliar tool you’re only using so they can decide whether to promote or fire you.

              Instead, they give you some requirements, you leave, you work alone on them at a computer, maybe talk to team members, maybe status reports (mostly typed), and eventually deliver the results to then.

              These are the skills the interview process should assess. That’s why a lot of shops do take-home assignments or just not whiteboarding. Now, whiteboarding in non-hostile environment can be great for testing presentation and collaboration skills.

              1. 4

                Exactly.

                When multiple people STARE AT ME while working (even in familiar environment), I can’t work productively.

                Also, nobody ever comes with the random problem and ask you to solve it yesterday in 1 hour. You always have some context, some continuancy.

                1. 2

                  I can appreciate where you’re coming from here and I’ve done take home assignments in the past. I stand by my comments around whiteboarding as a valuable interviewing tool. I think if interviewers are getting hung up on details they’re missing the point, and I also think that as a prospective interviewee having the expectation that you won’t be asked to give a sample of your work on the spot doesn’t seem realistic to me.

                  However, if all of you can restrict your job search to companies that never do whiteboarding, good on you!

                  1. 2

                    The problem with whiteboarding is that while it measures something, the something it measures is not the thing you’ll care about if you hire them. Which in turn is why there are books out there that teach whiteboard interview coding as a separate skill from actual programming, and why even prestigious universities now include a unit on interview coding as a separate skill from programming.

                    Which raises the inevitable question: why not actually test for the skills you’ll care about on the job? If you don’t test for the job skills you’ll hire people who don’t have the job skills.

                    1. 3

                      why not actually test for the skills you’ll care about on the job?

                      That is what they’re trying to do. It takes a lot of time to find out if someone can actually do a good job as part of your team, and the only way to really test it is to employ them for a few months (A few months of probation is quite common for this). Given that you can’t afford to make that sort of investment in every candidate that applies, companies use whiteboarding and other forms of technical interview to try and guess whether a candidate might have suitable skills before investing more time in them.

                      1. 0

                        That is what they’re trying to do.

                        Well, no. What they’re trying to do is cargo-cult what they think Google is doing, because they think “Google is big and successful, so if our company does what Google does, our company will be big and successful too”.

                        Of course, Google openly admits their process has a high false-negative rate (it flunks qualified people), but they get enough applicants that they can afford to reject some qualified people. The average company isn’t in that position and can’t afford it.

                        And Peter Norvig has explained that Google found a negative correlation between doing well on competitive programming tasks and performance on the job, which throws a wrench in any argument that on-the-spot coding under pressure measures something useful.

                        Interviewing as typically practiced in tech is fundamentally broken. It measures the wrong things, draws the wrong conclusions from what it does measure, and is more or less random. I think it’s long past time to stop defending that.

                        1. 2

                          I never said that the processes were effective, nor am I defending them. I am merely pointing out that they are ultimately an attempt (however ineffective) to select candidates with relevant skills and reject those without.

                          “Why not actually test for the skills you’ll care about on the job?” is unhelpful in that the intention is blatantly obvious, but offers absolutely no suggestion of how to achieve it.

                          1. 5

                            “Why not actually test for the skills you’ll care about on the job?” is unhelpful in that the intention is blatantly obvious, but offers absolutely no suggestion of how to achieve it.

                            Tthere are a ton of articles and talks floating around on how to do better tech interviews – I’ve even written/presented some of them, based on my own experience helping to rebuild interview processes at various places I’ve worked – and people could quite easily find them if they wanted to.

                            But here goes. Again.

                            As I see it, there are several fundamental problems, and various ways to avoid them.

                            Problem #1 is playing follow-the-leader. People implement processes based on what they think bigger/more successful companies are doing, without considering the tradeoffs involved or indeed whether those processes had anything to do with the size/success of those companies. Google’s admitted high false-negative rate is the quintessential example: they really can afford to throw away qualified applicants, because tomorrow another hundred will have submitted applications. The typical tech company can’t afford this, or can’t afford other unquestioned assumptions baked into large-company interview processes.

                            The solution here is to question the assumptions and expose the tradeoffs. The extremes are “Never hire an unqualified person” and “Never pass on a qualified person”. Google optimizes for the former at basically all costs. Many companies, on the other hand, need to push the needle further toward the latter, which means a more forgiving process that doesn’t flunk people as quickly or for reasons as minor as large companies do. True story: I know one person who flunked Google because they had to write and then read out a bash script over the phone and the interviewer mistranscribed it. I know another person who flunked at Twitter because they provided one of two textbook algorithms for the problem posed, but the interviewer only knew the other and didn’t bother looking more deeply than just “that’s not the answer I know”.

                            Those should be unforgivable interviewer errors at any company, but are especially unforgivable at companies which can’t afford to just throw qualified applicants into the trash can.

                            Problem #2 is poor targeting. A lot of interview processes, especially at BigTech, explicitly or implicitly target fresh graduates, by quizzing on the sorts of things fresh graduates will have top-of-mind. Many of those things are not top-of-mind for working programmers who’ve been in the industry a while, since they’re rarely used in actual day-to-day programming (this includes a lot of algorithm and data-structure questions, since most working programmers don’t implement those from scratch on a routine basis). This biases away from experienced programmers, and creates a self-reinforcing cycle: you hire a bunch of recent grads, and they come in and start interviewing people which pushes even more toward preferring recent grads, so you hire even more of them, and… then one day you look around and wonder why it’s so hard to find experienced people. This is especially bad in the places that do algorithm challenges, because usually they’re posing things that want you to come up with a solution that took top people in the field decades to come up with while not under any particular pressure, and they want it from you in 20 minutes. On a whiteboard. While they watch. The only way to pass these is to “cheat” by already knowing the answers in advance, which you do either by reading a book about interview problems, or by being a recent grad who just passed a bunch of exams on the material.

                            The solution here is to interview based on things that are actually relevant to day-to-day programming. You can, if you want to, find out about someone’s problem-solving skill while using questions and problems that involve things real working programmers actually do.

                            Speaking of which, problem #3: far too many interview problems are contrived and unrealistic.

                            You can do interviews based on real-world practical problems. Two of the best interview processes I’ve ever gone through did exactly this: one had a work-sample test where they gave you a simplified version of an actual problem from the domain they worked in, the other did a collaborative session where you had to debug a piece of code extracted from their real system and find the real problems in it. Putting together an interview based on these kinds of problems doesn’t take a ton of time, and gives you a much more realistic idea of how someone will perform in your company than the million and first shibboleth problem that tries to test for “fundamentals” but really only checks whether someone was taught the test.

                            Problem #4 is measuring things that don’t matter. Whiteboard design can be useful, but whiteboard coding isn’t. Algorithm regurgitation isn’t. Trivia isn’t. Having open-source contributions on GitHub isn’t. Having lots of free time to do competitive “challenges” isn’t. And a lot of “soft” factors like confidence aren’t.

                            Measure the things that matter. Measure how well someone can ask questions about a problem or communicate ideas on how to solve it. Measure how well someone works with others (pair programming can make a great interview session if you do it right). Measure how well someone finds and uses resources to help them work through a problem. Measure how well someone interacts with non-engineer colleagues. We’ve all worked with people who were good and people who weren’t so good; figure out what the good ones had in common and measure for that. It’s almost never going to be things like “they were really good at writing linked-list implementations on a whiteboard”.

                            Here are some concrete ideas for more useful interviews.

                            First, always let the candidate use a real computer with real programming tools and access to references, Google, and Stack Overflow. I make a point of telling people that I’ve written significant chunks of Django’s documentation but I still have that documentation open in tabs when I’m working; it’s outright nonsense to forbid that while also claiming you measure realistic performance.

                            Second, use technical sessions that avoid the problems outlined above. Here are ideas:

                            • Code review. Bring a piece of code (as realistic as possible) and have the candidate work through it with you. Have them demonstrate that they can read and understand it, ask productive questions about it, and offer constructive and useful feedback on it.
                            • Pair programming. Bring something with a known bug, have them debug and fix it with you. Have them demonstrate that they understand how to approach it, search for and identify the problem, and work up a fix.
                            • Give them notes from a real problem, and be prepared to answer questions about it, and have them write a brief post-mortem for it. Have them demonstrate they can take in the information you’re giving them, usefully probe for anything missing, and synthesize it all into an explanation of what happened.
                            • Bring them a rough feature spec and ask them to refine it and break it down into assignable units of work. Have them demonstrate they can ask good questions about it, figure out the needs and the tradeoffs involved, and come up with a sensible plan for how to go after it.

                            Third, use non-technical sessions! And not just a “culture fit” lunch with the team. Have them do a session with a PM or designer or other non-engineer colleagues to see how they interact and watch for signs of whether they’ll have productive working relationships with those folks.

                            Finally, standardize your evaluations. It’s OK if there are different possible combinations of sessions (some people may prefer to do a take-home work sample, others may prefer to pair live and in person, etc.), but it’s not OK if interviewers have different rubrics for grading. Write out, explicitly, the qualities you’re looking for, in specific terms (i.e., not “confidence” or “professionalism” – those are vague weasel words that shouldn’t be allowed anywhere near an interview scorecard). Write out how interviewers are supposed to look for and evaluate those qualities. Set explicit baseline and exceeds-expectations bars for each session. Write scripts for interviewers to practice on and follow when presenting problems. Have interviewers practice running the sessions with current employees, and have some of your acting “candidates” try to pull sessions off-script or fail, to make sure interviewers know how to handle those cases gracefully.

                            And finally, treat candidates like people. Someone you’re interviewing should be seen as a colleague you just haven’t worked with yet. Designing a process to be adversarial and to treat everyone as a con man will yield miserable and unproductive interviews.

                            Now, I got voted “-2, troll” in my previous comment for citing sources that the typical coding interview doesn’t measure things that actually matter and in fact selects for things that correlate negatively to on-the-job performance. But I could cite plenty more. This video, for example, is a former Google employee who at one point recounts the story of a frustrated recruiter who revealed to a hiring committee he served on that they’d all just voted “no hire” on slightly-anonymized versions of their own interview packets and how it exposed the brokenness that was going on there. This article from a company that provides interviewing tools goes into detail on just how unreliable it is to use something that seems like it might be a proxy for real on-the-job skills (key takeaway: scores assigned to candidates by interviewers were all over the place, and inconsistent even for the candidates with the highest mean performances, fully one-third of whom outright bombed at least one interview).

                            Interviewing is broken. It can be fixed. Both of these should be uncontroversial facts, but the fact that multiple people here saw them as “trolling” is indicative of the real problem.

                            1. 2

                              I regret I have but one upvote to give for your comment.

                              1. 1

                                Excellent comment. I particularly like the part that goes from practices focusing on recent grads that becomes self-sustaining. That could be actionable under anti-discrimination laws. I’ll try to remember that.

                                If -2 was what I think it was, then that might be how intro had a tone that looked aggressive, dismissive, or something else. People here are more sensitive to that than most places both in terms of most of us wanting civility and what a large group deems as inclusive speech. Your comments will get better reception if you present your counterpoints without any rhetoric that comes off as a personal attack or dismissal.

                      2. 0

                        In my CV you can see that I have created number of large services to be used by entire countries, 24/7. This can easily be checked even real time (I have admin access to all of them which I can demonstrate immediately ). You want me to whiteboard ? No!

                        Furthermore, I will make sure all of my senior IT friends and colleagues know how much you suck as a company (in this land, you can count seniors on fingers ATM) so good luck for you finding one.

                        It looks to me that many people do not get it - senior developers are celebrities today.

                        Me? I’ll rather collect peanuts then make somebody else rich(er) on my work without fair compensation, respect and professionalism.

                        1. 6

                          Me? I’ll rather collect peanuts then make somebody else rich(er) on my work without fair compensation, respect and professionalism.

                          Your response made me take a step back and think about what we’ve all been saying here, and I came to a couple of conclusions.

                          1. I am not a software developer per se. I work in the devops space. I do write software, but it’s nothing even remotely on the order of magnitude that the average Crustacean does. I write simple process automation scripts in Python and occasionally Ruby or Bash. This informs both my world view and the kinds of things I would ask people to whiteboard. As in, they are not at all algorithmically hard, things like “Print the duplicates in this list of numbers” and the like.

                          2. Reading all of you express such vehement opposition to the idea certainly has me questioning its wisdom when interviewing software devs, and also wondering if the experiences you’ve all had were at the hand of people who weren’t very mindful of candidate experience in how they were conducting their interviews in general.

                          In any case, it’s all very good food for thought, and I will now shut up on this topic and think on all of this.

                          1. 5

                            Based on my own experience, it is not that uncommon to find someone with “years of programming experience” on their resume, but has trouble solving basic programming tasks. This is because experience comes in a ton of different shapes and sizes. For that reason, before I can be okay with hiring someone in a technical capacity, I really need some kind of evidence that they can write code well. Whether that’s looking at code they wrote previously, giving them a new coding assignment, a white board interview, a presentation or maybe just prior experience working with them, I don’t really care. But there needs to be something. If I just took peoples’ word for it, the results would be pretty bad.

                            I think people in this thread generally underestimate just how many people are out there that claim coding experience but fail to meet some really minimal standards. I’m talking about things like Fizz Buzz. Some kind of evaluation process is necessary in my experience.

                            Personally, I think the person you’re talking too is being way too unreasonable.

                            Everyone gets way too hung up on this shit. There’s a saying that goes something like, “all models are wrong, but some are useful.” It applies just as well to hiring practices. You can’t get it right all of the time, and some of your techniques might indeed yield false negatives. This is basically impossible to measure, but since a false positive is generally perceived as being much more costly than a false negative, you wind up trying to bias toward rejecting folks that might be otherwise qualified in favor of avoiding false positives.

                            The whole thing is super hand wavy. People just seem to get obsessed with the fact that Hiring Practice X is wrong one dimension or another. And they’re probably right. But so what? All hiring practices are wrong in some dimension. And even this differs by experience. For example, a lot of people love to poo-poo whiteboard interviewing because that’s not reflective of what the job is actually like. But that’s not true for me. I use the whiteboard all the time. It’s a pretty useful skill to be able to go up to a whiteboard and communicate your ideas. Obviously, the pressure of evaluation isn’t there when you’re on the job, but I don’t see how to relieve that other than by limiting yourself to hiring people you already know.

                          2. 1

                            In my CV you can see that I have created number of large services to be used by entire countries, 24/7.

                            I want to remind you that this doesn’t matter at all in terms of good, software engineering. There’s both lots of crap tech out there with high usage and countries that demand even crappier tech. High uptake doesn’t mean anything except you worked on something that had high uptake due to you or more likely someone else given how businesses/governments decide what to use. If you doubt this, I hit you with my COBOL argument: why not hire someone how knows what billions of lines of mission-critical code are written in? Must be better than almost everything else if argument by uptake and net worth in critical areas is meaningful.

                            Or that’s just social-economic forces making that happen in a world where we need to consider other things. ;)

                        2. 1

                          I think the whole point of the article here is that there is no one best type of interview. Some candidates have anxiety attacks when mild pressure is applied, and do much better in a lower-pressure situation like a take-home assignment, and don’t mind spending the time on it. Some candidates have families or other obligations and can’t spend (unpaid) hours outside of their normal job writing up a solution to somebody’s take-home assignment, but do just fine solving problems on whiteboards. Probably some others have issues with both of those and need something else again.

                        3. 9

                          I think that’s a very different situation when you’re already comfortable with your environment.

                          1. 9

                            Couldn’t disagree more with you, either. I’ve also been in this industry for 30 years with almost all of them being an actual ‘software developer’ and I can count on one hand the number of times that I’ve had “produce now!” moments. Perhaps I’ve been lucky, but in my experience, these are rare, 1% times, when there’s a demo or something the next day and you’re trying to get something out the door. Given that, why exactly should we measure this? Even given those high pressure situations, you’re not standing alone, at a whiteboard or in front of a panel, with no resources ( google, library, other engineers ), no context ( mystery problem… you don’t get to know until you have 2 hours to solve! ) and no backup plan. Even with all of those caveats, I have NEVER had to cough up a random data structure with no reference material/resources/etc. in less than an hour or two EXCEPT in an interview.

                            1. 1

                              What I’m hearing is that a lot of people have had really bad experiences with whiteboard interviewing.

                              If the interviewer is hung up on syntax, they’re IMO doing it wrong.

                              1. 2

                                If someone presents a problem for you, do you immediately recall all algorithms you’ve learned and start whiteboarding that? That’s generally what happens in these whiteboarding sessions that I’ve been in. I don’t remember a ton of stuff off-hand and search for it all the time. Should that count against me? If it does, I don’t really want to work at that place anyway.

                                1. 1

                                  I realized (as I wrote in another reply to someone else) that there’s a disconnect here.

                                  I’m not a software developer. I do devops work which means I write a lot of VERY simple process oriented automation scripts in Python and occasionally Ruby or Bash.

                                  When I do whiteboarding with candidates, the most algorithmically complex I get is “print the duplicates in this list of numbers” but mostly it’s things like “Here’s a json blob that lives on an http server. Write code that grabs the blob and connects to each host in a group given on the command line” type things.

                                  But even so I certainly see where you’re all coming from, and can appreciate the idea that there are definitely better tools out there, especially for cases where you’re not doing hiring at scale.

                                  Which makes me wonder, how might one apply all this anti-whiteboarding sentiment in a large corporate environment? How do you get take home exams, pairing, or whatever to be effective when you need to hire 50 people in a month?

                                  1. 2

                                    I used to work at Capital One and we were always trying to hire people. Each team largely handled its own hiring, though, and we had .. so many teams. Some teams do a whiteboarding session, some do a take home assignment, some do a little pair programming exercise. It really depends on the team and the people.

                                    I’m not anti-whiteboarding for things like what you mentioned, but if someone is asking me to regurgitate a CS algorithm that I haven’t touched in decades, I don’t really get the point.

                            2. 3

                              Hmm. I’m not up to 30 years experience at this point (closer to 20), but I’ve never had this happen to me. Even when production bugs hit, it’s not a “We need you to produce. NOW” scenario, but an all-of-us circling the wagons type of deal to track it down. Even in startups where I was the only person writing code, it was never like that. That seems really unusual to me. I don’t know what country you’re in, but for context, I’m in the US.

                              1. 4

                                OK. We need you to produce. NOW

                                Oh, you work for those type of guys who think programming is like manufacture so you can measure it by the number of bricks in the wall, LOC, or something. I don’t accept that kind of job. I know better. I witnessed it dozens of times for other companies I consult for and that kind of job is always substandard.

                                I envy you

                                Its better idea to fight for your rights (our rights, really).

                                but I’m unsure whether or not your experience is one that many others share

                                I am sure its not. I am sure its opposite. But that is where I draw the line. I don’t mind fixing simple things here and there ASAP, but designing entire/parts of app/service without adequate time is no bueno.

                                1. 4

                                  Oh, you work for those type of guys who think programming is like manufacture so you can measure it by the number of bricks in the wall, LOC, or something. I don’t accept that kind of job. I know better. I witnessed it dozens of times for other companies I consult for and that kind of job is always substandard.

                                  This is kind of a cheap shot and also? It’s not true.

                                  But even if it was, people work for different reasons. I live in a society that doesn’t provide much of a net if you fall, and I have some medical conditions which require some fairly expensive care.

                                  So, yes I work for a company that pays well. I put a LOT of blood, sweat and tears into my job, and I won’t apologize for that.

                                  Might it not be the kind of place you’d want to work? I don’t doubt that, and I’m glad you’ve been able to find work that suits your particular needs and wants, but please consider that not everyone is you before making broad statements about people and workplaces when you have zero information.

                                  1. 4

                                    Besides the broad generalizations, I find myself between both of you. majkinetor strikes a cord when he says one must fight for their rights. I’d add that once one finds ones self (if ever) in a position wherein one can enlighten the company masses than one should do what one can. But, feoh, isn’t incorrect either. Sometimes you just gotta play the game. I have only worked for places that tend to think of programming in terms of lines of code. It’s really why I am always looking for a new place to work. And, yes, I hate interviewing as well, which makes my conundrum annoying.

                                    1. 1

                                      But even if it was, people work for different reasons. I live in a society that doesn’t provide much of a net if you fall, and I have some medical conditions which require some fairly expensive care.

                                      We do generalization here, any kind of behavior may be adequate in specific context.

                                      So, yes I work for a company that pays well. I put a LOT of blood, sweat and tears into my job, and I won’t apologize for that.

                                      Who asked you to do this ? When I speak, I speak about myself and what I think, I am not giving lectures or judgement.

                                      when you have zero information

                                      Not zero, I am in this business more then 30 years (also with medical conditions). Don’t be angry :)

                                      1. 2

                                        Not zero, I am in this business more then 30 years (also with medical conditions). Don’t be angry :)

                                        I can see where I came off as defensive there. I’m not angry, I’m just surprised at the expectations some folks have around the interview process.

                                        If you can find work with those expectations, that’s fantastic.

                                2. 1

                                  I think you’re combining two different things here - having a candidate write things on a whiteboard in an interview, and interviewing a candidate based on solving highly abstract problems that have little relation to the actual day-to-day work of building software, stuff like writing binary tree manipulation algorithms.

                                  I think the second is a ridiculous thing to judge candidates based on almost all of the time, unless of course you’re doing one of the few jobs that actually involves doing stuff like that.

                                  The first though, is a perfectly good way to determine whether candidates can actually produce code.

                                  1. 0

                                    The first though, is a perfectly good way to determine whether candidates can actually produce code.

                                    Producing code on whiteboard? I had that on university on exams and many “programmers” came out of it without using a computer single time during the entire studies (I know, my sister was one of them, never turned a computer on until she finished computer studies! and got a job). Doesn’t inspire…

                                3. 4

                                  Mine is kind of like the protagonist’s job in Office Space, but with more SIP trunking.

                                  1. 3

                                    At most places I’ve worked pressure was 99% of the simmering kind. Less than 1% I’ve had that flashpoint pressure but frequently it was artificially created to “increase productivity”. Basically the only time when you have to deliver NOW is during major production incidents.

                                    I’d bet that even at your “Major Cloud Company” the percentage of people involved in solving a major production incident are in the low double digits.

                                    I have frozen during whiteboard interviews and I’m far from a shy person. Day to day I’m more likely to be the kind of person that makes others freeze - and I don’t say that as some sort of self-praise, quite frequently it’s not a good trait.

                                    In my opinion, even with your highly data driven process, if you work for a “Major Cloud Company” you can just ask for a painting competition among your candidates and you’d get the same results. Your company probably pays a very high salary compared to the market so competition would be fierce and people willing to train for 6 months on the intricate details of painting while holding a full-time job at the same time would probably make at least half-decent coders :)

                                    TL;DR: It’s the high salaries and very competitive recruitment pipeline that produce results, not necessarily the actual interviewing details, in my opinion.

                                    1. 1

                                      I share your opinion - note that, in my mind, the “increase productivity” methods actually are very harmful in the long run

                                    2. 1

                                      But like any other fight / flight response where no actual mortal peril is involved, you can work through the anxiety and learn to process the situation differently, and once you do, it’s incredibly empowering!

                                      I’ve been thinking about this point for several days now.

                                      I haven’t been in the habit of thinking about interviewing in terms of a set of skills one can acquire/improve. However, perhaps if I can get myself into this mindset, it will help me to do something other than just walk into a new place and fly by the seat of my pants :)

                                      1. 1

                                        I’ve been thinking about this point for several days now.

                                        I haven’t been in the habit of thinking about interviewing in terms of a set of skills one can acquire/improve. However, perhaps if I can get myself into this mindset, it will help me to do something other than just walk into a new place and fly by the seat of my pants :)

                                        I am sincerely grateful for the fact that at least one person took the advice I was giving in the spirit in which it was meant :)

                                        It most definitely is a skill, and here’s how I know that:

                                        There was a period in my career when I was much better at interviewing than actually doing the work. My work ethic has never been in question, I will work to the point of stupidity at times, but I had made some rather near sighted choices around which skills to build (learning new technologies in a very shallow way VERY quickly) over skills I should have focused on (attaining mastery in a few key tools and subject areas)>.

                                    3. 3

                                      I used to love technical interviews and was quite good at them. I think out of 40 or 50, I flunked one time. But heck if I know what they’re good for. (I’ve conducted a bunch too.)

                                      This article starts down the right road by acknowledging that we don’t take the interviewee into consideration enough. I think a deeper problem is that the selection process rarely mirrors what we actually want in a new hire.

                                      Can work through tough tech problems? Ok, but do we really spend a lot of time wondering whether something is O(n) or O(log n)? More directly, is that something everybody in the building needs to know? Or is it something that only a few people need to know as long as we work well together?

                                      Analogies in tech suck, but the best I can come up with is that a good tech team/company is like a Jazz band, or an improv comedy troupe. Your job is to first master your skills, but just as importantly you have to pay attention to others, the bigger picture, and the people you’re trying to help. You work together fluidly and at the appropriate technical depth each task requires without letting the show sag.

                                      That means three things: 1. This is extremely dependent on the personalities and situations of the various people involved, and 2. There is no “right” person for a large group of 20-2,000 projects. You can rock one situation and be the dead weight in another. And 3. It’s gotta be a mutual-selection, audition-style tryout. You can try various tricks to simulate an improv comedy sketch or Jazz set, but hell if I think you’re going to accomplish much of anything. The best I’ve seen are people who can reliably demonstrate that they’ve worked in a huge number of various environments and everybody liked them and things came out okay. But really, all that gets you to is “mediocre”

                                      Here’s a mental experiment to try out. Let’s say you find a person that is a complete tech idiot. They couldn’t work a calculator, much less code a system. But when that person is added to this team you’re hiring for, suddenly the team gets 3x as effective, the customers are loving what they see, and everybody loves their work. Would you hire them?

                                      I’d hire them in a heartbeat. As long as they’ve got a good attitude and will work with the rest of the team, who cares about the tech interview? But wait, you say. If you did that consistently you’d end up with teams full of people who couldn’t code?

                                      Really? Then how could they be so productive and the customers love them? And more to the point, even if they couldn’t code – suppose they’ve got some special magic wand that let’s them outsource everything – who cares again? Are we here to code, or to be happy, productive, and provide value?

                                      1. 3

                                        Regarding a more detailed feedback and code review. I’ve had bad experiences before.

                                        I did a code review for a candidate applying for dev. He asked for more detailed code review and then didn’t take the feedback well. I suspected something like this could happen so I actually asked a very Senior dev to check my feedback before sending it.

                                        After a couple similar cases, HR stopped sending detailed feedback

                                        1. 1

                                          While it’s a nice idea that even when you don’t get the job, the company can still give you feedback to help you learn something from the experience, I can’t imagine it ever really working out:

                                          1. Can you be sure that the person reviewing your code is suitably experienced? You’re potentially just getting a random employee’s opinions.
                                          2. Do you expect a thorough code review, given that it requires the company to continue investing employee time in a candidate they have decided not to hire?
                                          3. Besides the waste of time, there are potential legal pitfalls for a company giving any sort of feedback: Better for them to say nothing or quote an HR-approved response than to risk anything that could even suggest discrimination in their hiring process.
                                          4. Assuming you get some feedback, at least part of the feedback will be focussed on why you/your code aren’t a good fit for that particular company, and won’t necessarily apply anywhere else.

                                          Put all this together and I can’t see why you’d even bother asking for feedback. I would, however, be interested to hear from anyone who has received useful feedback after an interview or similar.

                                          1. 1

                                            I agree that this isn’t a great idea. There are too many things that can be used against you as a company. On the flip side, I’ve been in scenarios where I wanted to know why they passed and wished they would’ve provided information. It ended up making me seek out help outside of my own circle though, and got me involved (briefly) with some open source stuff.

                                          2. 3

                                            I also really struggle with technical interviews. The timed take-home tests are probably my weakest area right now, it is something about that clock ticking that quite literally makes my hands start shaking. I have been writing code for 20 years, the last 10 years in some kind of lead/staff role. I have recently worked my way through all the exercism puzzles and yet… add a clock and I can’t function. These are new to me since they weren’t around the last time I interviewed - or not as prevalent.

                                            I don’t think it is about pressure, one area I thrive is when there is a critical outage and the solution requires quick and decisive actions - that doesn’t bother me.

                                            Anyway, FWIW - I really like your suggestions here but what I have come to terms with is that if that is how they want to hire and if these are the things they care about - I probably am not going to be a good fit for the company. It does not take all of the sting out of the rejections, but it does help in reflection. I keep doing them because I do believe I can get better if I practice. But in the meantime, I just trust that the right opportunity will come along.

                                            1. 3

                                              That’s really interesting as many people are the exact opposite. The stress of not having a time limit causes anxiety because you’re constantly wondering how much time and energy the rest of the applicants are spending on this challenge and whether your submission will look bad in comparison.

                                              I have a horse in this race as I run what I believe to be the first time-limited technical challenge tool that allows you to use your own editor/IDE. - https://takehome.io

                                              I think what this underscores is the author’s point about building flexibility into your process that can allow for the different preferences of your candidates. There’s no one-size-fits-all interview technique.

                                              1. 2

                                                How do you, at takehome, make sure candidates don’t fiddle with the git history so they maybe look like they performed better than they did?

                                                1. 1

                                                  What do you mean by fiddle?

                                                  If just rebasing, moving commits around, etc. to make the commit history look Good(TM), when you’re worried that your natural workflow looks unprofessional e.g. if it’s anything like mine:

                                                  git add -A && git commit -m "more stuf" && git push
                                                  git add -A && git commit -m "dang" && git push
                                                  git add -A && git commit -m "ttt" && git push
                                                  

                                                  I don’t see that as a problem. The end result is a clean commit history - whether it happened during the writing of the code, or after the fact, is irrelevant - the candidate demonstrated that they could A) write a functional POC and B) release it with (eventually) good commit history.

                                                  Also, the above disorganized git commits I find are almost unavoidable for “first commit” style POCs, when there is no codebase.

                                                  Once the codebase is established, maybe then creating clean commits is expected as a skill, when working with a team on a large codebase - but even so you can do dirty crap in a branch and then fix the commits before creating a PR.

                                                  1. 1

                                                    My understanding of takehome is you use git (partly) to make sure assignments are done inside a desired schedule. But since you can edit git history, people can easily use three hours but edit the history to make it look like they used two.

                                                    But maybe I misunderstood what takehome uses git for?

                                                    1. 1

                                                      Ah, I see - didn’t think of time limit cheating.

                                                  2. 1

                                                    Takehome has a custom git server implementation. When the candidate fetches the repository from their unique URL a timer starts. When that timer expires, the git server stops accepting pushes from the candidate into that repo.

                                                    While the submission window is open they’re welcome to fiddle with the history all they like. All I care about is that they didn’t spend longer than allowed writing their response.

                                                    1. 1

                                                      That makes sense. I thought you where relying on normal git history to track time.

                                              2. 1

                                                The hiring process for developer roles, especially better paid ones – is really multi-dimensional.

                                                I think many companies who can afford to pay risk free 250+ K USD dev salaries, have over-designed negative filters (eg reasons why not to hire).

                                                Some of those filters are for technical proficiency, some are not.

                                                Most companies will not disclose which ones of the negative filters were triggered in particular job interview. Legal action by a rejected applicant, (with a demand to disclose how others faired against same filters) – is, probably, one of the reasons, there may be others.

                                                On a somewhat positive side of things, I suppose, for companies that cannot afford to pay as much but looking for quality candidates, at least have a chance. Over time those companies, if successful, will be able to offer (via a raise or bonus) similar compensation to a candidate.

                                                In my view, the bigger and, probably more unethical practice – is giving higher salaries for same job to new hires (often decorated with made up titles and made up responsibilities) – Compared to folks who performing similar jobs at same skills, but who have been with a business for a few years.