1. 11
  1.  

  2. 10

    I don’t think it is impossible to do estimation… but one needs to remember a couple of very hard facts about it.

    • To do it meaningfully takes a significant chunk of effort and time, both in data collection and processing. This costs money.
    • It will have fuzzy probabilistic answers. (If it doesn’t, its a meaningless lie).
    • It will need to be updated and revised as new data comes in. (More work/cost, and somebody has to collect that data, and the person who wanted the estimate has to read and react to the new results.)
    • It’s value may be less than the cost to provide it.
    • Higher order moments of the distributions in the estimates will not be finite. (Get use to it. It’s a maths / stats thing. Real life things involving humans can have finite means and non-finite variances.)
    • The way you use the answers can have extreme negative effects on morale / loyalty / productivity / profitability. Think carefully before using it.
    • The answers will come with fine print. Read it. If they don’t come with fine print, it’s junk.

    I’m not say #NoEstimates.

    I’m saying keep your brain switched on and do the right thing whatever that may be.

    If you want an “instant” sanity check….

    Whenever somebody asks you for a number…..

    Instantly say, “It’s 42, now what are you going to do with it?”

    By far most of the time they go away and stop bugging you.

    1. 9

      I think that the correct, moderate approach is to show a hatred toward the one-sided transparency that programmers are expected to allow into their work (fuck Scrum, fuck user stories, and fuck all of that low-status shit) but not to throw out the entire concept of estimation. It is (deep breath) sometimes a useful concept.

      Most programmers experience requests for estimates in the following setting: they’re in an open-plan office (visibility from behind suggests low social status) and some person, ego inflated a bit by position (we’re all susceptible to this), swings by and interrupts our work, in order to ask “How long will X take?” Programmers hate this. It suggests four things that are all aggressive and negative.

      1. even if I’ve said, “This is not a commitment”, I might decided to hold it against you if you don’t have X done by that date. I have plausible deniability here; I’m a busy man and can’t be expected to “babysit” you, and it was your responsibility not to say “January 15th” if you weren’t confident in that date
      2. if your estimate is too far in the future, I might jerk you around and put you on something else, or attribute to you a lack of confidence. My own confidence is waning. Drop what you’re doing and FEED ME, or I’ll start making erratic decisions that create, for you, that embarrassing pattern of being jerked-around without a chance to finish anything. (Then, what will you be able to say for yourself at year-end review?)
      3. I’m more important than you and therefore my emotional itchiness about timeframes matters more than your state of flow and your efficiency at actually-getting-the-work-done.
      4. I don’t trust you and I expect you to estimate times even on micro-features, and even though the sum of estimates is not always a good estimate of the sum (the Central Limit Theorem doesn’t apply to estimated human labor) I am going to act as if it does.

      None of these are good messages.

      Don’t get me wrong: I’m more than willing to discuss what I think deserves a year of focus and what deserves only a week or two, but on my terms, and if I trust that what I say won’t put me at risk of some radical change in course. If some manager tells me to do something but to “timebox it to a week”, then I question why the hell I’m doing it in the first place. If it’s so low in importance that I should walk away from it (with the work uncompleted and the embarrassment that this involves) if it takes more than a week, then shouldn’t I be doing something that actually matters? A “timeboxed” low-importance project has only downside (you’ll be embarrassed if it takes too long) and if you’re going to use an engineer at my level on that kind of nonsense, then get the fuck out of here. Give me something that is high enough in importance and status that I’m trusted to call time frames, or fire me and hire someone younger, dumber, and cheaper.

      The other thing I’d say about project planning is that it is an important skill for some fields. NASA isn’t going to let its engineers get away with “It’ll be done when it’s done”, because they’re hurling complex machines into space and that’s hard and it requires professionalism. They have adult project planners (not 24-year-old Ivy grads who learned Scrum and became PMs after getting a C+ in their first hard CS class, but people who actually understand how to plan massive million-dollar projects and who have enough professionalism to set milestones that can actually be met with 90+ percent confidence, rather than aspirational/unrealistic nonsense deadlines set by executives who think they’ll make the grunts work faster) who do every thing they can to make sure that deadlines are generous but slippage doesn’t occur… and who have recovery plans in event that slippage does happen. If engineers are treated as equals, and not meat in the machine, you can get enough information out of them to form reasonable confidence intervals. Of course, this applies to a different kind of development. At NASA or JPL, slow coding is a given because the quality of the software has to be extremely high, while requirements aren’t going to change for stupid reasons. When engineers aren’t expected to churn out features at a manic pace and planning and analysis are made part of the job, you can get relatively high assurances that something acceptable will be available by a certain time. It requires a completely different set of assumptions about software and what the programmer’s job is, though. Most extremely-high-reliability teams (e.g. NASA, JPL, defense think tanks) are 30-60% female and have average ages of 35-45, because this kind of work requires people who are mature enough to put team-wide correctness over individual-level flashiness. If you’re a startup where each engineer starts with sub-0.1% equity and is jockeying for the slim chance at an executive position where real equity/upside might be given, you won’t get that kind of work or culture; instead, you get the typical startup culture where every engineer is out for him- or herself and the appearance of being an individual high performer matters more than the final reliability of the team’s code.

      Realistically, we can’t expect estimates to go away. For one thing, they’re sometimes useful. No one likes the emotional, arbitrary deadlines that control-freak executives set, but there are real deadlines in this world that actually must be met– rocket launches, proposal deadlines for governments, regulatory demands– or bad things happen. What we do need is for the discussion to be more two-sided. If there’s a real deadline (and not, “some executive will have an emotional fit and change priorities in a stupid way”) then everyone should know what the deadline is and what must be delivered, and the engineers should be expected to proactively raise the issue if they anticipate slippage.

      Estimation isn’t as important as it’s made out to be, though. Most corporate IT tasks don’t need to be estimated at all because these deadlines are just shitty resource limits (to which, blowing through it– the “whoops” approach– is an acceptable response if other communication breaks down) and not real, hard deadlines. I once heard an Agile coach say that the difference between a senior engineer and a junior one was the ability to estimate tasks, and I was disgusted, because that is flat-out insulting to what I do for a living.

      What needs to die, though, is the expectation of one-sided transparency, which represents low social status. Individually, one has to fight against it by turning the estimate request into a multi-round negotiation (I give you some information, then you give me some, and we iterate) and therefore establishing positional equality. If you want to know how long I think X will take, then I have a right to know why you are asking me. Is there a new requirement or deadline that I ought to be aware of? Then tell me. Is there an executive who’s getting an itchy trigger finger? Let’s talk, as equals, about how to get him off our backs. Of course, the “Why do you want to know?” approach rarely works directly, because businessmen are much better negotiators (and more energetic at it) than most of us, so they’re not going to show their cards first, and bluntly asking them to do so is going to turn them off. (It’s not socially acceptable to say, “My answer depends on why you want to know”; people read it as dishonest. You have to give some of your answer before you can ask questions.) So you do it in multiple rounds: giving low-value information, then asking for something in return, and iterating until you have some insight into how the estimate will be used (for good, or for evil?) If you make it a multi-round discussion, you have better odds of being able to protect yourself. However, most programmers just want the businessman or manager to go away and so they make up bullshit numbers, figuring that they’ll deal with it later. This would be the right approach to the 90% of deadlines/estimate requests that are manager-level emotional nonsense… but it leaves the managers feeling that they can’t trust us when it comes to that 10% of deadlines that actually matter. So the “make the manager go away, so I can get back to work” strategy hurts us in the long term.

      I suppose what all of the above is here to say is that the Estimate Question is complicated. Ultimately, #NoEstimates is a great idea insofar as the Scrum/Jira commodity-programmer culture needs to die a fiery death, and anything that encourages corporate programmers to professionalize and assert themselves as adults is welcome. That said, I don’t think we can abolish all forms of estimation, or that it’s even desirable. What we should do is murder the concept of programmers as business subordinates and replace it with a world in which programmers are trusted professionals who may be asked to estimate or justify time, but only when there is a hard case (i.e., a rocket launch, not some middle manager’s image) for his or her having to do so.

      1. [Comment removed by author]

        1. 1

          Maybe I’m idealizing NASA’s capacity for project management, then. I would assume that there are some time-sensitive projects that require planning and easy-to-meet deadlines… but it sounds, from what you’re saying, that NASA imposes “soft” deadlines too, the virtue of it being that people are reasonable enough not to take them too seriously.

          I guess what makes it worse in business is that there’s the same lack of rigor in setting timeframes, but that people are actually held to them and that projects get cancelled or jobs lost when those arbitrary dates aren’t met and the TWiGs (Toddlers With Guns) management throws an emotional fit.

          1. 2

            TWiGs

            Acronym noted for future use.

            Thanks!

        2. 3

          If some manager tells me to do something but to “timebox it to a week”, then I question why the hell I’m doing it in the first place.

          I completely agree with you on this subject, well said. I would add, though, that this bit depends, at least to some extent, on the project in question. Being asked to timebox a research or experimental project seems very reasonable to me if the goal isn’t to generate production code and the amount of time allotted isn’t absurd (“take the afternoon and choose the best distributed message queue for our needs”).