1. 21
  1.  

  2. 30

    I’ve read a bunch of articles like this over the years and I never find them useful. Every paragraph boils down to something like “You shouldn’t do this thing because it’s slow. Unless the alternative is worse. Use your judgement.”

    If I already have good judgement, the article doesn’t tell me anything I don’t already know. And if I don’t have good judgement, all it does is give me more ammo to defend either of my possible uninformed opinions. If I decide to use an array, Joe tells me I’m doing the right thing:

    They may choose a linked list, for example, because they want zero-allocation element linking via an embedded next pointer. And yet they then end up with many lists traversals throughout the program, where a dense array representation would have been well worth the extra allocation. The naïve programmer would have happily new’d up a List<T>, and avoided some common pitfalls; yet, here the senior guy is working as hard as humanly possible to avoid a single extra allocation. They over-optimized in one dimension, and ignored many others that mattered more.

    But if I decide a linked list, I’m also doing the right thing because arrays cause all of that reallocation:

    The ToArray method doesn’t know the size of the output, so it must make many allocations. It’ll start with 4 elements, and then keep doubling and copying elements as necessary. And we may end up with excess storage. If we end up with 33,000 elements, for example, we will waste about 128KB of dynamic storage (32,000 X 4-byte ints).

    Should I allocate something on stack? Yes!

    Because they are so easy, it’s doubly important to be on the lookout for allocations in C#. Each one adds to a hard-to-quantify debt that must be repaid later on when a GC subsequently scans the heap looking for garbage.

    Or wait, on the heap?

    Most GC-based systems, for example, are heavily tuned to make successive small object allocations very fast.

    The worst category of optimization is one that can lead to brittle and insecure code. One example of this is heavy reliance on stack allocation. In C and C++, doing stack allocation of buffers often leads to difficult choices, like fixing the buffer size and writing to it in place.

    Articles like this are a good reminder that you need to write with a specific audience in mind. Duffy has a ton of expertise, but he doesn’t know which subset of it is relevant to the reader, so he shotguns it all out. But much of it cancels each other out, leaving little guidance behind.

    I have definitely worked with programmers who really need to take “premature optimization is the root of all evil” to heart. The kind of person who micro-optimizes every line of code into an unreadable calcified mass of clever hacks like using xor to swap values. I know exactly what missive I would write for that person.

    And I’ve worked with others, architecture astronauts who seem to think a compiler can work magic and that the best way to cure too many layers of abstraction is by hiding them behind another layer of abstraction. I know what essay I’d send to them.

    But I don’t know if I could write a good article that I would send to both of them.

    I do like all of the points that Duffy makes: allocation matters, readability matters, the compiler and runtime aren’t magic, think about performance, measure. I’m on board with all of that. I just worry that none of it is very actionable to someone who doesn’t already have the expertise Duffy is trying to pass along.

    1. 3

      But I don’t know if I could write a good article that I would send to both of them.

      Perhaps you might try sending them this article?

      A common difficulty in convincing people – especially about something like this, that really is a matter of taste and judgement, is that to really appreciate the argument you already need to have good taste and good judgement.

      And yet there are people who clearly do not have good taste and good judgement.

      One way I find success is by finding other perhaps convoluted arguments that I feel say something I already find obvious.

      I am not certain whether this method is successful because by showing more people “saying the same thing” I am making the position appear more popular, or it is because the different articulations resonate somehow and find its way in.

    2. 9

      What I do advocate is thoughtful and intentional performance tradeoffs being made as every line of code is written. Always understand the order of magnitude that matters, why it matters, and where it matters. And measure regularly!

      I have an issue with this - NONE of these things has anything to do with premature optimization! Knuth didn’t say “don’t be mindful of performance” or “don’t benchmark your code”. These are very different things.

      1. 5

        Premature optimization is evil because we humans are too stupid to keep to many “smart” concepts in our heads. Every author who tries to disclaim that is just too full of himself to admit this fact, and in my experience pushing the optimization at the end prevents a lot of bugs from being created in the first place.

        Using a language to avoid side effects is a good choice. I am explicitly not talking of functional programming languages only, as you can go far enough with procedural languages as well if you split your problems up into simple sub-functions well enough. Inside those functions, which cover a small scope our brains can comprehend, you can apply optimization techniques because that is more or less the only thing you have to think about in a “safe” context. Job done.

        1. 3

          and in my experience pushing the optimization at the end prevents a lot of bugs from being created in the first place.

          No one ever lost their job for optimizing later^0. This is sound advice. So is the advice of making small side-effect free functions.

          you can apply optimization techniques because that is more or less the only thing you have to think about in a “safe” context.

          And yet, 97% of the time these optimizations won’t be meaningful. ;)

          0: Completely unconfirmed. :)

          1. 2

            The Wikipedia page about Twitter has a good summary of their early history, and the scaling challenges that could have killed the company. It references, among other things, this Macworld article from 2007 about the first time it became clear they were growing faster than they could handle.

            To Twitter’s credit, they then treated optimization as an emergency, and had substantially mitigated the situation by late 2009. If the company hadn’t reacted, perhaps nobody would have been fired directly because of not optimizing, but they would certainly have lost their jobs eventually.

            It was definitely an exceptional situation, and I don’t think there’s any clear lesson to be drawn from it. After all, it worked out for them. And very few companies' products are ever going to scale to this extent or in these dimensions; it is not surprising that Twitter had to learn by experience.

            I do wish that articles like this would talk about these choices as business questions; they stand at the intersection of business and engineering, and almost everything I’ve ever seen written about them has only addressed the engineering side.

            1. 7

              about the first time it became clear they were growing faster than they could handle.

              What if Twitter had assumed it would take off and grow to millions of users overnight? How long would it have taken to build out that system? And, if they had, would it have actually worked? Chances are they would have had to rewrite everything anyway, as some of their assumptions as to what would actually matter would be completely wrong.

              I do wish that articles like this would talk about these choices as business questions; they stand at the intersection of business and engineering, and almost everything I’ve ever seen written about them has only addressed the engineering side.

              I completely agree.

              Back to the case of Twitter, I can’t help but think they did everything right from a business perspective, early on, as rocky as it was. Over optimization up front might have killed them, in terms of missed market opportunity, cost, etc. Instead they built a service that people couldn’t live without, got a lot of PR (no such thing as bad PR, right?), and their passionate users either a) joined the company to help, or b) waited it out and made fail whale jokes (adding even more buzz, PR, etc).

              1. 1

                What if Twitter had assumed it would take off and grow to millions of users overnight? How long would it have taken to build out that system? And, if they had, would it have actually worked? Chances are they would have had to rewrite everything anyway, as some of their assumptions as to what would actually matter would be completely wrong.

                Totally agreed. Even in general, it’s hard to know where the pain points are - and that’s a point for “don’t optimize up front” - but in this case, it was more difficult than usual because it’s a distributed system with novel properties.

                Agreed on the rest, as well. :) It’s an interesting example to think through, hopefully.

              2. 1

                Why did Twitter succeed and pownce fail? Was pownce too performant?

                1. 3

                  Pownce was pounced by Facebook. Facebook implemented file sharing, events, etc, which is the value proposition for Pownce. Six Apart bought them right before I joined in 2009, but mostly for the tech (which actually never got used…), and shut down the service.

                  1. 1

                    Honestly, I have no idea. I wasn’t aware of Pownce during its existence. :)

                2. 1

                  It depends really. Most of the time, a program will be stuck with I/O, so optimizing I/O can have benefits. I’ve been working on high-throughput tools in the last few weeks and managed to get a 75% performance increase by reading and writing larger chunks instead of doing atomic reads and writes. YMMV.

                  On the other hand, I agree that the compilers can really optimize the shit out of a given algorithm. Never go with snake-oil-approaches, but in many cases, you can approach any problem numerically and get more for your bucks.

                  1. 1

                    Most of the time, a program will be stuck with I/O, so optimizing I/O can have benefits.

                    I certainly don’t disagree, but notice; if most of the time is spent in I/O, that might be your fabled 3%.

              3. 4

                Waiting for a “Declaring premature optimization is evil a myth considered harmful” article :-)

                1. 2

                  I believe the premise here is that premature optimisation can lead to non-optimal code. Which is true.

                  However, I’ve always understood Knuth’s maxim as saying that premature optimisation can lead to unreadable and unmaintainable code. Which is a different thing.

                  Specifically, that optimising while in early stages of the project while the architecture is still in fluid state can lead to bad architectural choices.

                  As an example consider a function that creates object foo. The most natural expression of the concept is:

                  handle = create_foo();
                  

                  However, premature optimiser may be worried about the cost of the implied memory allocation and thus they change the API like this:

                  struct foo f;
                  create_foo(&f);
                  

                  This immediately leaks internals of foo to the user (forward declaration of struct foo won’t work in this case!) It also means that user has to trace two interconnected lifetimes (that of the structure allocation and that of its use) instead of a single one. In the end, user’s code gets much more ugly.

                  As a long-time C programmer (where you can’t easily pretend that optimisation is someone else’s job) I am often struggling to avoid the temptation.

                  1. 3

                    However, premature optimiser may be worried about the cost of the implied memory allocation and thus they change the API like this:

                    Letting the library user control allocation is always better because they are in a better position to recognise how much stack is available, how long the lifetime of the allocation is, and how many foo they’re actually going to need.

                    It also avoids a whole class of bugs, like the malloc in the library not being the malloc in the main program simply because of how the dynamic linker works, or knowing who’s responsibility it is to make sure the memory is “freed”.

                    This immediately leaks internals of foo to the user (forward declaration of struct foo won’t work in this case!)

                    This is not necessarily a bad thing.

                    Having access to the internals of foo is extremely useful for tracing and debugging, and whilst it may prompt inexperienced programmers to create layer violations, I’d argue that those people might also consider hardcoding offsets within the opaque structure to get what they want.

                    Encapsulation for encapsulation’s sake is something “Object Oriented” programmers think is important. I think that if you want to program Java in C you might as well use Java.

                    However, I’ve always understood Knuth’s maxim as saying that premature optimisation can lead to unreadable and unmaintainable code. Which is a different thing.

                    I think this is a good point, however understanding exactly what optimisations are “premature” and which ones are thoughtful is an important skill to develop, and a major point in the article.

                    One observation I have is that many things programmers do in an effort to be “more clear” is to use many more, but simpler words. This makes the program much bigger and more complex, and therefore makes it harder (longer) to read, and slower to execute. I think it is important to appreciate this person argues writing code this way because “premature optimisation is the root of all evil”, so I think in order to have any real discussion about this, we need to make sure of what we’re actually talking about.

                    1. 2

                      Letting the library user control allocation is always better because they are in a better position to recognise how much stack is available, how long the lifetime of the allocation is, and how many foo they’re actually going to need.

                      I wrote a reasonably sized personal library with pointers everywhere. “They call this library function to create, this function to delete, it’s perfect!”.
                      I’m currently in the process of changing it to user allocated so that the user can use smart pointers, or use their own allocator, or put stuff on the stack etc.

                      1. 1

                        Letting the library user control allocation is always better because they are in a better position to recognise how much stack is available, how long the lifetime of the allocation is, and how many foo they’re actually going to need.

                        Only if they’re going to do a better job of that than the system. Even with all that knowledge, they very commonly won’t, and certainly they will mess it up more often.

                        Having access to the internals of foo is extremely useful for tracing and debugging, and whilst it may prompt inexperienced programmers to create layer violations, I’d argue that those people might also consider hardcoding offsets within the opaque structure to get what they want.

                        Not remotely as likely in practice, and much easier to notice in a code review. Tools nudge us to behave one way or another even when they don’t make a particular route completely impossible.

                        Encapsulation for encapsulation’s sake is something “Object Oriented” programmers think is important. I think that if you want to program Java in C you might as well use Java.

                        Yes, and you almost always should. But occasionally there’s a hard requirement on C, and more frequently there is a silly policy decision that leads to using C.

                        1. 1

                          Only if they’re going to do a better job of that than the system. Even with all that knowledge, they very commonly won’t, and certainly they will mess it up more often.

                          That’s utter nonsense.

                          Of course the programmer will do a better job than “the system” if they have even the slightest idea what “the system” is doing.

                          Telling people to do stupid things like malloc() all the damn time just because Java looks like it is malloc()ing all the time is why so many people think Java is “almost as fast as C”.

                          Yes, and you almost always should.

                          You almost always should use Java or another higher level language than C: The relief in cognitive load is what is valuable here.

                          However when you are using C, then these things are important, so in C, No: you almost always should not.

                          more frequently there is a silly policy decision that leads to using C.

                          When your architect is an idiot, then your architect is an idiot: This says absolutely nothing about C.

                          1. 1

                            When your architect is an idiot, then your architect is an idiot: This says absolutely nothing about C.

                            It says that just because you’re programming in C doesn’t mean you necessarily do need to be doing careful memory management.

                            1. 1

                              Nonsense. exit is a perfectly good garbage collector for many problems.

                              1. 1

                                Um you seem to be agreeing with me?

                      2. 2

                        You can always build the first on top of the second, but you can’t build the second on top of the first.

                        Why not have both?

                        1. 2

                          Because defaults are important; because making something impossible is much more effective than avoiding it “by hand”. All programming is an exercise in deciding which differences are important enough to model and which it is better to gloss over.

                          1. 1

                            To prevent API bloat. Although people may argue for more flexibility I believe they are in fact happy that POSIX makes it as simple as:

                            int s = socket(TCP);

                        2. [Comment removed by author]

                          1. 2

                            Hire a UX Researcher to prevent engineers from bikeshedding in areas which they have no legitimate experience in.

                            1. 1

                              I’ve even had people go on and on about why you should never ever have a function that takes no other arguments than a boolean.

                              At the last place I worked we had functions like this:
                              void EnableControl();
                              void DisableControl();
                              void EnableControl(bool bEnable);
                              void DisableControl(bool bDisable);

                              I’m not even kidding.

                            2. 1

                              Sounds like if it wasn’t evil it wouldn’t be premature.

                              1. 1

                                Also, why does Donald Knuth hate root vegetables? Carrots are yummy.