1. 32
  1.  

  2. 20

    I didn’t read all the summaries in depth, but after reading the summary of summaries it sounds roughly like what I suspected: none of these show any particularly strong results, and most of the research is problematic due to data inconsistencies, methodology issues, faulty assumptions, etc. etc.

    At this point after spending a lot of time on this topic including reading seemingly endless arguments by advocates of various perspectives, I’ve come to believe that

    • statically-typed languages allow programmers to eliminate certain classes of errors very easily compared to dynamically-typed languages, at a cost. I do not believe this is a controversial point.
    • the costs and challenges involved in using statically-typed languages are non-obvious for both inexperienced programmers, who often have mistaken assumptions about what makes a statically-typed language frustrating or costly to use (e.g. “annotating my types must create tons of boilerplate” or “if I’m going to use a statically-typed language it should be able to infer all my types”), as well as experienced programmers, who seem to have forgotten the learning curve they went through or are otherwise blinded to the tradeoffs (“well, you can simulate that aspect of dynamic typing with a GADT so what’s the problem?”).
    • there is a complex interplay between programmer experience, programmer experience with a specific language, the specific subset of features the language provides outside of static typing, the affordances of the type system itself (Haskell’s type system is very different from Java’s), and expected RoI for a given software project. I think this blog post illustrates that quite well.

    Looking at this specifically as an industry programmer, I find it hard to debate the notion that dynamically-typed languages offer a lower barrier to getting useful code out the door, at the cost of less reliability and “lower fidelity,” in relation to the amount of time spent. I also strongly suspect that the effort taken to ensure higher quality and reliability with a dynamically-typed language past a certain point–in the dimensions that statically-typed languages help with–is an order of magnitude greater when compared to a statically-typed language. That aside, it’s not clear to me that using a model-checker like Alloy or TLA+–or, for example, simply adding more rigorous code-review practices to your organization’s processes–wouldn’t end up providing greater benefit than using a statically-typed language. But as far as I’ve been able to tell, nothing I’ve just written in this paragraph is confirmed or will be any time soon (although I’d like to be proven wrong if it means that we engage in pointless arguments on this topic less).

    …which leads me to my final conclusion, underscored by this post, which is that most of the time these conversations are little more than a tempest in a teapot, and lead to very little useful information. Fundamentally there doesn’t seem to be much difference in using a statically-typed language vs. a dynamically-typed language as far as creating a viable business is concerned. And no one ever got fired for choosing Java.

    Knowing all of that, I’d love to see us shift our focus in these conversations to talking explicitly about features of individual languages and what they provide, and how they help in specific contexts. I’d like to see more discussion, exploration, and sharing of knowledge, vs. the contentious spewing of anecdata-driven arguments that seems to be the norm. I’d like to see everyone acknowledge their ignorance and assumptions and avoid identifying so strongly as a member of a given community advocating a certain position. Frankly aligning in tribes along the lines of language communities or as static-typing advocates or the like has always seemed rather ridiculous and beside the point to me, and to a lot of other programmers I know, who, wisely, simply avoid these conversations altogether.

    A postscript: I spend a lot of time hanging out with the Haskell community in various contexts and the vast majority of the time I hear Haskell programmers say “if it compile, it works” it’s tongue-in-cheek. So I’m always a bit disappointed to see that one get trotted out in these discussions.

    1. 6

      the cost and challenges involved in using statically-typed languages are non-obvious for both inexperienced programmers, who often have mistaken assumptions about what makes a statically-typed language frustrating or costly to use (e.g. “annotating my types must create tons of boilerplate” or “if I’m going to use a statically-typed language it should be able to infer all my types”), as well as experienced programmers, who seem to have forgotten the learning curve they went through or are otherwise blinded to the tradeoffs (“well, you can simulate that aspect of dynamic typing with a GADT so what’s the problem?”).

      If “inexperienced” programmers are able to write correct code, then the learning curve to static types is small because you already write code that is amenable to annotation, you just need to figure out how to map your intuition into formal annotations.

      The largest obstacle (IMO) for many inexperienced dynamic language users is that they write lots of bugs that a compiler makes them deal with immediately. If you care very little about quality (which might be valid e.g., for prototyping but often is simply immaturity), this is a frustrating experience because you can’t defer the problem and move onto the next ticket. If you appreciate quality, the experience is mostly gratifying.

      1. 18

        If “inexperienced” programmers are able to write correct code, then the learning curve to static types is small because you already write code that is amenable to annotation, you just need to figure out how to map your intuition into formal annotations.

        It’s also the idiosyncrasies of the specific type system. Each type system makes design tradeoffs which might not mesh with your actual problem representation. If your “intuition” needs row polymorphism to formalize and you’re using Haskell then you’re SOL. You have to find some other way of implementing that diverges from the abstract model, and that diversion can itself be a source of bugs.

        (You could use a language with row polymorphism, but then you’ve just shifted your problem over. What if a different part of the abstract model is incompatible with the new type system? And then there’s the space of things where we don’t have any production language type systems at all! That’d be things like non-monotonic types.)

        The largest obstacle (IMO) for many inexperienced dynamic language users is that they write lots of bugs that a compiler makes them deal with immediately. If you care very little about quality (which might be valid e.g., for prototyping but often is simply immaturity), this is a frustrating experience because you can’t defer the problem and move onto the next ticket. If you appreciate quality, the experience is mostly gratifying.

        First, it’s pretty rude to reduce the entire “static vs dynamic types” design space into “do you care about quality or not.”

        Second, the entire point of the article is that we haven’t actually found a link between type systems and quality. People keep claiming there is one but it never quite holds up under scientific scrutiny. If you actually appreciate quality, you’d do the things that have actual, empirical evidence they reduce bugs. That’d be things like

        • Code review
        • Being fully rested
        • Eating healthy and exercising
        • 40 hour (or fewer) workweeks
        • Cutting scope or delaying projects instead of crunch-timing them
        • A psychologically-safe, low-stress work environment

        These all matter a lot more for quality than what language you use. I’d pick the person who has these and writes clojure over the person who has none of these and writes haskell any day.

        (Of course, a lot of these aren’t individually controllable by the engineer, and are whole-system properties. In many cases, though, so is your programming language.)

        1. 1

          First, it’s pretty rude to reduce the entire “static vs dynamic types” design space into “do you care about quality or not.”

          This is a straw man. I didn’t claim or even imply that this is the entire design space. I merely mentioned that a given individual’s concern for quality impacts their perception of static types systems; I even noted that it’s often appropriate to eschew quality (e.g., prototyping) in some cases (although I’m oversimplifying in that ‘quality’ itself is not a binary proposition, etc). You can disagree with that premise if you like, but please don’t put words in my mouth.

          Second, the entire point of the article is that we haven’t actually found a link between type systems and quality. People keep claiming there is one but it never quite holds up under scientific scrutiny.

          Scientific scrutiny has hardly been applied; not that I can fault anyone–software is a hugely multivariate problem space and I can’t conceive of an economically feasible way to control for the majority of variables. I would be enormously skeptical of anyone claiming to have scientifically conclusive evidence.

          It’s also the idiosyncrasies of the specific type system. Each type system makes design tradeoffs which might not mesh with your actual problem representation. If your “intuition” needs row polymorphism to formalize and you’re using Haskell then you’re SOL. You have to find some other way of implementing that diverges from the abstract model, and that diversion can itself be a source of bugs.

          These are good points; I’ll have to mull these over.

          1. 1

            [edit: redacting first half because I think it was a bit too unkind, will try to find a better way to convey]

            Scientific scrutiny has hardly been applied; not that I can fault anyone–software is a hugely multivariate problem space and I can’t conceive of an economically feasible way to control for the majority of variables. I would be enormously skeptical of anyone claiming to have scientifically conclusive evidence.

            This is also explicitly discussed in the article.

            Other than cherry picking studies to confirm a long-held position, the most common response I’ve heard to these sorts of studies is that the effect isn’t quantifiable by a controlled experiment. However, I’ve yet to hear a specific reason that doesn’t also apply to any other field that empirically measures human behavior. Compared to a lot of those fields, it’s easy to run controlled experiments or do empirical studies. It’s true that controlled studies only tell you something about a very limited set of circumstances, but the fix to that isn’t to dismiss them, but to fund more studies. It’s also true that it’s tough to determine causation from ex-post empirical studies, but the solution isn’t to ignore the data, but to do more sophisticated analysis. For example, econometric methods are often able to make a case for causation with data that’s messier than the data we’ve looked at here.

            1. 1

              This is also explicitly discussed in the article.

              Me: Not enough research to conclude one way or the other

              The article: the fix to that isn’t to dismiss them, but to fund more studies.

              These sound pretty compatible to me. Specifically, just because I don’t accept the studies as gospel doesn’t mean I dismiss them.

              However, I’ve yet to hear a specific reason that doesn’t also apply to any other field that empirically measures human behavior

              It’s the same insofar as those other fields are plagued with problems that ultimately prevent them from delivering tidy (and accurate) conclusions; it’s different in that the static-vs-dynamic question receives tremendously less attention than, e.g., questions of racism or sexism (and rightly so IMO).

        2. 2

          I think your second paragraph makes a lot of sense, but I take issue with this line in your first paragraph (emphasis mine):

          then the learning curve to static types is small because you already write code that is amenable to annotation, you just need to figure out how to map your intuition into formal annotations.

          …easier said than done, I think:

          as well as experienced programmers, who seem to have forgotten the learning curve they went through

          And let’s also be clear that we’re probably not talking about C++ here.

          EDIT: after re-reading your second paragraph I want to clarify–I don’t necessarily think that a programmer wanting to defer dealing with type-checking errors means that they are immature or don’t appreciate quality, but if this hypothetical programmer is experienced with using dynamically-typed languages, dealing with the type-checker is not a natural thing, and having to defer implementing something until you address opaque (even if due to a lack of experience) type errors can be frustrating until you’ve gotten to a certain level of facility with a type-checker. I just wanted to highlight that because “If you appreciate quality, the experience is mostly gratifying.” makes it sound as though if you don’t find dealing with a type-checker gratifying even as someone inexperienced with statically-typed languages, you necessarily don’t appreciate quality, but I don’t think this is the case.

          1. 0

            easier said than done, I think

            I don’t think it is that hard—it is dwarfed in comparison to learning to program correctly (when you program well, your code is inherently type-correct—you just need to annotate); however I think it’s easy to misattribute the difficulty of learning to write correct code to the task of learning to map intuition into type annotations. You can generally identify this misattribution when the type checker requires you to rewrite your code as opposed to simply annotating it. Of course, there are other confounding factors at play as well, such as the quality of implementation of these type systems (including error messaging), with (especially older versions of) C++ and Java making the task far more difficult than necessary.

            1. 4

              I don’t think it is that hard—it is dwarfed in comparison to learning to program correctly (when you program well, your code is inherently type-correct—you just need to annotate); however I think it’s easy to misattribute the difficulty of learning to write correct code to the task of learning to map intuition into type annotations. You can generally identify this misattribution when the type checker requires you to rewrite your code as opposed to simply annotating it.

              Again I don’t fundamentally disagree with the point you’re making, it just seems to me like you’re downplaying what a barrier developing that intuition can be. And perversely, I suspect this barrier is much higher for experienced programmers who have only ever used dynamically-typed languages.

              Personally it was a huge frustration when I was first learning Haskell to understand how to use the correct numeric types to do basic stuff like division. Now it seems trivial and obvious. The real question is why I continued to bang my head against the type-checker until I developed that intuition, but others did not. I am not arrogant or foolish enough to believe it’s because I have a greater appreciation for correctness–one of the best and most correctness-focused programmers I know gave up on Haskell because he found the learning curve too challenging–so what is it? Maybe Haskell’s ergonomics simply suck (there is some truth here)? Or maybe it’s really hard to see the value until you’ve actually developed the intuition, and some of us are more masochistic than others? Maybe I had faith that eventually it would pay dividends when he did not? Or maybe he evaluated the cost/benefit of spending time learning it and very appropriately decided it wasn’t worth it (where I just decided to blunder along until I figured it out, without thinking)? I really don’t know.

              1. 4

                maybe it’s really hard to see the value

                I think this is it. Programmers don’t see themselves as the kinds of people who ‘can make type errors’, and if they do, it’s just because it was a silly mistake, not the status quo. And so when a typechecker blocks them from doing something they think it shouldn’t have, there’s a lot of frustration. I’ve succumbed to this myself!

                Contrast this with people using Rust who absolutely adore its type errors because they know from hard experience they can’t do manual memory management and concurrency correctly without help.

                1. 2

                  100% agree, and great insight into why novices in Rust’s flavor of static typing may find it more palatable up front. I’ve never considered that.

        3. 2

          statically-typed languages allow programmers to eliminate certain classes of errors very easily compared to dynamically-typed languages, at a cost. I do not believe this is a controversial point.

          The controversy is probably in the details, i.e. how much do statically-typed languages help, and how large the cost is in specific languages. Now we have languages like Nim, which are quite static in their typing but where the coding experience is not very far from a dynamic language.

          1. 4

            This is a great point. I feel like a major theme of the current newer generation of industry languages is about trying to lower the barriers to using statically-typed languages, even if it’s using wildly different techniques or philosophies, whether we’re talking about Go or Kotlin or Nim or Rust…etc. And this is also one reason I’m bullish on statically-typed languages–I suspect through a combination of the concepts and language of type systems becoming more and more widespread in the industry, along with increased effort to lower the barrier of entry of using sophisticated type systems, we are going to end up somewhere in the future that is very much like using dynamically-typed languages but with the guarantees of static type systems.

          2. 1

            From the perspective of someone who generally prefers dynamically-typed languages, but reads and dabbles i a lot of stuff, I see a continuum of things people mean when they say “statically typed”.

            So, consider a function named add. At one end of the continuum there are static type systems which just verify that any time add is called, the arguments are numeric, and that the return value is of the same type. Call this the Java end of the continuum. At the other end there are static type systems which either verify, or people are working to have verify, that any time add is called, it returns a value of type sum of the given arguments. Call this the theorem-prover end of the continuum.

            In languages at or near the Java end, because the static type checks don’t verify all that much, you end up writing test suites, and it often feels as if the type check is redundant: if the test suite is doing a good job of exercising the codebase, it’s already going to catch the “you tried to add an int to a string” case, and if the test suite isn’t doing a good job of exercising the codebase, the type checks might well catch some things for you from time to time, but it’s still likely that you’re shipping low-quality/buggy code and the type checks aren’t doing all that much to protect you from yourself.

            In languages at or near the theorem-prover end, ideally you never have to write tests because the goal here is “if it compiles, it is correct”. But out at that end of the continuum, the amount of effort, and cognitive load, you have to take on in order to get those benefits, or even to get close to them, is quite large and seems to eventually grow without bound, while the classes of bugs you’re catching get smaller and less important at the same time, and it probably would have been a more efficient use of engineering effort to pick a language with less type-system load and just write a test suite.

            Which is not to say there aren’t interesting things you can do or automate via type checks, or that there aren’t interesting or useful ideas that can be disentangled from the statically-typed languages they get developed in. And is not to say that I wouldn’t ever use static checks; most of my day-to-day work is in Python, and now that Python 2 is finally EOL I’ve been gradually sprinkling annotations into my projects and adding mypy to my CI runs. I also have full test suites, and I treat this as a belt-and-suspenders approach. But I haven’t made up my mind whether to stick to it long-term, and probably won’t be able to for a while.

            It’s just to say that it feels like most approaches to static typing are either too little or too much, and I haven’t seen anything really nail a happy medium.

            (also, for the theorem-prover end of the continuum, I tend to be highly skeptical in general, and wish I could sit those folks down with the history of the logical positivists and point out they’re steering right for the same shoals the positivists eventually grounded on)

          3. 15

            This post changed my career path. It showed me just how important aggregating, curating, and technical writing are to the health of our industry. I did a little writing before this, but it was mostly either programming tricks or opinions. afterwards I focused much more on in-depth research pieces. It was the right choice.

            One interesting paper this doesn’t cover is To type or not to type, where the researchers caught errors by adding type annotations to existing Javascript code. I haven’t dug deep on it yet, but it’s the most intriguing evidence in favor of type systems directly improving quality.

            1. 3

              One interesting paper this doesn’t cover is To type or not to type, where the researchers caught errors by adding type annotations to existing Javascript code. I haven’t dug deep on it yet, but it’s the most intriguing evidence in favor of type systems directly improving quality.

              That’s pretty uninteresting, IMO, if they took existing code, added type annotations to it (i.e. spent many hours annotating it, looking at it, analysing it, studying it) and then compared the resulting code with the previously existing code.

              Comparing the result of adding types to the result of another group of equally skilled and knowledgeable people simply analysing and studying the code for bugs in some other way for the same amount of time with the same amount of effort.

              Obviously if you spend 200 hours adding type annotations to code you’ll improve its quality, because if you spend 200 hours with any codebase analysing and studying it and looking for bugs you’ll improve its quality.

              1. 6

                That’s not what they did; read the paper.

            2. 9

              One thing people seem to often overlook when discussing the merits of typing when writing code is that programmers usually spend much more time reading other people’s code than writing their own. When I had to read a large existing Python codebase a couple of years ago, the hardest thing about it was that there were no types to tell me what input arguments or return values were supposed to be.

              1. 7

                Working in Python is quite fun for the initial spurt of development, but a year later it’s hard to tell exactly what’s expected and where.

              2. 2

                I’m skeptical whether static vs dynamic matters much at all in real life.

                From what I’ve seen, a person’s experience with the language they’re using is far more important than the exact details of the language itself. Given enough time, people adjust to the strengths and weaknesses of whichever language they’re using.

                1. 2

                  Given enough time,

                  How much time can you afford to spend?

                  1. 1

                    I suppose I worded it poorly, but the amount of time is besides the point.

                    What matters most is experience with and understanding of the tools and language being used, and there are no shortcuts for getting that experience, other than using the tools.