Saying “type systems improve productivity” is worthless without specifying more about the type system you’re proposing. Is it like Standard Pascal’s, where an array’s length is part of its type, making it impossible to write functions which handle arrays of unknown length? Is it like C’s, where most types are size specifications and have no information about semantics?
We have to be very careful about equivocation here, and not mix unlike concepts.
Also, in Ada, you specify types plus their internal representation. The common example from a rocket disaster is having miles and kilometers as types for distance. They’ll be the same internally but compiler will complain if you mix them.
I don’t understand the sentence: “In two of the four cases, there’s an obvious positive correlation between perceived skill and actual skill, which is the opposite of the pop-sci conception of Dunning-Kruger.” I’m not seeing such correlation on any of the graphs, as I’m interpreting the “actual skill” to match “Actual test score” on the graph. Could someone shine some light on what author meant by this?
I never understood Dunning-Kruger to mean that the competent rate themselves as less skilled than the incompetent, just that the incompetent tend to overrate their own ability due to lack of a fuller understanding of what they don’t know, while the competent tend to be more aware of their weaknesses. That’s exactly what the graphs show to me.
The first and fourth graphs show the same trend. Perceptual is a compressed version of actual, but the slope is trending upward. Low achievers rate themselves lower than high achievers.
This is in contrast to the popsci version of “experts think they’re idiots; idiots think they’re experts.” That would show a negative slope.
I’m not familiar with the studies about type systems, but I feel pretty confident many large projects I’ve worked on would fall apart if they were dynamically typed. So in the spirit of the article, what do those studies actually say? In particular, what do they say about projects of different code sizes? Are there even enough multi-million line dynamic language projects to analyze?
The article quite literally links to a review of studies made. I can’t help but notice that you kind of prove their complaint: people not even doing the simplest review of linked sources.
Yes, it’s true, I didn’t review the linked sources. Because they are long and I don’t quite care enough. I’m just curious enough to ask, since it’s likely someone here would already know and have an answer slightly more concise than several pages. :)
The problem with comparing static and dynamic languages is that that variable is hard to isolate between all the other differences between individual languages that have a static type system and individual languages that have a dynamic type system. I’ve seen danluu’s analysis before, and not been particularly impressed that any of them were really answering the question of “all else being equal, is static typing better than not?”, as opposed to showing that you can write code with fewer bugs in Haskell specifically than with Python specifically, or that Java’s type system doesn’t gain you all that much, or some result along those lines. Not all type systems are equally powerful or ergonomic in any case. I’m a big fan of strong typing myself, from my own intuitional experience of maintaining large codebases written in dynamically-typed languages and debugging things that a type-checker would’ve caught, but I think that has as much to do with the fact that a specific set of popular strongly-typed languages encourage a different mindset around creating code compared with a specific set of popular dynamically-typed languages, than about the raw fact of having types or not.
The fact that it’s hard to analyze is itself evidence that static typing does not play a dominant role. If the type system was the defining feature of the language it would dominate other factors, in practice it clearly does not.
I’m not talking about disregarding anything. To put it another way, we have to observe a statistical difference for a language or a group of languages compared to others. For example, if projects written in Haskell were statistically of higher quality than those written in other languages, we could make a hypothesis that Haskell type system has a positive impact on quality and test that hypothesis.
This is a much different statement than the first one I responded to. Your first statement was that “X being hard to measure is evidence X isn’t a big factor”. But that is clearly wrong. It means you cannot say much about X, not evidence for one particular interpretation of it.
I’m saying that you have to show that statistical trends exist before you can start having discussion about different factors and their potential impact.
I’m not changing anything. If a particular factor had a significant contribution that would be an invariant in all languages that have this factor. I think at this point you know exactly what I’m saying, and you’re just playing word games with me.
But that is only true if what one is measuring is well-defined, which it doesn’t appear to be, right? It feels like you’re saying the absence of evidence is evidence of absence, which is clearly false.
I’m not playing word game with you, I’m being precise in order to understand what you are saying. Based on what you’ve said, it seems like one would have to conclude that that one cannot say one way or the other the effect of the type system on a project rather than that it has no effect. Isn’t this like the difference between not-rejecting the null hypothesis vs accepting the null hypothesis?
Once again, what I’m saying is that you have to demonstrate that there is a statistical difference before you can discuss the cause of that difference. What part of that statement are you having trouble with?
That statement is perfectly reasonable, but that is not the first statement you made where you claimed that the mere fact that something is difficult to measure is evidence of a particular conclusion. Did I misinterpret what you said? The exact quote is:
The fact that it’s hard to analyze is itself evidence that static typing does not play a dominant role.
That directly follows from the fact that statistical differences have not been demonstrated. If a particular language is shown to be a statistical outlier, then you could reasonably say that some feature or combination of thereof are responsible for that. Since we are not seeing such outliers, that strongly implies that the choice of language is not a dominant factor.
It’s entirely possible that things like developer skill, development process, and testing practices eclipse any effect a language might have.
I’m not familiar with the studies about type systems, but I feel pretty confident many large projects I’ve worked on would fall apart if they were dynamically typed.
Sure. And ancient Greek scholars felt pretty confident that rubbing garlic on a magnet would demagnetize it.
We need to start by recognizing that a personal feeling of confidence is an absolutely terrible basis from which to derive objective facts about the world.
Fall apart in what sense? I worked for 15 years on a codebase that was maintained by about 500 engineers. 90% of it was written in TCL.
You might argue the codebase would have been better if it had been typed, but we continued to make money just fine, and despite competitors (internal and external) trying to rewrite or replace it, they couldn’t. It didn’t ‘fall apart’.
Saying “type systems improve productivity” is worthless without specifying more about the type system you’re proposing. Is it like Standard Pascal’s, where an array’s length is part of its type, making it impossible to write functions which handle arrays of unknown length? Is it like C’s, where most types are size specifications and have no information about semantics?
We have to be very careful about equivocation here, and not mix unlike concepts.
You can have both if your language allows parametricity over the length. Most dependently typed languages don’t have any problem with this.
Also, in Ada, you specify types plus their internal representation. The common example from a rocket disaster is having miles and kilometers as types for distance. They’ll be the same internally but compiler will complain if you mix them.
My favorite thing about Dunning-Kruger is how many people who cite it in an argument get it wrong. They’ve been Dunning-Krugered about Dunning-Kruger.
I don’t understand the sentence: “In two of the four cases, there’s an obvious positive correlation between perceived skill and actual skill, which is the opposite of the pop-sci conception of Dunning-Kruger.” I’m not seeing such correlation on any of the graphs, as I’m interpreting the “actual skill” to match “Actual test score” on the graph. Could someone shine some light on what author meant by this?
I never understood Dunning-Kruger to mean that the competent rate themselves as less skilled than the incompetent, just that the incompetent tend to overrate their own ability due to lack of a fuller understanding of what they don’t know, while the competent tend to be more aware of their weaknesses. That’s exactly what the graphs show to me.
The first and fourth graphs show the same trend. Perceptual is a compressed version of actual, but the slope is trending upward. Low achievers rate themselves lower than high achievers.
This is in contrast to the popsci version of “experts think they’re idiots; idiots think they’re experts.” That would show a negative slope.
I’m not familiar with the studies about type systems, but I feel pretty confident many large projects I’ve worked on would fall apart if they were dynamically typed. So in the spirit of the article, what do those studies actually say? In particular, what do they say about projects of different code sizes? Are there even enough multi-million line dynamic language projects to analyze?
The article quite literally links to a review of studies made. I can’t help but notice that you kind of prove their complaint: people not even doing the simplest review of linked sources.
https://danluu.com/empirical-pl/
Yes, it’s true, I didn’t review the linked sources. Because they are long and I don’t quite care enough. I’m just curious enough to ask, since it’s likely someone here would already know and have an answer slightly more concise than several pages. :)
The problem with comparing static and dynamic languages is that that variable is hard to isolate between all the other differences between individual languages that have a static type system and individual languages that have a dynamic type system. I’ve seen danluu’s analysis before, and not been particularly impressed that any of them were really answering the question of “all else being equal, is static typing better than not?”, as opposed to showing that you can write code with fewer bugs in Haskell specifically than with Python specifically, or that Java’s type system doesn’t gain you all that much, or some result along those lines. Not all type systems are equally powerful or ergonomic in any case. I’m a big fan of strong typing myself, from my own intuitional experience of maintaining large codebases written in dynamically-typed languages and debugging things that a type-checker would’ve caught, but I think that has as much to do with the fact that a specific set of popular strongly-typed languages encourage a different mindset around creating code compared with a specific set of popular dynamically-typed languages, than about the raw fact of having types or not.
The fact that it’s hard to analyze is itself evidence that static typing does not play a dominant role. If the type system was the defining feature of the language it would dominate other factors, in practice it clearly does not.
That may not be a valid way to look at the problem. See McNamara Fallacy:
https://en.wikipedia.org/wiki/McNamara_fallacy
But I might not be completely understanding what you’re saying.
I’m not talking about disregarding anything. To put it another way, we have to observe a statistical difference for a language or a group of languages compared to others. For example, if projects written in Haskell were statistically of higher quality than those written in other languages, we could make a hypothesis that Haskell type system has a positive impact on quality and test that hypothesis.
This is a much different statement than the first one I responded to. Your first statement was that “X being hard to measure is evidence X isn’t a big factor”. But that is clearly wrong. It means you cannot say much about X, not evidence for one particular interpretation of it.
That was not my statement. The statement was that X does not appear to play a more significant role than other factors.
Does that mean you do know how to measure the factors?
I’m saying that you have to show that statistical trends exist before you can start having discussion about different factors and their potential impact.
Does that mean you’re changing your original statement? Your original statement was:
Being hard to analyze is not evidence for a particular interpretation other than the interpretation that it’s hard to analyze.
I’m not changing anything. If a particular factor had a significant contribution that would be an invariant in all languages that have this factor. I think at this point you know exactly what I’m saying, and you’re just playing word games with me.
But that is only true if what one is measuring is well-defined, which it doesn’t appear to be, right? It feels like you’re saying the absence of evidence is evidence of absence, which is clearly false.
I’m not playing word game with you, I’m being precise in order to understand what you are saying. Based on what you’ve said, it seems like one would have to conclude that that one cannot say one way or the other the effect of the type system on a project rather than that it has no effect. Isn’t this like the difference between not-rejecting the null hypothesis vs accepting the null hypothesis?
Once again, what I’m saying is that you have to demonstrate that there is a statistical difference before you can discuss the cause of that difference. What part of that statement are you having trouble with?
That statement is perfectly reasonable, but that is not the first statement you made where you claimed that the mere fact that something is difficult to measure is evidence of a particular conclusion. Did I misinterpret what you said? The exact quote is:
That directly follows from the fact that statistical differences have not been demonstrated. If a particular language is shown to be a statistical outlier, then you could reasonably say that some feature or combination of thereof are responsible for that. Since we are not seeing such outliers, that strongly implies that the choice of language is not a dominant factor.
It’s entirely possible that things like developer skill, development process, and testing practices eclipse any effect a language might have.
Sure. And ancient Greek scholars felt pretty confident that rubbing garlic on a magnet would demagnetize it.
We need to start by recognizing that a personal feeling of confidence is an absolutely terrible basis from which to derive objective facts about the world.
Fall apart in what sense? I worked for 15 years on a codebase that was maintained by about 500 engineers. 90% of it was written in TCL.
You might argue the codebase would have been better if it had been typed, but we continued to make money just fine, and despite competitors (internal and external) trying to rewrite or replace it, they couldn’t. It didn’t ‘fall apart’.