1. 12

  2. 5

    The title is a bit extreme, boring, and clickbaity and makes me less likely to read this article.

    Funny that someone so into statistics is making such an emotional plea and threat.

    I work with a lot of statisticians and I think they could learn programming as much as progs should learn statistics.

    1. 17

      For a while Zed Shaw was known for his rants. If you saw a Zed Shaw article you knew it would be over the top ranty and if that was what you were in the mood for then it would be a great read. Getting past the rant to determine whether his content had a valuable insight was always an exercise left up to the reader.

    2. 4

      Yeah, I feel my lack of statistics knowledge sometimes leads me to the wrong conclusion (especially about performance measurements). Any crustaceans have stats programming book recommendations? Preferably Python (R is fine too) and project-driven?

      1. 2

        The end of the article lists a bunch of them.

        1. 2

          Oh yeah, but I was curious if anyone had further suggestions (particularly more modern books since the programming ones looked a bit dated)

          1. 1

            I enjoyed Statistics Done Wrong by Alex Reinhart. It’s not really a programming book but a catalog of common statistical errors, and is written in a more accessible style than your average textbook.

      2. 3

        I think a lot of this applies to the general public just as much, or even more. Things like averages have their uses, but often they’re much less useful than you’d much think; especially in things like politics and such I often see average represent a non-truth; some examples (with invented numbers):

        • “The average income is $60k” doesn’t describe that some people earn much less than $60k, and some earn much more.

        • “The average American owns 4 guns” doesn’t tell you much, as many people have 0 guns or just 1 gun, and some enthusiasts an entire shed of guns. There are many variations on this such as “the average person eats 7 pieces of fruit a week”, etc. Whenever you see “the average person {has,does,eats,thinks,..}” then it’s almost always a useless number.

        • “Doing X will increase your risk of cancer by 20%” is much less significant than it seems; if only 1/10,000 people that don’t do X get cancer, then the 50% increase means 1.2/10,000 people who do X will get cancer.

        • “You only have a 2% chance to die of COVID-19” doesn’t describe that for some people that’s much lower, and for some people that’s much higher.

        Almost every time I see “averages” being used in politics it’s almost always simplistic to the point of meaningless. I wish math education in general would focus more on getting a decent grasp of statistics and their limitations. The situation probably differs per country, but I got almost no statistics at school beyond “how to calculate an average” and such (and it certainly didn’t include an examination of the limitations of averages).

        1. 2

          Whenever you see “the average person {has,does,eats,thinks,..}” then it’s almost always a useless number.

          This insight led to the development of movable seats in cars! IIRC, the US military was trying to improve cockpit design on fighter aircraft back in the 60s. They measured a lot of pilots to figure out what “average” was so they could build their cockpit to fit the average pilot. They measured several things for each person, such as height, shoulder width, and reach. It turned out that although you could come up with averages for each measurement, and most people would be pretty close, nobody was average on everything. Each subject would have something unusual… longer legs, narrower shoulders, etc. If they built a cockpit for the average pilot it wouldn’t actually fit anyone at all.

          The solution was to support a range of sizes with an adjustable chair. This allows the system to account for expected variation in the user’s measurements. Fast forward a few decades, and you have that same tech in civilian cars.

          1. 1

            One of these is not like the others.

            But now I really want to know the median number of fruit people eat ;)

            1. 1

              One of these is not like the others.

              Hm, which one?

              Aside: a few years ago I got sent a government survey to collect (anonymous) demographic data; my country doesn’t do a census and apparently they do these kind of surveys instead. It asked a bunch of standard question like age, religion and whatnot, and also a lot of question about personal habits like how often you sport, whether you smoke, and … how many pieces of fruit you eat. The question had a big disclaimer attached to it: “One tray of grapes counts as only one piece of fruit!” I guess people filled in “50” after eating a bunch of grapes? heh. I thought that was pretty funny.

              1. 1

                Yeah, I meant exactly the fruit thing. A monster apple that’s as big as my hand or a tiny peach. Even that has like 50% difference.

                I think there are several problem with statistics. The average is the only one that seems to be 100% intuitive to most people, the rest not so much, especially the median or standard deviation (totally advanced stuff here, yeah)

                I remember Statistics in math in high school, it was… interesting, if you looked how people usually good in math were struggling with the most basic things.

                1. 1

                  A standard deviation gives you some more information, but not all that much. It mostly just tells you that the average is misleading :-) Essentially, a lot of stuff is just too complex to be boiled down to just 1 or 2 numbers, and we should probably stop trying.

                  I think for most purposes a simple chart would probably be best: it gives a reasonably good overview and is understood by my most people (there are, of course, plenty of ways to get charts wrong too, but that’s a different topic). It’s not perfect as it’s a lot more effort to create, takes up a lot more space in written content, and you can’t really verbally communicate a chart. I’m not sure of if there’s a concise way to communicate non-trivial statistics, which probably explains why averages are used so often.

          2. 3

            Considering the tone of the article, and the theme running through his war stories, I don’t think learning statistics would have fixed anything here. Sounds more like bad attitudes in resonance. Subtract the bad attitudes, and the statistical aspects could be explained in minutes to anyone with a firm grasp on grade-school arithmetic and common sense.

            1. 2

              Besides the obvious (and obnoxious) non-PC rantiness, this article does lay out some decent points. I can say I learned something.

              …then you’ll never get that massive 300k flash animation to your users in time to sell them your latest Gizmodo 9000

              Laughs in modern webdev