1. 9

  2. 8

    Nice article!

    There are now tens of thousands of researchers (at a low-end estimate) taking the basic idea of machine learning and either applying small tweaks to it, or finding new problem domains to apply it to. It can seem like there’s an almost infinite set of small problems to work on in machine learning, which means that no-one needs to do too much thinking before choosing one and charging forwards. It’s easy to dismiss the individual value of most of this work as very low — yet, collectively, it’s clearly pushing this field forwards.

    I definitely feel this … I happened to work on a machine learning paper in 2016, and recently discovered that it has thousands of citations. I’m not sure it’s particularly valuable (although I can’t say it’s not either!).

    I was recruited to work on it because of my skills in Python, R, and using big clusters, not because I’m interested in ML.

    I might be slightly less charitable about it than you … I would say there are literally thousands of “shitty machine learning intern projects” wasting compute power that have been going on for the last ~6 years.

    I know this because I worked on a shitty machine learning research project as an undergrad over 20 years ago, at Xerox PARC! This cured me of my interest in ML.

    And I worked adjacent to many such projects in 2016.

    (Funny thing is that I heard Richard Hipp of sqlite said he worked on NLP as a grad student in the 90’s. He basically thinks it’s BS now, and said as much on a couple occasions.)

    Here’s another example I sent my mom:


    This paper evaluated ** 415 ** different machine learning models for detecting COVID from images, and none of them really worked. That’s a crazy amount of resources that amounts to very little IMO.

    I also agree with this section:

    For example, an astonishing number of people I’ve come across tout WebAssembly as a solution to their problems (and a wide variety of problems at that!). I see it as a neat solution to a niche problem (running C/C++-ish code in a browser) but I am unable to see it as a general solution to other problems. I hope, however, that other people are right and I’m wrong!

    And maybe I’ll be less charitable than you again. What I’m seeing with WebAssembly is similar to what people said about the JVM.

    There is the famous quote that Java would turn Windows into a collection of poorly debugged device drivers or something. Java wanted to be an OS.

    Likewise I see this same “inner platform effect” tendency with WebAssembly enthusiasts – they want it to be an OS.

    But the JVM ended up as “just another Unix process” (largely), and I think WebAssembly will have a similar role. It’s important and useful, but it’s not the foundation of all computing.

    Likewise I think the current branch of ML is a dead end, with too many researchers gathering around it. But of course we will see many other improvements / paradigms in ML in the future, and they will be important.

    I think it’s just a manifestation of the fact most research amounts to very little, and most impact is achieved by the rarest kind of work.

    Another clear example from COVID was that Katalin Kariko was essentially pushed down and out of U Penn in the 90’s and early 2000’s. She was one the main inventors of the mRNA vaccines, and probably deserves a Nobel Prize!


    I guess I wrote this to remind myself and others: following trends can be a waste of time, and ultimately not very impactful ! But trends usually have a core kernel of promise, so distinguishing what’s worthwhile and what’s not is hard.