1. 6

  2. 5

    When I was but a wee child I was told by a fellow colleague that UML would fundamentally change the way code was written, that it was the next revolution. I didn’t really believe him back then mostly because I hated his guts but it’s almost ten years later now and I have decent experience now, and I like to think I’m also slightly wiser.

    “I assure you this one tool will fix all our problems”, isn’t that exactly what we talk about when we talk about silver bullets? Don’t get me wrong, I’m convinced AI has a very important place in our collective future as programmers (heck, even the present for a lucky few). Experience has shown for me that it’s unlikely to be a magical problem solving thing, very much like UML was not. I’m sure history is littered with such broken promises of things that are nowadays tools we use daily.

    1. 2

      Exactly. And it’s actually easy to spot the problem right in the text where the author says that “we specify some constraints on the behavior of a desirable program”, and the “program” is then deduced by a neural network. Shifting the actual hard part — decision making — to some other stage and not consider it a part of coding still doesn’t make it go away.

    2. 2

      The article’s claim seems bold. Could you not apply this quote to Prolog?

      Instead, we specify some constraints on the behavior of a desirable program (e.g., a dataset of input output pairs of examples) and use the computational resources at our disposal to search the program space for a program that satisfies the constraints.

      1. 4

        Yeah, one of the more common criticisms of this article floating around is that it seems unaware of large parts of the idea not being new. Which is a bit odd since Karpathy is a smart and well-read guy, so maybe it just leaves out the “related work” section for punchiness and rhetorical effect. But since the whole claim is that this is a totally new way of looking at software it makes for a weird read.

        Prolog itself doesn’t do exactly that; with standard logic programming, you encode the logic directly by writing clauses, rather than giving input/output examples. But inductive logic programming is a version that does; you give it input/output examples and it induces the program’s clauses. There’s also genetic programming as a somewhat better-known set of techniques. As well as program synthesis, a more formal-methods take on it.

        The real new part is the implicit claim that, essentially, “it works now”. GP is notoriously difficult to get anything useful out of, and clearly Karpathy thinks NN-based program induction won’t suffer the same fate. But that to a large extent remains to be seen…

        1. 1

          Gonna blame tiredness for that one. The bit in parens in my quote definitely doesn’t fit Prolog! The part I’m contesting is the idea that specifying programs in terms of constraints and relying on computers to explore a program space is new.

      2. 1

        Neural networks are not just another classifier, they represent the beginning of a fundamental shift in how we write software. They are Software 2.0.

        Statements like this make me suspect that another AI bust is just around the corner.