1. 15

I call Lisp’s biggest shortcoming the mysterious tuple problem. It stems (from) Lisp’s idiomatic over-use of lists as product types, which are more commonly called tuples. In this essay, I’ll explain the problem, how more popular languages don’t suffer from it, and some ways to have the power of Lisp without it.

  1.  

  2. 18

    Hmm, the key argument here seems to be that idiomatic Lisp uses untagged general-purpose datatypes (like lists or tuples) more than idiomatic usage of other dynamic languages like Python or Ruby does. That doesn’t mesh with my own experience. My experience of Common Lisp, at least, is that people usually use a lot of defstruct and defclass (this may not be true in elisp or Scheme). My experience with Python, meanwhile, has been that it’s extremely common to just use tuples/lists/dictionaries for absolutely everything (or in numerical code, NumPy arrays), and not to define classes, especially in user-level code. But neither I nor this article have any actual data on what is typically used in either language, so it’s kind of duelling anecdotes.

    1. 4

      In Common Lisp (CL), especially at the begin of a project in the exploratory phase I use (and might even overuse) property lists (plists) a lot in favor of structs and classes because they’re so convenient: they’re easy to change and very clear on the REPL, during debugging and printing.

      Here is what a plist looks like CL both in the code (input) as in the output:

      (:id 123 :name "Herman Zersteurer" :balance 456.789)
      

      One can access or modify elements with GETF:

      > (getf plist-variable :name)
      "Herman Zersteurer"
      
    2. 11

      Look up “multiple values” in the Hyperspec. Mind. Blown :)

      Also, no realistic Common Lisp code would use lists for storing vectors. There is actually a VECTOR datatype..

      EDIT: you certainly have a point though when speaking of some simplest Lisps, but it hasn’t been state of the art from 1970s.

      1. 4

        I still see this every once in a while. Can be rephrased as, “makes prototyping too easy”. Which can be a strength rather than a weakness provided you eventually create data structures.

        It’s no longer restricted to Lisp these days. I use arrays all the time when prototyping Python.

      2. 9

        Not sure I agree with the article. Sure, I’ve seen that kind of code before and it is mildly annoying, but most of the code I’ve seen uses proper structs or classes. It seems that people who like LISP always try to find “problems” in the language that make people not want to adopt it.

        I actually think that the problem with LISP (i.e., Common Lisp) is that it is a language that stopped in time. As examples:

        • There’s no reasonable package manager (I know of quicklisp, but it is not nearly as convenient as Rust’s Cargo, for example).
        • No friendly language documentation. Hyperspec is complete, but it is in no way pleasant to use. I know there are efforts to improve it, but it would have to get at least close to Go/Rust documentation quality to attract people. Lack of examples is also a deal breaker.
        • No centralised community. Pretty much everybody ends up reinventing the wheel, possibly due to the lack of a good package manager and docs.

        Clojure is arguably successful, but it comes with the “cost” of being bound to the Java VM – which pretty much requires a PhD to properly tune.

        1. 2

          Hyperspec is complete, but it is in no way pleasant to use.

          Funny, I find myself missing the Hyperspec every single day (I now write mostly Clojure). It remains the standard against which I judge most other documentation. (And I’m not sure what you mean when you say it doesn’t have examples; the Hyperspec page for defstruct has over a dozen separate examples of different uses.)

          Yes, it’s unfortunate that there is no realistic way to change or grow the language, but as far as documenting what does exist in the standard the Hyperspec does a great job.

        2. 8

          As other lispers have already mentioned, the code in the article is not idiomatic. It is not common to use lists as product types. Nor is it common to use structs. It is common to use classes. It has been so for a long time. I’m wondering what code the author has read? The MIT AI code from the 90’s? And even there, there is not much code that uses lists how they suggest.

          But worse, even if one wanted to use lists as a record, one can define ‘setf’able accessors and a constructor to improve legibility. And if the code is going to be more than 20 lines one would expect people to do so.

          One can write unreadable code in any language.

          Btw, regarding exploratory/prototyping, Classes in lisp can be redefined, and redefining them updates the classes of all the instances of that class that are in the running image. So little flexibility is gained by using lists instead of classes.

          1. 1

            One can write unreadable code in any language.

            I think that there’s an argument about difficulty to write unreadable code though.

            For example, in Javascript, you end up working with dictionaries most of the time. Dictionaries have the advantage (compared to lists/tuples) of giving names to all the data within. It would be a bit surprising to see JS code using lists in the same way that Python uses tuples, for examples (Python tuples can take advantage of destructuring)

            This is a bit trite, of course. There’s definitely messy JS.

            If anything this discussion has made me more curious about what idiomatic lips (common lisp mainly) looks like. Lots of people seem to have seen it

            1. 4

              I think that there’s an argument about difficulty to write unreadable code though.

              Yes, though not in that article.

              For example, in Javascript, you end up working with dictionaries most of the time. Dictionaries have the advantage (compared to lists/tuples) of giving names to all the data within.

              In lisp we cover that space with alists as /u/jlarocco hinted at when they said It is common to use a cons as a pair (because that’s what a cons is). Alists are a list of pairs. Which, if you are writing out the items of the dictionary by hand, most likely performs better than a hash-table(good dev UX). But more importantly, one can decide at run-time which function to use to retrieve, among other things allows you to almost keep your alist sorted by MRU w/o allocating additional memory.

              If anything this discussion has made me more curious about what idiomatic lips (common lisp mainly) looks like. Lots of people seem to have seen it

              That is hard to come by because as /u/glutenfreebytes said the community is more like an archipelago, so everyone has their own variations (some people prefer to use structs a lot f/e). Also idiomatic lisp has some archaisms like the -p suffix convention instead of using ?. With that said, two lispers that write straight forward lisp code imho would be are ruricolist and orthecreedence. Check out the repos below

              Coleslaw is a static site generate designed to be updated with git, it is also written in idiomatic Lisp.

              And cl-6502, the project that brought me to Lisp is also fairly readable.

              1. 2

                Thank you for this detailed reply! Seeing some of this code is really interesting. Lots of these projects feel like an alternate universe into computing, somehow.

                I will look more deeply into this code later on, but even skimming in it has been valuable

          2. 5

            In real code this isn’t a problem. Common Lisp, at least, has arrays, structs, custom types via deftype, and a full object system.

            It is common to use a cons as a pair (because that’s what a cons is), but other than that, my experience has been that most Lisp programmers are good about defining new types when appropriate.

            1. 5

              The author is either basing his thesis on superficial experience with Lisp, or making a straw man argument. As others pointed out, people don’t actually use lists as product types in real world code.

              1. 8

                I’ve had this problem, and it was part of why I moved away from Lisp and on to Haskell.

                At one time in my life, I very much bought in to the idea that dynamic typing allowed faster development. Then I got a chance to compare and contrast, by implementing the same algorithms (SLR parse-table generation with GLR nondeterministic evaluation) in both Common Lisp and Haskell. I had also, earlier on, written SLR with deterministic LR evaluation in C, so I was already experienced in the area. This was all about fifteen years ago now (2003-ish).

                When I did the CL version, I started with tuples. It turned out to be very challenging to debug. These algorithms have several different concepts which involve grouping alternatives together, and I found that I had a lot of trouble identifying whether a dumped data structure was correct or not.

                When I realized this was the reason I was having trouble, I rewrote the CL code to use the defstruct construct. It made development from that point a lot faster, although it still wasn’t ideal because nothing is actually enforced. It’s a lot more verbose, and honestly very frustrating to use. If I were designing a language today, I would make sure that the safest construct is also the simplest to use, and that would among other things mean leaving untyped tuples out of it completely, since they’re somewhat of an attractive nuisance.

                I then moved on and wrote the Haskell version. Haskell records, though much criticized, were at that time still slightly less verbose to use than CL defstruct classes. They also provided much stronger guarantees of correctness. The Haskell code wound up being un-idiomatic and convoluted (I was new to the language), but I got it debugged very quickly.

                Overall, Common Lisp has a lot of very impressive and clever language constructs which I enjoy understanding the subtleties of. Writing code in CL is fun because it panders to my desire to solve interesting problems, through the wealth of well-designed tools it offers. As someone with an interest in language design, though, I think it’s probably a really bad idea to offer that complexity to the client programmer. I agree with Rob Pike on this point - maintaining code is harder than writing it, so we shouldn’t max out our cognitive resources when we do the initial writing.

                1. 4

                  Awesome to see OCaml’s polymorphic variants mentioned as an alternative to lisp! They’re easily my favorite part of OCaml. This is the link given in the article.

                  1. 2

                    digression: Irken has come a long way!. Almost a full ML-in-lisp, something that Pre-scheme aspired to I think. If only scheme48 had gotten more attention, there is (still) so much good stuff in there.