Good article, but man, I’m so conflicted about having links to paulg and esr at the top.
Like on the one hand, they’re tedious blowhards that no one should listen to, but on the other hand, if you’re about to get into lisp, you need a fair warning that there are a lot of lisp programmers who somehow still take them seriously.
Also it’s a bit weird to see Emacs Lisp barely merit a mention under “Honorable Mentions” while Racket is a “top-tier” one despite being just another Scheme dialect; Emacs Lisp is many times more popular and widely-used than Racket. Sure as a MACLisp descendant it’s ugly and weird, but it’s also ridiculously practical.
On your points about ESR and PG, I largely agree. PG is the quintessential smug lisp weenie, and I dislike both politically in a pretty big way - but I still find their posts relevant. If you do enough digging in the Lisp world you’ll eventually come across PG at a minimum.
Regarding Emacs Lisp I also agree. This was pointed out to me when I posted this to Reddit, and on reflection it feels pretty misleading to call this a Landscape of Lisp and exclude Emacs. It’s just not something I use or know much about at all. I’ll edit the page this weekend to have a dedicate Emacs/Emacs Lisp section, and if you’ve anything you think should be included I’m open to suggestions.
I also think it’s a bit dismissive to call Racket “just another Scheme dialect”. It certainly doesn’t need defended by me but it definitely has an identity, culture, community and interesting contributions to PL theory behind it. It’s inclusion despite Emacs Lisp not being included was because of my own ignorance, not really any favouritism.
I mean, you could spend days writing about this topic, and you have to draw the line somewhere, so I don’t mean to presume to tell you how to write your article. =)
I know I’m biased, (elisp was my first lisp) but to me Emacs Lisp is interesting not only because it gets used in an extremely different way from other lisps, but also because of its ties to historical dialects. Of all other extant lisps, it’s closest to CL, but it kind of shows the “road not taken” of how CL could have turned out if certain things had been decided slightly differently.
There’s also the cultural angle that historically for any lisp, you could pretty much assume that editing it in Emacs would be a way more polished and featureful experience than any other editor. (SLIME, Geiser, Cider, etc) In today’s age of LSP this is maybe less true than it used to be, but Emacs gets a lot of old lispers wistful for the ancient unattainable dream of a usable lisp machine, because writing enough lisp eventually makes you pine for “parens all the way down”.
I agree that Emacs Lisp deserves a bigger spot for the reasons you state, but I want to chime in that Racket is not just another Scheme dialect. That’s how it started, but they diverged quite some time ago. Racket has great features that are not in Scheme, and while every serious Scheme goes beyond the standard, Racket has some things that are unique. The most important one, in my opinion, is the macro system. I know you don’t seem to love or think highly of macros, and they can definitely be misused. But Racket has far and away the most advanced macro system, with loads of features that work together to make it best-in-class. Eg. it has the #lang protocol, ways for macros to “work together” much more than is typical, the syntax-parse DSL for pattern matching, cleanly separated macro phases, a different hygiene algorithm (ultimately necessitated by some of its other advanced macro features), syntax-parameters, etc. After years of soaking in Racket in grad school, every other macro system just pales in comparison. I’m always wanting other languages to have a feature to annotate source locations so that I can use Racket as a pre-processor to use its macro system, because no other macro system is remotely capable of what Racket’s can do. I’m always wishing that Racket were (1) more popular to give more opportunity to use it, and (2) faster to start up so I would more often feel like it’s the best choice for little scripts that I want to run instantly.
I’ve invested many hundreds of hours into Lisps, cut my teeth with them, love them dearly, and still have romantic feelings about them. But I heard something on Twitter once that somewhat haunts me:
Lisp did everything first and better; except “get used and adopted”
I think there’s something to that. I think Lisps are wonderful dopamine factories for curious and playful programmers, and you could use it to ship world-class software, but I’ve struggled to believe its feature set meaningfully provides a whole lot more leverage? Maybe I’m just too smooth-brained to use it, but I have a big appetite for different approaches to programming and it mostly results in “I have more fun,” not “I ship more value, or higher value.”
While doing things in Scheme or CL always felt like I was close to something True about computing, if I consider what the experience has been like delivering features, I don’t think it gave me functionally a whole lot more than, say, Python. I think “dynamically-typed, garbage collected language with decent performance” was more of a draw in the 90’s, and even then, Perl was more popular for that use case. I tell myself I was learning lessons that I could take with me to the next codebase where I’d get that great boon of productivity, or that eventually it’d all become a bit more comfortable, but it never came, even after stretching myself.
In theory macros are a game-changer and unique to the language, but even in those communities I’ve met so, so few people who deploy them, and in my experience, it’s hard to find a use case for them that’s not well-served by a collection of functions. It reminds me of the myths of those people who claim to be 10x better in Forth. I feel like in order to harvest that your brain has to be just so different.
Some of the features in the Land of Lisp promo comic (the blinking blue ones) are pretty rare in other places, like restarts and continuations; I get a lot of power of restarts from BEAM languages, and continuations are still a black magic to me other than things like exception handlers. It always felt like a solution in search of a problem. Has anyone here used continuations to great effect, for things that weren’t developing language features like cooperative scheduling, or nondeterminism? (e.g. amb)
Not saying any of this to agitate. I’m just stuck in this place of “should I sink more time into it and try to Make It Happen (because it is great fun, even if I don’t ship much in it) or play somewhere else, like logic programming, array programming, concatenative programming…?”
Anyways, wonderful article, so much to follow-up on.
Lisp did everything first and better; except “get used and adopted”
This is a good quote, and it reminded me of one I heard from a Roger Hui talk (creator of J) about some advice he had been given:
Don’t worry about people stealing your idea. If it’s any good, you’ll have to force it down their throats.
I think we all have some in-built feeling that good ideas are like sparks that catch fire, but sometimes they never do, and that’s not really because of the quality of the idea.
Not saying any of this to agitate. I’m just stuck in this place of “should I sink more time into it and try to Make It Happen (because it is great fun, even if I don’t ship much in it) or play somewhere else, like logic programming, array programming, concatenative programming…?”
Interesting post, and I understand what you mean - I’ve had a lot of the same thoughts myself. I don’t really see Lisp(s) as the be-all-end-all for me personally. They’re something I find neat and they’ve helped change how I view software development, but I can see there’s also plenty of other types of programming that will probably have a similar effect. The next thing I want to get into are array based languages: APL, J, Uiua etc. Variety is the spice of life and all that.
I don’t think it gave me functionally a whole lot more than, say, Python. I think “dynamically-typed, garbage collected language with decent performance” was more of a draw in the 90’s, and even then, Perl was more popular for that use case
I am myself super glad it gives me this over Python and other dynamically-type languages:
compile-time warnings and errors, function by function, instantaneous, built-in, no config or external tools required, with a keybinding (SBCL is good and constantly improves. It finds: typos, unused code path, (some) bad argument types, (some) bad returned types… for a Haskell on top of CL, we now have Coalton)
compiling to a self-contained binary
include static assets, rsync to my server, I deployed.
(SBCL binaries rely on the glibc)
stability (while the implementations and the libraries improve)
performance
light in resources
can install libraries from within the running image
interactive and fun development / debugging
restarts: fix a bug and resume the program from where it failed, don’t restart from scratch: do this every days, for small or long computations, you save time.
I never wait for my webserver to restart: I work from within the image
As a hobby programmer I feel privileged as I’m not contrained by the necessities of production software in an industrial setting. I’m free to pick whatever is most fun and playful to me. Over the past decades I checked out and read about many great languages but, so far, what really resonates with me, fulfills these needs, and sticks is Lisp.
In theory macros are a game-changer and unique to the language, but even in those communities I’ve met so, so few people who deploy them, and in my experience, it’s hard to find a use case for them that’s not well-served by a collection of functions
This is consistent with my experience of 20+ years writing lisps. Some lisp communities tend to treat macros with a kind of holy reverence, because they know that macros are the one thing inherent to lisps that can’t be copied by non-lisps. But the cases where they’re justified are … few.
In my mind the benefits of lisp syntax have more to do with consistent notation and structural editing than the macro system.
Has anyone here used continuations to great effect, for things that weren’t developing language features like cooperative scheduling, or nondeterminism? (e.g. amb)
I’ve never found a use for full continuations, but “full” or “stackful” coroutines offer you a subset of their power and allow you to do some really cool things that most languages don’t support: https://technomancy.us/202 (Continuations can be used to implement coroutines but can be thought of as a superset.)
I’m just stuck in this place of “should I sink more time into it and try to Make It Happen (because it is great fun, even if I don’t ship much in it)
If you’re not doing it for career reasons, then do it for as long as it’s fun, then stop once it isn’t! =)
Yes, I didn’t mean to imply macros are unique to lisps; I meant that lisp’s approach to macros cannot be copied by non-lisps, because any language that uses lisp’s approach to macros (where programs are written using the same notation as data structures) by definition is already a lisp.
(But my initial comment was written in haste and glossed over that nuance.)
I have a big appetite for different approaches to programming and it mostly results in “I have more fun,” not “I ship more value, or higher value.”
It’s quite wonderful that when you’re so passionate about the field, the lessons that you can share become very valuable to the “trade” of programming, in addition to the art. I’ve added your article to my “PLT” bookmark list, but it’s really a “how I learn” type of thing. Muscle memory, documentation, invariants, tooling and automation. Thanks for sharing, very inspiring!
hard to find a use case for them that’s not well-served by a collection of functions
I use a lot for data DSLs (e.g. full financial models in a single line, so you can fit in many models in a document), especially for expanding those into more data (e.g. let 1-5 expand to 1, 2, 3, 4, 5) or slots for fuzzing, calling APIs to get data, prepare an environment etc. The goal is condensing everything, to make it easier to read, reconfigure etc. although my notations aren’t an optimal tool of thought.
I think the lisp tools empower library/framework writers, while library-connectors/application makers only enjoy better notation but a good framework is well fitted to its problem, whether in Python, Lisp or anything else. The lisp promise’s that it’s easier to write well-fitted frameworks/libraries/DSLs, hence Practical Common Lisp demonstrating so much library writing instead of sewing outside libraries together.
Non-opinionated, Common Lisp and Racket give you more tools than you need, so you can utilize your preferred approach. I’m sure if I embraced an image-based deployment, restarts etc. functions would make more sense for a lot of this but most of my time is building new models, so most of my code condenses their representation.
If you want a modern lisp which is not fragmented into several incompatible dialects, uses arrays rather than linked lists and does not name its functions based on registers on a 70 years old computer, there is Janet.
I love Scheme, but I don’t love how reactionary the community is, it used to be that it could at least bring in new things and it pioneered a lot of cool stuff. But somewhere along the line it stopped. Actual powerful macros in the form of syntax-case got in to R6RS and then got thrown out. Delimited continuations has been around for 30 years, are quite a step up from unrestricted continuations, are simple to implement and they haven’t had a shot to get in. So I understand why Racket broke away.
I made the mistake of clicking on the link about why Scheme is not a Lisp, and now I’m convinced that all the explanation needed for why Lisp isn’t popular is comp.lang.lisp, what a cesspool that was.
FWIW, Common Lisp has concurrency, functional programming, immutable data structures (if you want them), and can target the JVM, too.
And the rich history is a double edged sword. I personally love it, but a lot of people see “car” and “cdr”, miss that there’s “first” and “rest”, and decide the language is clunky and old fashioned. Even this post kinda makes CL sound old fashioned compared to Scheme and Clojure.
Common Lisp has concurrency, functional programming, immutable data structures (if you want them)
Yeah, the article doesn’t really go into detail about the difference between opt-in features vs opt-out features, which, fair enough, it’s trying to be a high-level overview.
You can get immutable data structures on any lisp if you want them, but you can’t assume that they’ll be supported by every library you want to use. You can write a macro in Scheme to let you destructure arguments, but you won’t have pervasive destructuring everywhere and you probably won’t bring in that macro to a brand new file just for a couple functions. You can implement pattern matching in Clojure, but since it’s not built-in to the language, libraries will not structure their APIs around the assumption that return values will be matched against, so it’s a lot less useful than it could be.
Yeah, it’s not perfect, by any means. Built-in pattern matching and parametric types would probably be my top feature requests for CL.
I personally think immutablility as a language feature is overrated. It’s a good ideal to strive for, but in practice it either ruins performance or makes data structure implementation 10x more complicated (and probably still doesn’t match the mutable performance).
What’s the status of strong static typing in lisps? It’s something that always strikes me as much harder to implement (especially since it doesn’t help you when writing macros) and one of the things I miss the most when I do mess around with a lisp.
With the advent of jank, babashka, fennel and clojurescript, I wonder if the Clojure family covers more ground than the Common Lisp and Scheme families.While the latter two families also have their corresponding implementations (Clasp for example), I am thinking more in terms of userbase, tooling and packages.
Good article, but man, I’m so conflicted about having links to paulg and esr at the top.
Like on the one hand, they’re tedious blowhards that no one should listen to, but on the other hand, if you’re about to get into lisp, you need a fair warning that there are a lot of lisp programmers who somehow still take them seriously.
Also it’s a bit weird to see Emacs Lisp barely merit a mention under “Honorable Mentions” while Racket is a “top-tier” one despite being just another Scheme dialect; Emacs Lisp is many times more popular and widely-used than Racket. Sure as a MACLisp descendant it’s ugly and weird, but it’s also ridiculously practical.
On your points about ESR and PG, I largely agree. PG is the quintessential smug lisp weenie, and I dislike both politically in a pretty big way - but I still find their posts relevant. If you do enough digging in the Lisp world you’ll eventually come across PG at a minimum.
Regarding Emacs Lisp I also agree. This was pointed out to me when I posted this to Reddit, and on reflection it feels pretty misleading to call this a Landscape of Lisp and exclude Emacs. It’s just not something I use or know much about at all. I’ll edit the page this weekend to have a dedicate Emacs/Emacs Lisp section, and if you’ve anything you think should be included I’m open to suggestions.
I also think it’s a bit dismissive to call Racket “just another Scheme dialect”. It certainly doesn’t need defended by me but it definitely has an identity, culture, community and interesting contributions to PL theory behind it. It’s inclusion despite Emacs Lisp not being included was because of my own ignorance, not really any favouritism.
I mean, you could spend days writing about this topic, and you have to draw the line somewhere, so I don’t mean to presume to tell you how to write your article. =)
I know I’m biased, (elisp was my first lisp) but to me Emacs Lisp is interesting not only because it gets used in an extremely different way from other lisps, but also because of its ties to historical dialects. Of all other extant lisps, it’s closest to CL, but it kind of shows the “road not taken” of how CL could have turned out if certain things had been decided slightly differently.
There’s also the cultural angle that historically for any lisp, you could pretty much assume that editing it in Emacs would be a way more polished and featureful experience than any other editor. (SLIME, Geiser, Cider, etc) In today’s age of LSP this is maybe less true than it used to be, but Emacs gets a lot of old lispers wistful for the ancient unattainable dream of a usable lisp machine, because writing enough lisp eventually makes you pine for “parens all the way down”.
I agree that Emacs Lisp deserves a bigger spot for the reasons you state, but I want to chime in that Racket is not just another Scheme dialect. That’s how it started, but they diverged quite some time ago. Racket has great features that are not in Scheme, and while every serious Scheme goes beyond the standard, Racket has some things that are unique. The most important one, in my opinion, is the macro system. I know you don’t seem to love or think highly of macros, and they can definitely be misused. But Racket has far and away the most advanced macro system, with loads of features that work together to make it best-in-class. Eg. it has the
#langprotocol, ways for macros to “work together” much more than is typical, the syntax-parse DSL for pattern matching, cleanly separated macro phases, a different hygiene algorithm (ultimately necessitated by some of its other advanced macro features), syntax-parameters, etc. After years of soaking in Racket in grad school, every other macro system just pales in comparison. I’m always wanting other languages to have a feature to annotate source locations so that I can use Racket as a pre-processor to use its macro system, because no other macro system is remotely capable of what Racket’s can do. I’m always wishing that Racket were (1) more popular to give more opportunity to use it, and (2) faster to start up so I would more often feel like it’s the best choice for little scripts that I want to run instantly.eh, before he turned into chief opinion-haver with yc, he wrote two good textbooks (also mentioned later in OP): ANSI Common Lisp and On Lisp.
It’s widely used but I also wouldn’t use it unless I was, well, doing stuff with emacs. (And I say this as someone who used emacs for a decade!)
I’ve invested many hundreds of hours into Lisps, cut my teeth with them, love them dearly, and still have romantic feelings about them. But I heard something on Twitter once that somewhat haunts me:
I think there’s something to that. I think Lisps are wonderful dopamine factories for curious and playful programmers, and you could use it to ship world-class software, but I’ve struggled to believe its feature set meaningfully provides a whole lot more leverage? Maybe I’m just too smooth-brained to use it, but I have a big appetite for different approaches to programming and it mostly results in “I have more fun,” not “I ship more value, or higher value.”
While doing things in Scheme or CL always felt like I was close to something True about computing, if I consider what the experience has been like delivering features, I don’t think it gave me functionally a whole lot more than, say, Python. I think “dynamically-typed, garbage collected language with decent performance” was more of a draw in the 90’s, and even then, Perl was more popular for that use case. I tell myself I was learning lessons that I could take with me to the next codebase where I’d get that great boon of productivity, or that eventually it’d all become a bit more comfortable, but it never came, even after stretching myself.
In theory macros are a game-changer and unique to the language, but even in those communities I’ve met so, so few people who deploy them, and in my experience, it’s hard to find a use case for them that’s not well-served by a collection of functions. It reminds me of the myths of those people who claim to be 10x better in Forth. I feel like in order to harvest that your brain has to be just so different.
Some of the features in the Land of Lisp promo comic (the blinking blue ones) are pretty rare in other places, like restarts and continuations; I get a lot of power of restarts from BEAM languages, and continuations are still a black magic to me other than things like exception handlers. It always felt like a solution in search of a problem. Has anyone here used continuations to great effect, for things that weren’t developing language features like cooperative scheduling, or nondeterminism? (e.g.
amb)Not saying any of this to agitate. I’m just stuck in this place of “should I sink more time into it and try to Make It Happen (because it is great fun, even if I don’t ship much in it) or play somewhere else, like logic programming, array programming, concatenative programming…?”
Anyways, wonderful article, so much to follow-up on.
This is a good quote, and it reminded me of one I heard from a Roger Hui talk (creator of J) about some advice he had been given:
I think we all have some in-built feeling that good ideas are like sparks that catch fire, but sometimes they never do, and that’s not really because of the quality of the idea.
Interesting post, and I understand what you mean - I’ve had a lot of the same thoughts myself. I don’t really see Lisp(s) as the be-all-end-all for me personally. They’re something I find neat and they’ve helped change how I view software development, but I can see there’s also plenty of other types of programming that will probably have a similar effect. The next thing I want to get into are array based languages: APL, J, Uiua etc. Variety is the spice of life and all that.
Funny this is the second time I see this quote this week. Apaprently it is from H. H. Aiken.
I am myself super glad it gives me this over Python and other dynamically-type languages:
As a hobby programmer I feel privileged as I’m not contrained by the necessities of production software in an industrial setting. I’m free to pick whatever is most fun and playful to me. Over the past decades I checked out and read about many great languages but, so far, what really resonates with me, fulfills these needs, and sticks is Lisp.
This is consistent with my experience of 20+ years writing lisps. Some lisp communities tend to treat macros with a kind of holy reverence, because they know that macros are the one thing inherent to lisps that can’t be copied by non-lisps. But the cases where they’re justified are … few.
In my mind the benefits of lisp syntax have more to do with consistent notation and structural editing than the macro system.
I’ve never found a use for full continuations, but “full” or “stackful” coroutines offer you a subset of their power and allow you to do some really cool things that most languages don’t support: https://technomancy.us/202 (Continuations can be used to implement coroutines but can be thought of as a superset.)
If you’re not doing it for career reasons, then do it for as long as it’s fun, then stop once it isn’t! =)
I just don’t agree that macros are unique to lisp. Rust, scala, and presumably other ml descendants have full featured macros.
Yes, I didn’t mean to imply macros are unique to lisps; I meant that lisp’s approach to macros cannot be copied by non-lisps, because any language that uses lisp’s approach to macros (where programs are written using the same notation as data structures) by definition is already a lisp.
(But my initial comment was written in haste and glossed over that nuance.)
It’s quite wonderful that when you’re so passionate about the field, the lessons that you can share become very valuable to the “trade” of programming, in addition to the art. I’ve added your article to my “PLT” bookmark list, but it’s really a “how I learn” type of thing. Muscle memory, documentation, invariants, tooling and automation. Thanks for sharing, very inspiring!
I use a lot for data DSLs (e.g. full financial models in a single line, so you can fit in many models in a document), especially for expanding those into more data (e.g. let 1-5 expand to 1, 2, 3, 4, 5) or slots for fuzzing, calling APIs to get data, prepare an environment etc. The goal is condensing everything, to make it easier to read, reconfigure etc. although my notations aren’t an optimal tool of thought.
I think the lisp tools empower library/framework writers, while library-connectors/application makers only enjoy better notation but a good framework is well fitted to its problem, whether in Python, Lisp or anything else. The lisp promise’s that it’s easier to write well-fitted frameworks/libraries/DSLs, hence Practical Common Lisp demonstrating so much library writing instead of sewing outside libraries together.
Non-opinionated, Common Lisp and Racket give you more tools than you need, so you can utilize your preferred approach. I’m sure if I embraced an image-based deployment, restarts etc. functions would make more sense for a lot of this but most of my time is building new models, so most of my code condenses their representation.
If you want a modern lisp which is not fragmented into several incompatible dialects, uses arrays rather than linked lists and does not name its functions based on registers on a 70 years old computer, there is Janet.
Great article, would it be possible to add a RSS feed to your website? Thanks.
Thanks for reading - I added an RSS feed here:
https://churchofturing.github.io/feed.xml
Hopefully this works for you, I tested it on my MiniFlux client and it seemed to work.
Thanks!
Thanks, glad you like it. But I’m actually not the author, I just shared the post.
I love Scheme, but I don’t love how reactionary the community is, it used to be that it could at least bring in new things and it pioneered a lot of cool stuff. But somewhere along the line it stopped. Actual powerful macros in the form of syntax-case got in to R6RS and then got thrown out. Delimited continuations has been around for 30 years, are quite a step up from unrestricted continuations, are simple to implement and they haven’t had a shot to get in. So I understand why Racket broke away.
I made the mistake of clicking on the link about why Scheme is not a Lisp, and now I’m convinced that all the explanation needed for why Lisp isn’t popular is comp.lang.lisp, what a cesspool that was.
FWIW, Common Lisp has concurrency, functional programming, immutable data structures (if you want them), and can target the JVM, too.
And the rich history is a double edged sword. I personally love it, but a lot of people see “car” and “cdr”, miss that there’s “first” and “rest”, and decide the language is clunky and old fashioned. Even this post kinda makes CL sound old fashioned compared to Scheme and Clojure.
Yeah, the article doesn’t really go into detail about the difference between opt-in features vs opt-out features, which, fair enough, it’s trying to be a high-level overview.
You can get immutable data structures on any lisp if you want them, but you can’t assume that they’ll be supported by every library you want to use. You can write a macro in Scheme to let you destructure arguments, but you won’t have pervasive destructuring everywhere and you probably won’t bring in that macro to a brand new file just for a couple functions. You can implement pattern matching in Clojure, but since it’s not built-in to the language, libraries will not structure their APIs around the assumption that return values will be matched against, so it’s a lot less useful than it could be.
Also applies to attempts to build CL’s condition system as a 3rd-party library in other languages: https://lobste.rs/s/5xa3gt/farolero_common_lisp_style_conditions#c_13pm1b
Yeah, it’s not perfect, by any means. Built-in pattern matching and parametric types would probably be my top feature requests for CL.
I personally think immutablility as a language feature is overrated. It’s a good ideal to strive for, but in practice it either ruins performance or makes data structure implementation 10x more complicated (and probably still doesn’t match the mutable performance).
What’s the status of strong static typing in lisps? It’s something that always strikes me as much harder to implement (especially since it doesn’t help you when writing macros) and one of the things I miss the most when I do mess around with a lisp.
Coalton for CL reached 1.0, it’s used in the industry (quantum computing): https://github.com/coalton-lang/coalton/
With the advent of jank, babashka, fennel and clojurescript, I wonder if the Clojure family covers more ground than the Common Lisp and Scheme families.While the latter two families also have their corresponding implementations (Clasp for example), I am thinking more in terms of userbase, tooling and packages.