I can’t articulate this idea, because I’m not well versed enough in programming language implementation. Maybe some forth or lisp folks can help me. A programming language can be very minimal, which is great. But it does reach a certain minimum possible complexity with regard to syntax. Every non-trivial programming language has to have syntax for function definition. Even in these sorts of minimal syntax languages, a la forth and lisp, they still have some canned way to define a function, that breaks out of the usual execution control flow. I’m not saying this is true. Maybe in lisp it is not true. But is there an axiom or universal rule lurking in this idea? Maybe somebody familiar with the lambda calculus can educate me.
Well, “function definition” can be broken up into two parts, evaluating some expression that yields an anonymous function as a value, and then binding that function value to a name:
foo = lambda n: n + 42
Many languages (such as python here) have some syntactical sugar that smushes these two concepts together:
def foo(n):
return n + 42
So, I’m curious, are you inquiring about functions themselves, or about giving names to functions?
I’m thinking mainly of the lambda syntax in the first example, and the def syntax in the second. In forth this is :. The idea is you have to have some kind of syntax to indicate when you’re defining a function, versus a plain expression. You have to designate that n+42 is a function body, not an expression you want to yield, render, or print. There’s no way around it, in the most minimal syntax imaginable, there has to be some kind of “I’m defining a function here” syntax element. I’m wondering if that’s true, even for lisp, and if it’s worth mentioning.
It’s funny because you don’t even need to give a function a name. In javascript you can define a function without giving it any name. I’m pretty sure function(x){return x+42;} is valid, even if it won’t pass a linter. So I don’t think the convenience of naming the function is key here. The point is that you have to break out of executing the code, and establish that your defining a function. That it’s convenient to also give it at a name at the same time is likely, but not critical.
i think what you’re getting at is the idea of deferred evaluation? look up haskell’s lazy evaluation, and scala’s call by name for some other ways languages handle this.
Popular programming languages are indirectly descended from Church’s lambda calculus, and Lisp is directly inspired by lambda calculus. In this usual sort of language, you have to have function definitions or lambda expressions, as you have expressed. The body of the function is not evaluated until the function is called, that’s what I think you mean by “breaks out of the usual execution control flow”.
But there’s another lineage that derives not from lambda calculus, but from combinatory logic. Starting from this base, it’s possible to define an extremely minimal language where the only primitives are: a basic set of named functions, and function call syntax. Nothing else. No function definitions or lambda expressions. This “basic set of named functions” are combinators that take functions as arguments, and return functions as results. All new functions are constructed by calling combinators with other functions as arguments. A well known example is the SKI combinator calculus: https://en.wikipedia.org/wiki/SKI_combinator_calculus.
So it is not axiomatic or universal that a Turing-complete language must have function definition syntax.
The SKI calculus is not a practical programming language, but there have been more practical languages, more or less inspired by these ideas, that don’t have function definitions or lambda expressions. The simplest example I can remember is FP, designed by John Backus: https://en.wikipedia.org/wiki/FP_(programming_language).
Thanks, I appreciate you seeing through my opaque question :) “…that’s what I think you mean…” Yes that’s what I meant. I figured smarter folks than me must have already thought about this. Thanks for the links point me in the right direction! This SKI combinator calculus stuff sounds like what I was thinking of, from an academic sense. And anything by Backus is worth knowing more about, so I’ll read up on the FP language, too.
The most extreme examples of combinator calculs language are Iota and Jot, both languages that require no more than 2 symbols to run. Iota in particular is based on the single iota combinator.
I love the iota combinator, it’s a combination of S and K that’s Turing complete by itself:
S = λx. λy. λz. x z (y z)
K = λx. λy. x
ι = λx. x S K
We could inline it all, but that’s quite unreadable:
In a Forth-like, or Joy-like, language a lambda(an unnamed function) could be explained mechanically as a jump over a segment of bytecode, and putting a pointer to the start of that block on the stack. It doesn’t leave the normal execution control.
A named function is the same process but a label is assigned to the start of the lambda block.
In most any language, a program like print "hello world" will just print the message. There has to be some way of defining a function. A code block like def f(){print "hello world"} doesn’t actually print a message. It establishes a function that, when called, will print a message. So here the def syntax is necessary, you couldn’t get away without it. This crumb language seems to be trying to reduce everything down to the minimum possible amount of syntax. Even it has function definition syntax. My question is, doesn’t every language have to have function definition syntax?
NB: I’m not the author, though the excellent @jvns is!
I’ve written numerous tiny scripts and programs to mold my desktop workflow to my wants, one of my favorites a series of scripts that lets me search though files (filtering with rofi/dmenu) in predetermined locations in my homedir, and then pipes the file name to a custom plumber script (written in fennel) that uses file name and mime type to determine a program to open the file, it’s incredibly fast for finding random files! The script also has a few regexen for opening urls, it’s nice for things like opening youtube links directly in mpv.
Another one is a screenshotting script that combines slurp (choosing windows/rectangles on screen), grim (wayland screenshotting), and uploading directly to an image pastebin for really fast screenshot sharing.
In a similar spirit, I often end up writing many small elisp functions for individual projects to reduce the friction of the development cycle. I’m currently taking an undergraduate operating systems course, and I have a function that lets me quickly choose a user program to boot up the OS under qemu with gdb, of course taking advantage of emacs’ gdb mode. I love how easy emacs makes it to write tiny bits of software to support and integrate my workflows!
I think there are several reasons for regex hate and they’re slightly contradictory:
First, a lot of ‘regex’ languages are more expressive than regular expressions. If I remember correctly, Perl regexes are actually Turing complete (or, at least, they’re push-down automata - I can’t remember which). This means that they’re a very dense and expressive language for writing parsers. This language, as a side effect of being so concise, is very hard to read. This is fine for single-use one liners but you don’t want to be stuck maintaining a complex regular expression.
Second, the fact that there are a lot of regex languages. C++11 defined six different regex languages, with several additional modifiers and this didn’t include Perl-compatible regular expressions. Several regex languages use different syntax for the same thing or similar syntax for completely different things. This is hinted at in the article, which explicitly says that it’s using Vim’s regex syntax. Just because you’re familiar with regexes in one context doesn’t mean that you can correctly read them in another.
Third, and in contradiction to the first point, regexes can parse only regular languages. A lot of things that you parse are not regular languages. The canonical example here is matching brackets, which requires counting and so can be parsed with a Turing Machine or a push-down automaton, but not with a finite state machine (regular expressions are an encoding of FSMs). This leads to horrible fragile hacks. For example, the Lisp syntax highlighting in Vim defines 10 (I think - it’s been over a decade since I looked) different syntax rules for brackets inside other brackets. Each one is coloured differently and so if you nest brackets up to 10 deep then you get different colours. After that, it breaks. This rule is trivial to express in any programming language (define a set of colours, count the bracket depth, increment when before you reach an open bracket, decrement after you reach a closed bracket, pick colour depth mod number of colours to colour any bracket) but cannot be expressed with regular expressions.
Like many other things (object orientation, functional programming, machine learning, distributed systems, and so on), regular expressions are a fantastic tool for some problems and a complete disaster when applied to problems in the wrong domain.
For example, the Lisp syntax highlighting in Vim defines 10 (I think - it’s been over a decade since I looked) different syntax rules for brackets inside other brackets. Each one is coloured differently and so if you nest brackets up to 10 deep then you get different colours. After that, it breaks.
Couldn’t you just make it so that the last rule can contain the first?
If vim is using proper regexes, you cannot. As mentioned above, regular expressions are equivalent to finite state machines, which mathematically cannot parse arbitrarily deep recursive structures in strings. Here’s a good article which covers this in its first part.
It might be possible outside of the regex system but that’s not how it was defined. Each of the rules was a single regex. If you are a pair of brackets with no brackets inside them, you are the first colour. If you are a pair of brackets with one pair of brackets inside them, you are the second colour. If you are… and so on. That’s all that you can do with regular expressions.
Well, then they’re doing it wrong. Regexes are not meant to be used alone. Here is a fully functioning Vim script that highlights nested parentheses in alternating colors:
If I had never written Lisp and I wanted to attempt this in a Lisp-language which one would you suggest? Ideally I want something that I can learn quickly (the basics to solve some problems at least) and is batteries-included enough to let me parse strings or whatever without too much of a hassle.
Racket is quite user friendly and has good docs. The standard library is massive too, it comes with a pretty good GUI library and an IDE written with it in its default distribution.
I completed the advent in Clojure last year, and found it very suitable to the task.
The standard library is comprehensive, but small; the batteries-included (persistent) data-structures are a real treat; and the sequence-processing primitives are well suited to the advent problems.
Clojure was my first lisp, and I’ve enjoyed every minute of it.
Congratulations on the milestone! I’ve only shipped plain-old-lua so far, but I’ve been smiling as I watched Fennel get better for a while now. Nicely done. And I look forward to the game jam where using Fennel turns out to be a good idea for me :)
I was just talking to a friend of mine who is a game dev and used to use Lua a lot when he worked in cryengine. He complained about global variables in Lua and how Lua can return null values when a value isn’t present in a collection. I had brought up Fennel, but I wasn’t sure if it tackled either of those issues. I know that Fennel is semantically very similar to Lua. Does it provide any protections against either of those types of bugs?
On global variables: in fennel, only names bound with (global ...) produce global variables, all other binding mechanisms (pattern matching, let, var, local) all produce local variables. Very hard to accidentally make a global variable :P
On nil in tables, not directly. You’ll still get nil indexing into a table where there’s no value, but there is a macro like this: (?. tbl :foo :bar :baz) that will nil-check each index and short-circuit so you don’t accidentally index into nil, producing an error. You still have to nil-check individual indexing yourself though.
how Lua can return null values when a value isn’t present in a collection
This is honestly kind of an odd complaint since it’s also true of nearly every other dynamic language out there. (Racket and Erlang being the main exceptions I know of.) Lua’s behavior (and thus Fennel’s too) is sublty different in that nil is much more consistently used to represent the absence of data, rather than other languages where you can have absurd situations like “I put nothing in this list and now the list is longer than it used to be”. In Fennel/Lua there is no difference between [] and [nil]. This takes some adjusting to if you are used to the other behavior but once you learn it, it’s a lot less error-prone.
Fennel has a few advantages over Lua that make it easier to deal with nils when they do occur. The other comment mentions the nil-safe deep accessor macro ?. but there is also automatic nil-checks for function arguments with lambda as well as the fact that the pattern matching system defaults to including nil-checks in every pattern.
This is honestly kind of an odd complaint since it’s also true of nearly every other dynamic language out there.
They’re primarily working in statically typed languages (C++, C#) so it’s not totally a dig against Lua. Just normal workflow mismatch vs dynamically typed paradigm. Also, it was in the context of video game designers using scripting languages (e.g., Blueprint), getting out of their depth, and losing a couple days on bugs.
(Racket and Erlang being the main exceptions I know of.)
Python also throws when you try to access a value that isn’t present in a dictionary. Always reminds me of this blog post:
I liken this to telling each language to go get some milk from the grocery store. Ruby goes to the store, finds no milk and returns no milk (nil). Python goes to the store, finds no milk and BURNS THE STORE TO THE GROUND (raises an exception)!
In Fennel/Lua there is no difference between [] and [nil]. This takes some adjusting to if you are used to the other behavior but once you learn it, it’s a lot less error-prone.
That is interesting! I didn’t know it did that.
Thanks for taking the time to respond! I don’t really work in the video game space, but fennel seems like an interesting choice for video games. Piggy backing off of Lua for embedding in game engines and using macros to create DSLs seems like an interesting niche.
E.g., I could imagine game designers editing rules in a minikanren DSL to affect changes in the game world.
Haha; yeah can’t believe I forgot about Python. I listed Erlang and Racket there because they specifically do not even have a notion of nil/null in the entire language, whereas Python somehow manages to include null in their semantics, but not actually use it for this.
I liken this to telling each language to go get some milk from the grocery store. Ruby goes to the store, finds no milk and returns no milk (nil). Python goes to the store, finds no milk and BURNS THE STORE TO THE GROUND (raises an exception)!
On the other hand I’m actually somewhat sympathetic to this because nils are the most common type error in my experience by at least an order of magnitude, so I would characterize it differently. To me it’s more like you ask Ruby (or Clojure, or JS, or whatever) to go to the store to get some milk, it comes back, walks over to you, says “here’s what I got you”, holds out its empty hand, and drops nothing into your hand.
If a human did that it would be considered extremely disrespectful or overly sarcastic! Throwing an exception is indeed an overreaction, but there needs to be some middle ground where you don’t just pretend that the operation was successful, and most dynamic languages simply don’t have the ability to convey this. https://blog.janestreet.com/making-something-out-of-nothing-or-why-none-is-better-than-nan-and-null/
edit: the reason this works better in Erlang than other dynamic languages is that pattern matching is baked into the core of the language so it’s easy to represent success as a tuple of [ok, Value] which encourages you to match every time you have an operation which might in other languages return nil. Fennel has a pattern matching system which was in many ways inspired by Erlang.
the reason this works better in Erlang than other dynamic languages is that pattern matching is baked into the core of the language so it’s easy to represent success as a tuple of [ok, Value] which encourages you to match every time you have an operation which might in other languages return nil. Fennel has a pattern matching system which was in many ways inspired by Erlang.
That’s great to hear! I haven’t used Erlang, but I’ve used a bit of Elixir and I really love that design. Jose Valim described assertive code in an article a while back and I appreciate that perspective. That’s another reason why I like python burning down the store so to speak. Having the option to say “hey, I really mean it this time” is always useful.
I once spent a day (or more?) tracking down a bug in a poorly tested Node JS codebase. It turns out that the SQL Server stored procedure was returning ID and the Node process was looking for Id. Of course, JavaScript will happily work with whatever you give it so it was a pain to find. The exception occured way further down the line. It’s hard not to get little bitter after an experience like that.
The reMarkable is such a unique little device, I’ve had a lot of fun using my rM2 for reading and notes. I really appreciate the minimalism of the built-in software, and the open source community that’s developed around it is incredible. Look, other companies, at the kind of community you can have if you tuck root passwords into your devices’ settings app!
well they were forced to via the license (GPLv3 by not paying QT) - support for rM2 OSS from the dev angle seems to be more of a polite gesture as long as the main ‘own’ display control is via binary hacks (LD_PRELOAD on their display server). I love my Rm2 hardware wise, a few thousand notepages in, but I also want their software out of it and run my own.
Strong concur. As sold, the device is usable (with the very latest firmware) as an e-reader, a minimalist drawing/notepad setup, and a device specifically for reading and making marks on PDFs. That’s enough to be usable in certain workflows, since the display and the stylus input are really impressive, but it could be useful for so much more.
AFAIK Urn is effectively its own language with its own semantics, and Fennel I think is better thought of as a lisp-syntax-frontend for Lua. They both compile to Lua, sure, but I believe the Urn compiler is doing a lot more leg work.
That’s correct. The big difference is that Urn does a lot more static analysis; I think at one point they were building a gradual type system into it? But some of that made interop with Lua less direct, and it also made it difficult to support interactive development with a repl. (I started using Urn a bit before I discovered Fennel, but the lack of a repl was rather problematic for me, even tho I liked a lot of other things about it.)
That was definitely the original intent; it was designed to complement a larger board for when you’re away from your desk. (That said, I stopped using my larger board once I got used to it.) Enjoy!
Also, a proud owner of Atreus here. I can say that assembling Atreus was a fun exercise, it’s a compact and very ergonomic keyboard. My touch typing on Atreus is still slow (and even slower when I type in my native language) after I switched from Pok3r, but I enjoy using it.
I was curious why it was built on top of Lua specifically. Is that so Fennel can leverage its JIT’d VM and embedding? Or did you all also value Lua libraries wanting them? The latter I wonder about given Id guess there be lot more libraries for Pythkn, Ruby, etc.
Good question. For me personally it started because I was using the Love2D framework for games, and I became very impressed by the speed and conceptual simplicity of Lua.
But there were still a few remaining annoyances despite its simplicity; the statement/expression distinction, the ease with which you can accidentally create a global, and the lack of arity checks in particular were the only complaints I had with it.
It wasn’t enough for me to do anything about, but when I found Fennel, I realized that creating a compiler that targets Lua doesn’t need to be a big deal. The source for the compiler is under 2kloc. Fennel introduces virtually zero runtime overhead, and the compiler output (with a few exceptions like pattern matching) looks very close to the input.
Thanks for response. 2kloc? That’s impressive! That’s in bootstrapper territory.
Love2D looks neat. The first thing that jumped out is it runs on every major platform. Examples look easier than some GUI toolkits from long ago. So, my first thought is: “Could someone use this as a cross-platform, GUI toolkit with simple language to make great-looking interfaces?”
Yeah, actually 2k isn’t just the compiler; it includes the stock macros and the repl.
Love2D is much easier than any other GUI work I’ve done on any platform, yeah. If you don’t need native widgets or the ability to display more than one window it could certainly be used for non-games. In fact, I created a text editor in about 1.2kloc which is a mix of Fennel and Lua: https://git.sr.ht/~technomancy/polywell
It’s loosely based on Emacs and has syntax highlighting, a buffer switcher, support for repl buffers, completion, tetris, etc.
I’ve also had some success using imgui using the love2d bindings. It’s got a lot of handy UI widgets - I used it for game debug UI, but I could see it being used for non-game apps as well.
I’m thinking about buying some yarn and a hook and trying to learn crocheting. I’d like to make myself a nice winter hat, we’ve got some really cold weather coming and I lost my last nice one.
It is a font designed for use as placeholder text, but without actually displaying any text. This is useful because often non-designers will get hung up on the details of text or (if lorem ipsum is used) wonder what’s wrong with the design that it has all this gibberish in it.
I can’t articulate this idea, because I’m not well versed enough in programming language implementation. Maybe some forth or lisp folks can help me. A programming language can be very minimal, which is great. But it does reach a certain minimum possible complexity with regard to syntax. Every non-trivial programming language has to have syntax for function definition. Even in these sorts of minimal syntax languages, a la forth and lisp, they still have some canned way to define a function, that breaks out of the usual execution control flow. I’m not saying this is true. Maybe in lisp it is not true. But is there an axiom or universal rule lurking in this idea? Maybe somebody familiar with the lambda calculus can educate me.
Well, “function definition” can be broken up into two parts, evaluating some expression that yields an anonymous function as a value, and then binding that function value to a name:
Many languages (such as python here) have some syntactical sugar that smushes these two concepts together:
So, I’m curious, are you inquiring about functions themselves, or about giving names to functions?
I’m thinking mainly of the
lambdasyntax in the first example, and thedefsyntax in the second. In forth this is:. The idea is you have to have some kind of syntax to indicate when you’re defining a function, versus a plain expression. You have to designate thatn+42is a function body, not an expression you want to yield, render, or print. There’s no way around it, in the most minimal syntax imaginable, there has to be some kind of “I’m defining a function here” syntax element. I’m wondering if that’s true, even for lisp, and if it’s worth mentioning.It’s funny because you don’t even need to give a function a name. In javascript you can define a function without giving it any name. I’m pretty sure
function(x){return x+42;}is valid, even if it won’t pass a linter. So I don’t think the convenience of naming the function is key here. The point is that you have to break out of executing the code, and establish that your defining a function. That it’s convenient to also give it at a name at the same time is likely, but not critical.i think what you’re getting at is the idea of deferred evaluation? look up haskell’s lazy evaluation, and scala’s call by name for some other ways languages handle this.
Popular programming languages are indirectly descended from Church’s lambda calculus, and Lisp is directly inspired by lambda calculus. In this usual sort of language, you have to have function definitions or lambda expressions, as you have expressed. The body of the function is not evaluated until the function is called, that’s what I think you mean by “breaks out of the usual execution control flow”.
But there’s another lineage that derives not from lambda calculus, but from combinatory logic. Starting from this base, it’s possible to define an extremely minimal language where the only primitives are: a basic set of named functions, and function call syntax. Nothing else. No function definitions or lambda expressions. This “basic set of named functions” are combinators that take functions as arguments, and return functions as results. All new functions are constructed by calling combinators with other functions as arguments. A well known example is the SKI combinator calculus: https://en.wikipedia.org/wiki/SKI_combinator_calculus.
So it is not axiomatic or universal that a Turing-complete language must have function definition syntax.
The SKI calculus is not a practical programming language, but there have been more practical languages, more or less inspired by these ideas, that don’t have function definitions or lambda expressions. The simplest example I can remember is FP, designed by John Backus: https://en.wikipedia.org/wiki/FP_(programming_language).
Thanks, I appreciate you seeing through my opaque question :) “…that’s what I think you mean…” Yes that’s what I meant. I figured smarter folks than me must have already thought about this. Thanks for the links point me in the right direction! This SKI combinator calculus stuff sounds like what I was thinking of, from an academic sense. And anything by Backus is worth knowing more about, so I’ll read up on the FP language, too.
The most extreme examples of combinator calculs language are Iota and Jot, both languages that require no more than 2 symbols to run. Iota in particular is based on the single iota combinator.
I love the iota combinator, it’s a combination of S and K that’s Turing complete by itself:
We could inline it all, but that’s quite unreadable:
If we go this route I prefer De Bruijn indices:
Neatly fits in on a ring, and is arguably even more badass than Sauron’s quote.
In a Forth-like, or Joy-like, language a lambda(an unnamed function) could be explained mechanically as a jump over a segment of bytecode, and putting a pointer to the start of that block on the stack. It doesn’t leave the normal execution control.
A named function is the same process but a label is assigned to the start of the lambda block.
I’m not sure I can follow. What do you mean by a canned way to break out of the control flow? Like a way to define flow control?
In most any language, a program like
print "hello world"will just print the message. There has to be some way of defining a function. A code block likedef f(){print "hello world"}doesn’t actually print a message. It establishes a function that, when called, will print a message. So here thedefsyntax is necessary, you couldn’t get away without it. This crumb language seems to be trying to reduce everything down to the minimum possible amount of syntax. Even it has function definition syntax. My question is, doesn’t every language have to have function definition syntax?Not sure if this is what you’re asking, but I’m pretty sure there’re Forths where you can do something like:
dup * 'square defYou would need syntax to quote a string/symbol, but that’s it.
See also, part two! https://georgehart.com/bagel/knot.html
How does it compare to maddy?
NB: I’m not the author, though the excellent @jvns is!
I’ve written numerous tiny scripts and programs to mold my desktop workflow to my wants, one of my favorites a series of scripts that lets me search though files (filtering with rofi/dmenu) in predetermined locations in my homedir, and then pipes the file name to a custom plumber script (written in fennel) that uses file name and mime type to determine a program to open the file, it’s incredibly fast for finding random files! The script also has a few regexen for opening urls, it’s nice for things like opening youtube links directly in
mpv.Another one is a screenshotting script that combines
slurp(choosing windows/rectangles on screen),grim(wayland screenshotting), and uploading directly to an image pastebin for really fast screenshot sharing.In a similar spirit, I often end up writing many small elisp functions for individual projects to reduce the friction of the development cycle. I’m currently taking an undergraduate operating systems course, and I have a function that lets me quickly choose a user program to boot up the OS under qemu with
gdb, of course taking advantage of emacs’ gdb mode. I love how easy emacs makes it to write tiny bits of software to support and integrate my workflows!That sounds a bit like “godothecorrectthing” (https://github.com/andrewchambers/godothecorrectthing), which acts on the clipboard’s content.
Love this!
I think there are several reasons for regex hate and they’re slightly contradictory:
First, a lot of ‘regex’ languages are more expressive than regular expressions. If I remember correctly, Perl regexes are actually Turing complete (or, at least, they’re push-down automata - I can’t remember which). This means that they’re a very dense and expressive language for writing parsers. This language, as a side effect of being so concise, is very hard to read. This is fine for single-use one liners but you don’t want to be stuck maintaining a complex regular expression.
Second, the fact that there are a lot of regex languages. C++11 defined six different regex languages, with several additional modifiers and this didn’t include Perl-compatible regular expressions. Several regex languages use different syntax for the same thing or similar syntax for completely different things. This is hinted at in the article, which explicitly says that it’s using Vim’s regex syntax. Just because you’re familiar with regexes in one context doesn’t mean that you can correctly read them in another.
Third, and in contradiction to the first point, regexes can parse only regular languages. A lot of things that you parse are not regular languages. The canonical example here is matching brackets, which requires counting and so can be parsed with a Turing Machine or a push-down automaton, but not with a finite state machine (regular expressions are an encoding of FSMs). This leads to horrible fragile hacks. For example, the Lisp syntax highlighting in Vim defines 10 (I think - it’s been over a decade since I looked) different syntax rules for brackets inside other brackets. Each one is coloured differently and so if you nest brackets up to 10 deep then you get different colours. After that, it breaks. This rule is trivial to express in any programming language (define a set of colours, count the bracket depth, increment when before you reach an open bracket, decrement after you reach a closed bracket, pick colour depth mod number of colours to colour any bracket) but cannot be expressed with regular expressions.
Like many other things (object orientation, functional programming, machine learning, distributed systems, and so on), regular expressions are a fantastic tool for some problems and a complete disaster when applied to problems in the wrong domain.
Couldn’t you just make it so that the last rule can contain the first?
If vim is using proper regexes, you cannot. As mentioned above, regular expressions are equivalent to finite state machines, which mathematically cannot parse arbitrarily deep recursive structures in strings. Here’s a good article which covers this in its first part.
But Vim’s syntax highlighting isn’t just regexes. It’s based on regexes, but they’re connected by a more sophisticated system.
It might be possible outside of the regex system but that’s not how it was defined. Each of the rules was a single regex. If you are a pair of brackets with no brackets inside them, you are the first colour. If you are a pair of brackets with one pair of brackets inside them, you are the second colour. If you are… and so on. That’s all that you can do with regular expressions.
Well, then they’re doing it wrong. Regexes are not meant to be used alone. Here is a fully functioning Vim script that highlights nested parentheses in alternating colors:
If I had never written Lisp and I wanted to attempt this in a Lisp-language which one would you suggest? Ideally I want something that I can learn quickly (the basics to solve some problems at least) and is batteries-included enough to let me parse strings or whatever without too much of a hassle.
Personally I’d choose a Scheme variant if I was trying something Lisp-like for the first time. Probably Racket? https://racket-lang.org/
Or you could go minimal and choose one of the R5RS schemes like Chicken. But Racket has nice tool support.
Seconding racket. Massive standard library, maybe the most popular lisp these days alongside clojure.
Racket is quite user friendly and has good docs. The standard library is massive too, it comes with a pretty good GUI library and an IDE written with it in its default distribution.
I completed the advent in Clojure last year, and found it very suitable to the task.
The standard library is comprehensive, but small; the batteries-included (persistent) data-structures are a real treat; and the sequence-processing primitives are well suited to the advent problems.
Clojure was my first lisp, and I’ve enjoyed every minute of it.
I used Janet last year, and had a great time with it.
Fennel lead developer here.
I’ve posted about Fennel a few times in the past (https://lobste.rs/s/11p7mn/fennel_0_2_0_released https://lobste.rs/s/6bphbw/fennel_programming_language_rationale https://lobste.rs/s/hppjix/making_fennel_compiler_self_hosting_with) on this site for previous milestones and have had some great discussion ensue; feel free to ask me anything!
Congratulations on the milestone! I’ve only shipped plain-old-lua so far, but I’ve been smiling as I watched Fennel get better for a while now. Nicely done. And I look forward to the game jam where using Fennel turns out to be a good idea for me :)
Thanks! Of course you can use Fennel for any game jam, but the big ones are the Lisp Game Jams held every April and October on itch.io: https://itch.io/jam/autumn-lisp-game-jam-2021 / https://itch.io/jam/spring-lisp-game-jam-2021
I was just talking to a friend of mine who is a game dev and used to use Lua a lot when he worked in cryengine. He complained about global variables in Lua and how Lua can return null values when a value isn’t present in a collection. I had brought up Fennel, but I wasn’t sure if it tackled either of those issues. I know that Fennel is semantically very similar to Lua. Does it provide any protections against either of those types of bugs?
On global variables: in fennel, only names bound with
(global ...)produce global variables, all other binding mechanisms (pattern matching,let,var,local) all produce local variables. Very hard to accidentally make a global variable :POn
nilin tables, not directly. You’ll still getnilindexing into a table where there’s no value, but there is a macro like this:(?. tbl :foo :bar :baz)that willnil-check each index and short-circuit so you don’t accidentally index intonil, producing an error. You still have tonil-check individual indexing yourself though.Glad to hear that variables don’t default to globals!
This is honestly kind of an odd complaint since it’s also true of nearly every other dynamic language out there. (Racket and Erlang being the main exceptions I know of.) Lua’s behavior (and thus Fennel’s too) is sublty different in that nil is much more consistently used to represent the absence of data, rather than other languages where you can have absurd situations like “I put nothing in this list and now the list is longer than it used to be”. In Fennel/Lua there is no difference between
[]and[nil]. This takes some adjusting to if you are used to the other behavior but once you learn it, it’s a lot less error-prone.Fennel has a few advantages over Lua that make it easier to deal with nils when they do occur. The other comment mentions the nil-safe deep accessor macro
?.but there is also automatic nil-checks for function arguments withlambdaas well as the fact that the pattern matching system defaults to including nil-checks in every pattern.They’re primarily working in statically typed languages (C++, C#) so it’s not totally a dig against Lua. Just normal workflow mismatch vs dynamically typed paradigm. Also, it was in the context of video game designers using scripting languages (e.g., Blueprint), getting out of their depth, and losing a couple days on bugs.
Python also throws when you try to access a value that isn’t present in a dictionary. Always reminds me of this blog post:
That is interesting! I didn’t know it did that.
Thanks for taking the time to respond! I don’t really work in the video game space, but fennel seems like an interesting choice for video games. Piggy backing off of Lua for embedding in game engines and using macros to create DSLs seems like an interesting niche.
E.g., I could imagine game designers editing rules in a minikanren DSL to affect changes in the game world.
Haha; yeah can’t believe I forgot about Python. I listed Erlang and Racket there because they specifically do not even have a notion of nil/null in the entire language, whereas Python somehow manages to include null in their semantics, but not actually use it for this.
On the other hand I’m actually somewhat sympathetic to this because nils are the most common type error in my experience by at least an order of magnitude, so I would characterize it differently. To me it’s more like you ask Ruby (or Clojure, or JS, or whatever) to go to the store to get some milk, it comes back, walks over to you, says “here’s what I got you”, holds out its empty hand, and drops nothing into your hand.
If a human did that it would be considered extremely disrespectful or overly sarcastic! Throwing an exception is indeed an overreaction, but there needs to be some middle ground where you don’t just pretend that the operation was successful, and most dynamic languages simply don’t have the ability to convey this. https://blog.janestreet.com/making-something-out-of-nothing-or-why-none-is-better-than-nan-and-null/
edit: the reason this works better in Erlang than other dynamic languages is that pattern matching is baked into the core of the language so it’s easy to represent success as a tuple of
[ok, Value]which encourages you to match every time you have an operation which might in other languages return nil. Fennel has a pattern matching system which was in many ways inspired by Erlang.That’s great to hear! I haven’t used Erlang, but I’ve used a bit of Elixir and I really love that design. Jose Valim described assertive code in an article a while back and I appreciate that perspective. That’s another reason why I like python burning down the store so to speak. Having the option to say “hey, I really mean it this time” is always useful.
I once spent a day (or more?) tracking down a bug in a poorly tested Node JS codebase. It turns out that the SQL Server stored procedure was returning ID and the Node process was looking for Id. Of course, JavaScript will happily work with whatever you give it so it was a pain to find. The exception occured way further down the line. It’s hard not to get little bitter after an experience like that.
This is a great post. Props to the author for all of the explanations!
Surprisingly in-depth. Great read for understanding how compilers work, especially for functional languages.
The reMarkable is such a unique little device, I’ve had a lot of fun using my rM2 for reading and notes. I really appreciate the minimalism of the built-in software, and the open source community that’s developed around it is incredible. Look, other companies, at the kind of community you can have if you tuck root passwords into your devices’ settings app!
well they were forced to via the license (GPLv3 by not paying QT) - support for rM2 OSS from the dev angle seems to be more of a polite gesture as long as the main ‘own’ display control is via binary hacks (LD_PRELOAD on their display server). I love my Rm2 hardware wise, a few thousand notepages in, but I also want their software out of it and run my own.
Strong concur. As sold, the device is usable (with the very latest firmware) as an e-reader, a minimalist drawing/notepad setup, and a device specifically for reading and making marks on PDFs. That’s enough to be usable in certain workflows, since the display and the stylus input are really impressive, but it could be useful for so much more.
Full time dvorak user. I got used the positions of hjkl. I try to minimize one-character movements, though.
W and B, ctrl-D and ctrl-U with crtl on capslock, have become my friends using dvorak.
Anyone know how this compares to Fennel, another Lisp-on-Lua language?
AFAIK Urn is effectively its own language with its own semantics, and Fennel I think is better thought of as a lisp-syntax-frontend for Lua. They both compile to Lua, sure, but I believe the Urn compiler is doing a lot more leg work.
That’s correct. The big difference is that Urn does a lot more static analysis; I think at one point they were building a gradual type system into it? But some of that made interop with Lua less direct, and it also made it difficult to support interactive development with a repl. (I started using Urn a bit before I discovered Fennel, but the lack of a repl was rather problematic for me, even tho I liked a lot of other things about it.)
I backed for one of the second-generation Atreus boards, but you can also build your own first-generation right now.
Tiny, “split”, ergonomic/ortholinear, mechanical, hackable. Really looking forward to getting it.
(Atreus creator here) Thanks! I was just talking on #lobsters IRC about how I was working on a new scheme-based firmware for this that’s been a lot of fun: https://git.sr.ht/~technomancy/menelaus/tree/master/menelaus.scm
Figured folks here might find that interesting. It’s fully functional in about 300 lines.
It’s crazy how simple that is compared to some more popular firmwares written C… Lovely work as always
I just backed - not sure if I’ll end up converted from my Kineses Advantage, but I sure would like a portable option :)
That was definitely the original intent; it was designed to complement a larger board for when you’re away from your desk. (That said, I stopped using my larger board once I got used to it.) Enjoy!
Also, a proud owner of Atreus here. I can say that assembling Atreus was a fun exercise, it’s a compact and very ergonomic keyboard. My touch typing on Atreus is still slow (and even slower when I type in my native language) after I switched from Pok3r, but I enjoy using it.
I’ve hooked my Pi 4 to a 4TB usb drive and I’m using it as a borg-backup repository!
I lurk, but I’m trying to post more stuff. @oats@mastodon.social
I’m a linux enthusiast, bassoonist, computer engineering student, and amateur programmer!
Oh, that’s lovely. I really like the look and feel of your website! Could you describe a little what you used to put it together?
It’s based on a few things:
It is a very boring Go app.
Cheers!
Wow! Impressive project, hardware and software wise. I wonder what the battery life is like.
The creator on the reddit thread says about a week.
Wow that’s a lot more than I was expecting.
I’m one of the lead devs for Fennel; been working on it for about a year now and having a ton of fun.
We’ve got a free 1-day conference in Portland next month: https://conf.fennel-lang.org/2019
Ask me anything!
I was curious why it was built on top of Lua specifically. Is that so Fennel can leverage its JIT’d VM and embedding? Or did you all also value Lua libraries wanting them? The latter I wonder about given Id guess there be lot more libraries for Pythkn, Ruby, etc.
Good question. For me personally it started because I was using the Love2D framework for games, and I became very impressed by the speed and conceptual simplicity of Lua.
But there were still a few remaining annoyances despite its simplicity; the statement/expression distinction, the ease with which you can accidentally create a global, and the lack of arity checks in particular were the only complaints I had with it.
It wasn’t enough for me to do anything about, but when I found Fennel, I realized that creating a compiler that targets Lua doesn’t need to be a big deal. The source for the compiler is under 2kloc. Fennel introduces virtually zero runtime overhead, and the compiler output (with a few exceptions like pattern matching) looks very close to the input.
Thanks for response. 2kloc? That’s impressive! That’s in bootstrapper territory.
Love2D looks neat. The first thing that jumped out is it runs on every major platform. Examples look easier than some GUI toolkits from long ago. So, my first thought is: “Could someone use this as a cross-platform, GUI toolkit with simple language to make great-looking interfaces?”
Yeah, actually 2k isn’t just the compiler; it includes the stock macros and the repl.
Love2D is much easier than any other GUI work I’ve done on any platform, yeah. If you don’t need native widgets or the ability to display more than one window it could certainly be used for non-games. In fact, I created a text editor in about 1.2kloc which is a mix of Fennel and Lua: https://git.sr.ht/~technomancy/polywell
It’s loosely based on Emacs and has syntax highlighting, a buffer switcher, support for repl buffers, completion, tetris, etc.
I’ve also had some success using imgui using the love2d bindings. It’s got a lot of handy UI widgets - I used it for game debug UI, but I could see it being used for non-game apps as well.
Are you the same technomancy who created the Atreus keyboard?
Yep, that’s me!
https://atreus.technomancy.us
I’m thinking about buying some yarn and a hook and trying to learn crocheting. I’d like to make myself a nice winter hat, we’ve got some really cold weather coming and I lost my last nice one.
Could someone help me understand what the use of this is?
It is a font designed for use as placeholder text, but without actually displaying any text. This is useful because often non-designers will get hung up on the details of text or (if lorem ipsum is used) wonder what’s wrong with the design that it has all this gibberish in it.
I think it’s pretty clear on the site