In Lil, 0 is “falsey”. A common situation where one might reference an undefined variable is referring to a deck-part (Card or Widget) that may or may not exist. It is useful to be able to check for the existence of such parts directly, via truthiness, rather than by first consulting their container, and in many cases it is also useful to attempt modification of widgets with the understanding that if they do not exist, the writes will be ignored. Along similar lines, it is useful to send events at deck-parts irrespective of whether they or their ancestors contain a corresponding handler; if one doesn’t exist now, it might be added later.
Having tables as a first class datatype is really interesting! I started implementing something similar for JS after reading this. How are you querying them? I’m embedding sqlite and hooking it up to the GC.
Both Lil interpreters are self-contained and don’t rely upon any “external” database.
This is an essential part of how the query language works: you can define an ordinary Lil function and use it as an aggregation, or as part of the column expression for any clause. The fact that you can evaluate expressions in terms of columns or scalar values falls out of Lil having APL-like implicit iteration baked into primitives. The only “magic” is the way the (sub)table table columns are bound as local variables in the context of column expressions.
How do you feel about the less strict parts of the design of Lil?
automatic type conversions
e.g. “When a number is required, strings are parsed, and lists or dictionaries attempt to convert their first element. Otherwise, the number 0 is used.”
“referencing unbound variables returns 0”
“extra arguments [to a function] are ignored, and missing arguments are bound as 0”
When indexing a value, “an element of an empty string, list, dictionary, or table is the number 0.”
I certainly wouldn’t want to write my Corporate Job Million Dollar Code in Lil. I think it has a lot of features that are bad for Software Development, but very good for Writing Code on Saturday For Fun. But mostly, I think that avoiding null and substituting false is a pretty good failure case for most of the stuff I want to put together in Decker and Lil, which is largely “Slapped together GUI for a random web API” or a home built recipe manager that I only expect myself to use.
As the designer I’m biased, and I can’t speak for anyone else’s experience. I do, however, actively use Lil on a daily basis. Lil’s “looseness” frequently allows code to focus exclusively on the “happy path” and elide error handling and branching, making it simpler, more concise, and more robust to a dynamically changing environment. Keep in mind that Lil is primarily designed for writing short scripts that run in Decker while a deck is actively being modified and iterated upon!
Consider this example:
deck.cards..widgets.visited.value:0
This works fine on cards which do not have a widget named visited, since an invalid index into widgets returns 0, an invalid index into 0 is still 0, and an assignment through an invalid index of 0 is harmless.
If we lifted the code for marking visited flags to a deck-level script (which, outside a function definition, would execute along with any event handler),
deck.card.widgets.visited.value:1
It would, again, function just fine on cards that don’t have that widget. A more explicit formulation,
if "visited" in deck.card.widgets
v:deck.card.widgets
if "button"~typeof v
v.value:1
end
end
Is longer, but doesn’t meaningfully improve the robustness or clarity. Programming in-the-small benefits from different tradeoffs than programming in-the-large.
Those are the types of misfeatures that made me rage-quit JS long ago (until I discovered TypeScript). They seem helpful, but by hiding so many common mistakes — typos, forgetting an argument, expecting the wrong type — the program just keeps calm and carries on, and you’re faced with inexplicable behavior far downstream of the bug.
(The only form of this I will defend is Obj-C’s behavior of making messaging a nil pointer return nil — this is a quite useful special form of optional propagation that I think saved me a lot of time & complexity in my code and only rarely obscured bugs.)
Sooner or later I will actually mess around with Decker and Lil, and I definitely appreciate this style of language introduction. One thing that wasn’t readily apparent from reading this: why does the example each loop on the dictionary return the listed modified dictionary? I have to reason through a couple of steps:
A block evaluates to its last expression
each returns the iterated collection with the value at each index replaced by the value the block evaluates to (like map elsewhere)
show…prints all its arguments but evaluates to the first argument?
I think the each example calling show for its side effects and for its value made me think harder than I expected to. Perhaps that’s good, not bad.
Wait a minute : does show actually updates the v variables or is the comment wrong ? Very confusing
# the "each" loop iterates over the elements of a list, string, or dict:
each v k i in ("a","b","c") dict 11,22,33
# up to three variable names: value, key, and index.
show[2*v k i] # show[] prettyprints values to stdout.
end
# prints:
# 22 "a" 0
# 44 "b" 1
# 66 "c" 2
# ...and returns the dictionary {"a":22,"b":44,"c":66}.
And the Lil manual doesn’t explain how to define varadic functions or what their semantics are, so I don’t know if this is a bug or a weird behaviour or what.
I love tables/sql as a first-class language construct. How deep does SQL support go? Can we do stuff like GROUP BY, different kinds of JOINs, etc?
Also unrelated Decker question: I really love the simple prototyping / play nature of it, but am not crazy about the retro UI. Have you considered making the UI part pluggable/themeable? Or maybe it is already possible?
If you’d like to learn about Lil’s query functionality in more depth, Lil, The Query Language is much more detailed. Lil provides join (natural join) and cross (cross-join) as primitive operators; other types of join could be provided as user-defined functions. The Lil by clause permits grouping.
Decker’s color palette and fonts can be customized (even at runtime), and it is possible to define custom UI components as Contraptions, but it is not themeable per se. Decker’s appearance is a very intentional series of design choices.
But there so far there was no “interior” way to write it in code. But I think we could do it without any syntax, which is nice. It could be something like:
Table people {
cols name age job
type Str Int Str
row Alice 25 Development
row Sam 28 Sales
...
}
Looks like Lil has influence from K and SQL, which I’ll read up on! I’m a bit partial to the dplyr style because it’s more composable than SQL, it works more like Unix pipelines:
SQL reads pretty nicely for short use cases, but I find it gets unwieldy for longer expressions. I often “cut and paste” the dplyr pipelines and it works well.
whyyyyyyyyyy
In Lil,
0is “falsey”. A common situation where one might reference an undefined variable is referring to a deck-part (Card or Widget) that may or may not exist. It is useful to be able to check for the existence of such parts directly, via truthiness, rather than by first consulting their container, and in many cases it is also useful to attempt modification of widgets with the understanding that if they do not exist, the writes will be ignored. Along similar lines, it is useful to send events at deck-parts irrespective of whether they or their ancestors contain a corresponding handler; if one doesn’t exist now, it might be added later.Well, at least it’s a pretty good reason. Thanks!
I’ve said it before but: Beautiful. What a nice little language. First-class relations are something the world needs more of.
Having tables as a first class datatype is really interesting! I started implementing something similar for JS after reading this. How are you querying them? I’m embedding sqlite and hooking it up to the GC.
Both Lil interpreters are self-contained and don’t rely upon any “external” database.
This is an essential part of how the query language works: you can define an ordinary Lil function and use it as an aggregation, or as part of the column expression for any clause. The fact that you can evaluate expressions in terms of columns or scalar values falls out of Lil having APL-like implicit iteration baked into primitives. The only “magic” is the way the (sub)table table columns are bound as local variables in the context of column expressions.
Oh, so there’s no indexing or anything?
See also Lua’s tables.
Lua and Lil “tables” are very different.
In Lua, a “table” is an associative structure with mechanisms for overriding its indexing semantics.
In Lil, a “table” is a rectangular structure of values with named columns and numbered rows; more like a “table” in a SQL database.
Thanks for the correction! <3
I have fallen in love with Decker and Lil over the last few months. It’s just so, so good.
How do you feel about the less strict parts of the design of Lil?
Are these properties useful to you?
I certainly wouldn’t want to write my Corporate Job Million Dollar Code in Lil. I think it has a lot of features that are bad for Software Development, but very good for Writing Code on Saturday For Fun. But mostly, I think that avoiding null and substituting false is a pretty good failure case for most of the stuff I want to put together in Decker and Lil, which is largely “Slapped together GUI for a random web API” or a home built recipe manager that I only expect myself to use.
As the designer I’m biased, and I can’t speak for anyone else’s experience. I do, however, actively use Lil on a daily basis. Lil’s “looseness” frequently allows code to focus exclusively on the “happy path” and elide error handling and branching, making it simpler, more concise, and more robust to a dynamically changing environment. Keep in mind that Lil is primarily designed for writing short scripts that run in Decker while a deck is actively being modified and iterated upon!
Consider this example:
This works fine on cards which do not have a widget named
visited, since an invalid index intowidgetsreturns 0, an invalid index into 0 is still 0, and an assignment through an invalid index of 0 is harmless.If we lifted the code for marking visited flags to a deck-level script (which, outside a function definition, would execute along with any event handler),
It would, again, function just fine on cards that don’t have that widget. A more explicit formulation,
Is longer, but doesn’t meaningfully improve the robustness or clarity. Programming in-the-small benefits from different tradeoffs than programming in-the-large.
Those are the types of misfeatures that made me rage-quit JS long ago (until I discovered TypeScript). They seem helpful, but by hiding so many common mistakes — typos, forgetting an argument, expecting the wrong type — the program just keeps calm and carries on, and you’re faced with inexplicable behavior far downstream of the bug.
(The only form of this I will defend is Obj-C’s behavior of making messaging a nil pointer return nil — this is a quite useful special form of optional propagation that I think saved me a lot of time & complexity in my code and only rarely obscured bugs.)
I’m not the original commentor, but I’m pretty intrigued by unbound variables returning a fairly sane default.
I’d be even more intrigued if it were a little context-specific:
• for addition/subtraction, return 0
• for multiplication/division, return 1
• for used in list concatenation, return an empty list
• for string concatenation, return an empty string
• etc
Sooner or later I will actually mess around with Decker and Lil, and I definitely appreciate this style of language introduction. One thing that wasn’t readily apparent from reading this: why does the example
eachloop on the dictionary return the listed modified dictionary? I have to reason through a couple of steps:eachreturns the iterated collection with the value at each index replaced by the value the block evaluates to (likemapelsewhere)show…prints all its arguments but evaluates to the first argument?I think the
eachexample callingshowfor its side effects and for its value made me think harder than I expected to. Perhaps that’s good, not bad.I was puzzled by this too, but sort of make sense if you read it right to left?
Idk, anyway a more detailed language guide here https://beyondloom.com/decker/lil.html#liltheimperativelanguage
Wait a minute : does show actually updates the
vvariables or is the comment wrong ? Very confusingeachworks sorta likemapin other languages andshowreturns its first argument.run this example
This isn’t clearly documented, perhaps a later release of decker or lil will improve the documentation. Here’s all we get:
https://beyondloom.com/decker/lilt.html
And the Lil manual doesn’t explain how to define varadic functions or what their semantics are, so I don’t know if this is a bug or a weird behaviour or what.
I love tables/sql as a first-class language construct. How deep does SQL support go? Can we do stuff like GROUP BY, different kinds of JOINs, etc?
Also unrelated Decker question: I really love the simple prototyping / play nature of it, but am not crazy about the retro UI. Have you considered making the UI part pluggable/themeable? Or maybe it is already possible?
If you’d like to learn about Lil’s query functionality in more depth, Lil, The Query Language is much more detailed. Lil provides
join(natural join) andcross(cross-join) as primitive operators; other types of join could be provided as user-defined functions. The Lilbyclause permits grouping.Decker’s color palette and fonts can be customized (even at runtime), and it is possible to define custom UI components as Contraptions, but it is not themeable per se. Decker’s appearance is a very intentional series of design choices.
Fwiw I absolutely love your explanation, reasoning, and the prose which expresses it. Just not that old Mac style itself.
Hm very interesting! I like the table literal syntax.
I think we could do it in https://www.oilshell.org/ because we have Ruby-like blocks. We were already planning an “exterior” TSV8 format that’s like TSV - https://www.oilshell.org/release/0.21.0/doc/j8-notation.html
But there so far there was no “interior” way to write it in code. But I think we could do it without any syntax, which is nice. It could be something like:
(this is not likely to happen soon, but if anyone has interest in working on it let me know :-) YSH is meant to have a very expressive but strict syntax - https://www.oilshell.org/blog/2023/06/ysh-sketches.html )
Looks like Lil has influence from K and SQL, which I’ll read up on! I’m a bit partial to the dplyr style because it’s more composable than SQL, it works more like Unix pipelines:
What Is a Data Frame? (In Python, R, and SQL)
SQL reads pretty nicely for short use cases, but I find it gets unwieldy for longer expressions. I often “cut and paste” the dplyr pipelines and it works well.
Kind of a missed opportunity of calling the post “A Lil’ of Lil”