I really like this work and it goes to show that a lot of places where people use C are not because they need to be fast, close to the metal, or unsafe: they just need to tightly interop with code that has those requirements. There’s a huge amount of code in any monolithic kernel that is not performance-critical (though often is safety- and security-critical), that doesn’t need to do anything that would violate type safety in a type-safe language, but ends up being written in C. There’s also a load of code doing things the wrong way in C because it was prototyped in C and it was too hard to refactor later: Starting with a crappy Lua implementation, getting the algorithms the right shape, and then replacing it with a C/C++[/Rust/whatever] implementation with custom optimised data structures would get you to a much better place quickly.
Yes I agree, that is basically how I’m writing Oil shell, e.g.
In other words bash is 140K lines of C code. But really it’s about 100K lines of “groveling through backslashes and braces one-at-a-time in C”, which is hard to maintain and error prone, and it’s not even that performance sensitive. (bash has basic bugs in parsing in 20+ year old language constructs.)
I focused on the algorithms first and then I was able to speed it up by translating to C++. If you look at the C++, it looks far from “optimal” – not what you’d write by hand – yet it ended up faster than bash and much faster than zsh.
I believe that’s because nobody can write 100K lines of C optimally. You always end up making compromises and making a mess. Writing 1K lines of C optimally is easy (and pretty fun in its way), but writing 100K lines is close to impossible. You need multiple hands in the program at that point. (Rare execptions like sqlite took over a decade to reach 100K lines.)
And kernels have that property too. There’s a mass of code that does pretty basic/repetitive things and is not performance sensitive.
So I wish there were a language that’s tightly integrated with C (and C++) but that’s not so tedious. Most languages have FFIs which are not that pleasant, or wrapping tools that are OK for common cases, but fall down for complex things.
Having used Lua as the ‘safe’ keyword for C code since the early 2000s, I can (mostly) only agree. There are some ergonomics that are troubling (repeatedly needs to say to myself ‘now it’s base + offset’ and ‘now it is index’ when switching between the two), I would also happily take a reduced version that deals away with metatables and that jazz, as well as a switch to same:ish symbols and syntax. With the fragmentation caused by LuaJIT and some of the other decisions post 5.1 - I am now somewhat inclined to consider a secondary position.
Ideally this means that one can still use the C software in the Lua fashion, but to elevate security (seccomp/pledge with stricter profiles) / stability (attach / detach the high-level side) / debugability (not mixing language runtime semantics and stdlib in the same process) and language choice.
Later presentation (pdf) from 2015. I liked the 2012 slides better as an intro, though.
Does anyone know of anyone who used it for some practical purpose (incl. scientific)? BSDs are already quite niche so hard to stumble upon something while not being in the community, and I would guess driver authors/kernel hackers are a further sub-niche of NetBSD users, and then Lua users among them are even more niche… (vs. Ye Olde Plain C users)
I came across this, Fast, Flexible Packet Filtering - Lua Kernel Scripting in NetBSD