If you need to add customization or scripting to a larger application, what’s your preferred language to do it with?
Every language has its pros and cons. I know the top two answers are going to be Lua and JavaScript, which are both fine choices. Obviously, if those are what you prefer, please say so, but I’m definitely curious to hear about other options.
Fennel has all the lightweight and performance advantages of Lua because it uses the exact same runtime with zero overhead, but it offers a number of additional compile-time advantages like blocking accidental usage of globals, pattern-matching, opt-in arity checks, and user-defined macros: https://fennel-lang.org
I have also been interested in investigating a couple others:
Neither has very good type inference, unfortunately.
Forth! I’m writing an SDL-based game engine/backend/whatever with an embedded FORTH! for scripting.
Check out Fennel! It compiles to Lua, so you get the same level of embeddability by default. It has some other nice perks as well—variables aren’t global by default, you can write macros (such as the pattern matching macro and threading macros that ship with the language), it has lisp syntax that I much prefer to Lua’s, and the compiler is only 2.3k lines of code (including a built-in repl!)
You can even embed a repl into your larger application easily enough to help with interactive development, here’s an example of that being done in a LÖVE game (which admittedly isn’t Lua being embedded into another language, but the method can be adapted to fit the application). Plus, it has a small but very friendly and growing community, and since it compiles to Lua it can make use of any its versions and libraries. I’ve mostly been using it with luajit, which is blazing fast!
At work, we use Python for scripting. That’s led other departments to write their own frameworks on top of what engineering builds. That’s led to a host of problems (details if you really want to know).
I’ve been reading about Dhall, and I’m especially interested in the fact that it is not turing complete!
Many of our problems have come from other departments building their own features that we then have to munge into the real codebase, perhaps Dhall could help?
In short, I’m seeing more places where I want something that’s more than a config format but less than a scripting language. Anyone else thinking about this?
To answer OP, we use Python, it’s good at sewing things together but can easily go wrong without training users.
Random external url call looks like a major security concern.
That’s a good point! Have you read the section about safety guarantees? They explicitly talk about what you said. Do you think those features are enough?
Basically these safety guarantees are not able by default as such they will eventually not be used and will open an hole in several installations. (I don’t expect this format being widedpread thought)
I think TCL is underrated. A syntax similar to shells is great. Well, actually I think TCL is too complex. So maybe a smaller descendant like TH3.
I find languages like Lua or Javascript are too powerful. They get used to write entire applications and that is too much. Embedded scripts should be small and easy to understand. Providing powerful features misleads people into writing larger and complex scripts. If you want to go that way, then I would suggest to extend Python instead.
Another interesting language I stumbled upon recently is SparForte. It allows you to write small scripts (think bash) and when they (inevitably) grow, you can transform them piecewise into Ada. The approach is similar to starting with Python and extending it with Rust where necessary. The advantage is the language is explicitly designed to support that use case.
It’s pretty easy (like on the order of five lines of code) to take a chunk of Lua and sandbox it so it doesn’t have access to any libraries; even stuff in the standard library. The core of Lua is trivial to learn for anyone who has programmed before; you can pick it up in an afternoon.
[Comment removed by author]
Brr. I actually I think it is massively overrated.
Scheme and the “designed to be embedded” flavour Guile is much nicer.
Thanks for the SparForte reference, that looks interesting.
So, not seeing the entry for Lua yet, I’ll allow myself to hereby add it! By the way, I first learnt of the language exactly as part of an app where it was used as an application scripting language. I’m amazed by Lua as a language ever since.
Other than that, I’m not sure if Red is embeddable as of today; but if not, I believe it could become a really interesting option in this area eventually.
Finally, I’m recently learning Nim, and I find it very interesting. As subset of Nim called NimScript can be run in a VM, and can be made embedded in other programs (though it seems it’s not exactly a turn-key solution as of today).
I have been using Lua as well, for about 12 years now. My main reasons are:
The simplicity of the (reference) implementation. Lua is about 16000 lines of C, any experienced programmer can read them and understand how the whole interpreter works. The same cannot be said of most other mainstream languages.
The wonderful language of the design. Nothing in Lua feels like a kludge to me. That is probably because it is designed and implemented by a very restricted core team, and not by committee.
The syntax is not S-expressions. As much as I want to like Lisp, I cannot bring myself to find that syntax pleasant to use in practice.
The ease of embedding it in C and sandboxing it. Some people do not like the Lua - C stack-based API, but to me it is one of the main selling points of the language. I have written bindings for other dynamic languages such as Python and Ruby, and compared to that experience doing the same for Lua is wonderful.
And over the years, I have found an extra reason, which is the community. The Lua community is small but great, maybe in part because it is one of the few mainstream languages that does not originate from the Western block (Europe / USA). Because of that, its community is a mix from people from all over the world, from very different industries, who use the language in very different ways. Going to a Lua-related conference such as Lua Workshop is always a great time.
I think that the fact that Lua feels as well designed as it is is because it has had 5+ iterations with few considerations given to backwards compatability. Which isn’t to dog on it, but to more say that it’s had a lot more time to be refined in ways that would have not flown if an early version of Lua had been shipped in web browsers, like JS was.
Completely agreed.
All great points; but your third point reminded me that the unrelenting simplicity of the language does make it a fantastic compilation target. So Lua doesn’t support a s-expression-based syntax out of the box, but it can be added in roughly 1kloc of code without changing the semantics. Prefer syntactically-significant whitespace for some reason? You can do that in a compiler too! (Moonscript unfortunately does add a bunch of new semantics like the class system, which is a huge misfeature IMO, but it’s readily within the grasp of one person to create a moonscript-lite that’s just syntactic.)
The first two links in “can be made” are the same. Was that intentional?
Nope, sorry! The second one should have been: https://github.com/komerdoor/nim-embedded-nimscript. Thanks for asking!
I too use Lua, but at work, it’s more for LPEG (Lua Parser Expression Grammar). It takes a while to get used to it, but I find it much more expressive and powerful than the Lua patterns (or even regex). Not only can an expression be reused as part of another expression, but the resulting code is its own VM (distinct from Lua) that is geared directly for parsing.
I’m just putting a word in for AppleScript. It’s not so popular now, and even many Apple apps don’t have a good scripting dictionary, but fir a long while it turned Macs into places where you could do things with your stuff, rather than places where you could use apps for tasks developers had thought of.
AppleScript is a grammar without a syntax; rules for putting words together where the words will be defined later. It’s a DSL framework, for your language, my language, Adobe’s languages, Apple’s languages, and so on. This is both good and bad: good because a new application doesn’t need to fit an old paradigm; bad because the developer does need to take some care to make sure that their abstractions are exposed in the way that users would expect or could discover. I tried to script Safari last week, and found that while windows contain tabs, I couldn’t tell each a window’s tabs to do something.
It took Apple a couple of big cats to do Cocoa scripting (Jaguar), and a couple more to do it well (Tiger). By then, we had all got used to OS X apps not being consistently scriptable, and AppleScript being a weird legacy even though it was still supported and staffed inside the mothership.
I still miss ARexx on the Amiga and the interapplication scriptability that it gave me. What could’ve been…
How did you learn about ARexx? I was an amiganaut through the workbench 2-3 times and never found any documentation.
The inventor of AppleScript published his paper in 2006 right when Tiger was just out, but I didn’t read it until 2014, when it was far too late. There’s a lot of good ideas in that paper and AS, but it never really broke away from the “bad attempt at natural language programming” impression everyone got.
I like to make it usable through shell commands.
Maybe wren? I haven’t used it yet, but I’ve been eyeballing it for a project.
I enjoy lua for long time as a scripting language to do various tasks including shell scripting. However, I was tired of it’s syntax in some areas (such as defining closures). Now, moonscript (https://moonscript.org/) comes to the rescue for me. It has a nice tight syntax for function calls (without parens), table/list comprehensions, class/object stuff etc.. It’s transpilation process is very transparent and it can be directly integrated into the full lua eco-system. Although it is version 0.5 - so far I’m very happy with it, regarding stability and it is fun to experiment with it. You can compare moonscript with what ‘coffeescript’ does for ‘javascript’.
I do most of my scripting with Lumo nowadays. It runs on top of Node so you can leverage Js ecosystem while using a nicer language. Here is a bot that cross-posts to Mastodon from various sources as an example of what to expect. For embedded stuff I’d probably go with Fennel.
I find Lispy languages are especially good for scripting tasks because of their powerful macro systems. It’s very easy to create domain specific abstraction using them, and that’s what scripting is all about at the end of the day.
Visual Basic for Applications, of course. You are developing a COM application, no? ;)
If it’s a COM application, are Powershell and the JS and VB implementations that Windows Scripting Host provides also options?
(Sorry if this is a dumb question, I don’t know a lot about how Windows things relate to one another.)
A bit of a joke, actually. Active Scripting is also, AFAIK, deprecated. Not sure how PowerShell would work out for application automation though.
Excel plus VBA were the tools of choice for business-critical applications, err day to day activities, at most Windows shops back when I was in and around them. PowerPoint the key tool for useless activities highlighting useful ones. ’m pretty sure Excel remains as strong. Probably PowerPoint given human nature. I wonder if VBA is still big these days or replaced with something else?
I suspect one coud do programming idiomatically on the spreadsheet environment, no need for an external one like VBA. Spreadsheets are just 2D programs and data, after all.
Oh yeah, that was true then, too. I was just wondering if there was any extra stuff folks still did with VBA or if it dropped out of popularity.
I really like Tcl. It has a Lisp/Forth/Lua character, in that it take one concept (strings, in Tcl’s case) and runs with it — and it runs with it very well! It’s reminiscent of shell, done right and sanely (no mean feat in itself). It can be quite fast, and it interfaces very well with C.
It’s a powerful, elegant language which deserves more attention.
Also, of course, I like Lisp. If you’re willing to reärchitect your app to be like Emacs (a C or other language core driven by Lisp) then there is a tremendous amount of win there.
Dyon for Rust programs. Otherwise, JS via Duktape.
I love the implementation of Duktape but I dislike JavaScript. It’s a dichotomy.
if there is a C api i can bind to, i like ruby for scripting. i’ve even used it to good effect in more heterogeneous projects by exposing functions as command line utilities, and then using ruby to automate those. i like it because it’s an extremely productive language, and because it makes it relatively easy to pull out often-used functionality into higher-level abstractions and libraries.
You might like mruby, a mostly-compatible version of Ruby designed for embedding. It’s written, ARAIK, by Matz himself.
i’ve been keeping an interested eye on it! not had a chance to use it yet, but it’s definitely my top choice next time i need an embedded language.
Works pretty well for what I’ve seen. Hovever, the possibility to exclude some API such as I/O led some software to use it as a sandboxed VM which ended up pretty badly for them. I like Lua for this exact reason, the VM being really simple and made with security in mind allows it to be a pretty good choice to run untrusted script.