https://rash-lang.org

Full disclosure: I wrote it.

The documentation isn’t great, and I need to write a new line editor to improve interactivity. But it’s got a lot of strengths that are rare among shells or unique to Rash.

You should try Oil and let me know what happens! You can run your bash scripts with it, so you don’t have to switch wholesale.

It will give you a “sane subset” of bash. If you’re using ShellCheck, and strict mode, and find that to be incomplete, then Oil will do even more for you.

Some blog posts that talk about Oil’s increased strictness: http://www.oilshell.org/blog/tags.html?tag=real-problems#real-problems

The error handling is much more reliable, AND it even alerts you when you would have problems in bash: http://www.oilshell.org/blog/2020/10/osh-features.html#reliable-error-handling

That is, I took care not to add error handling that would not be present in bash, which would “reverse break” your shell scripts. The improved behavior is all OPT IN.

To get the strict behavior, add this one line to your program:

shopt -s strict:all 2>/dev/null || true

It will still work under bash.

https://www.oilshell.org/release/0.8.5/doc/oil-options.html

A list of the strictness options: https://www.oilshell.org/release/0.8.5/doc/oil-help-topics.html#option (the list is complete, but the docs aren’t)

The latest release is here: https://www.oilshell.org/release/0.8.5/

It’s also packaged in a few places: https://github.com/oilshell/oil/wiki/Oil-Deployments (not up to date, I could use help with this)

If you like lisp, this may be of interest: https://acha.ninja/blog/dsl_for_shell_scripting/

I haven’t used it but it seems amazing. You get a concise syntax and proper data structures

I have been using Raku at work for writing useful utilities with some success. I can easily create a command line interface without any libraries and call shell programs using qqx which I find easier than dealing with bash quoting rules. Also there are some libraries like raku-shell-piping which I’d like to try someday.

The next-generation regexes in particular have saved me a lot of time at work. Our tests runs were printing a lot of output, and it would result in a MB of text so it would be difficult to use Ctrl-F to search for failed tests, so I wrote a tool which parses this output and prints the essential information that a programmer needs to fix the failed tests.

When there’s a script to write, Raku is the first language I reach for. At some point I do intend to learn Perl as well due to the large number of available modules in CPAN, and also because it is easy to interface with Raku using Inline::Perl5.

Regarding performance, Raku is known to be slow, but performance is something that the team is actively working on. If I can complete a task in an hour of hacking up a Raku solution vs 1.5 hours (or more) in a different language, does it really matter if the program takes 5 seconds or 10 seconds to run?

I mostly use shell scripts because it’s almost always available and doesn’t change much; although I don’t really like it, I don’t really know of anything better as most alternatives are:

1. Somewhat cumbersome to quickly write scripts in.
2. Unlikely to already be installed.

Ruby does support running shell commands with backticks:

irb(main):005:0> `grep $USER /etc/passwd | cut -d: -f5-`
=> "Martin Tournoij:/home/martin:/bin/zsh\n"

And general string processing etc. in Ruby is fairly nice as well. I don’t really use it for stuff like this, mostly because of point 2.

Perl can do this as well I believe (and was originally conceived to replace shell scripts), but I never really got to gripes with the syntax. For many purposes, I consider Ruby to be “Perl done better”, but Perl is much more likely to be installed on most systems (and Ruby still has a rather complicated syntax and a bit of a learning curve).

Tcl is nice for interacting with shell commands as well:

exec grep $env(USER) /etc/passwd | cut -d: -f5-

But I never worked much with it, and it suffers from point 2 as well (although it’s a lot more light-weight than Ruby, and easier to install).

There are other shells as well, such as rc and oil, but they’re even less likely to be already installed. Alpine Linux, for example, doesn’t even seem to have packages for either in the standard repos.

Usually python, just because I know it well, and can get results within a predictable timeframe. My personal rule of thumb is when I need something like a hashmap or array – I give up on bash that instant.

However with time I’ve also gotten more comfortable with bash, so often it’s okay to mix and match. E.g. say, you want to find out the amount of free memory

$ cat /proc/meminfo  | grep MemFree: 
MemFree:        23727768 kB

Right, how do we pick out the number? Normally you’d use cut, or awk:

$ cat /proc/meminfo  | grep MemFree: | awk '{print $1}'
 23727768

, but what if you forgot, or need something more elaborate? Well, why not use python?

$ cat /proc/meminfo  | grep MemFree: | python3 -c 'print(input().split()[1])' 
23727768

Not as concise as awk, but you can type it quicker than the time you’d spend googling how to use awk.

Note that you also can use multiline input if you press enter after the opening quote, so even if you need imports etc, it doesn’t have to look horrible. Also if you have some sort of vi mode or a hotkey to edit the command in editor (Ctrl-X Ctrl-E in bash), it helps a lot for messing with long shell commands.

I also tried using xonsh a few times, a shell combining python & bash syntax. Cool idea, but I tend to forget how to use it, so never got into the habit.

For me, Julia recently completely replaced bash&Python for shell scripting. There are two reasons for that:

• it does “run an external program” thing correctly, unlike, eg, ruby: https://julialang.org/blog/2012/03/shelling-out-sucks/
• the rest of the language is nice for data manipulation

Of course, you need to have Julia installed. It also starts a bit slower than Python. This used to be a deal breaker, but was mostly fixed in one of the versions, it is now fast enough for short interactive scripts.

Here’s a example from my dot files: a script to remove merged branches:

https://github.com/matklad/config/blob/master/bin/gbda

sh is a cute-sy interface for shell scripts in Python. Examples from the docs:

sh.git.show("HEAD")
sh.wc(sh.ls("-1"), "-l")
p = sh.find("-name", "sh.py", _bg=True)
# ... do other things ...
p.wait()

Terra. Lets you call C library functions with a simple include. Example from my own code:

terralib.includec("signal.h").sigsuspend(nil)

After 20 years of avoiding it, I’ve found myself using Perl again a bit for this kind of problem.

I have yet to adopt it, but execline is a possible alternative to traditional shell utilities.

Bash, but as soon as the logic seems to get more complex, I use Python.

Lately, I’ve been using Go for small command line tools as well. This flag library is essential since the inbuilt flag library is wonky. It’s fast to compile, nice to easily make API calls in parallel, parse/spit out JSON.

I really like rc (plan9/v10 unix shell). It’s not too surprising compared to bash and fixes a lot of the issues I have with it. The only downside to it is that interactive execution is lacking compared to more modern shells.

If the Bourne shell frustrates you … welcome to the club. It frustrates a lot of people.

Some of your frustration is probably rooted in the process model itself, the operating system, rather than in BASH. Pipes. Exit codes. Signals. Environment variables. Scattershot argument and output conventions. Languages don’t really solve this mess, so much as hopefully expose it faithfully, so you can try and manage it yourself. The language determines what abstractions you can use, and use conveniently, to try and do that. In my experience, those that try to “hide” the complexity below usually fail to expose it completely. The popular scripting languages have all figured it out, eventually.

Frankly, if you’re doing something delicate, or need it to be more reliable, there’s nothing wrong with #!/usr/bin/python or #!/usr/bin/ruby or #!/usr/bin/node or the like. Yeah, there are efficiency and startup time trade-offs. But if you’re on a desktop, laptop, or server, the cost of one interpreter probably ain’t gonna break you.

For shell scripts on my local machine I use https://elv.sh, a shell with a rather neat scripting language that borrows from all kinds of languages.

ruby - I’ve mostly moved on to other languages for full fledged applications but for small scripts and exploratory coding it remains unmatched.

To be honest, they’re is not “better” as gluing together programs will always give you bullet for each toe. Try to provide a consistent interface between your ad hoc script so weird stuff pops up fast.

Previous work in research, we tended to use either bash or python depending on who set-up the glue and the project.most of the time,the ideas was to provide a cli with a few arguments to run the chain.

During a time, I looked at [Common Workflow Language] (https://www.commonwl.org/) to make the glueing more accessible and the main dev there wanted to create a GUI based on node to plugs together (like Node Editor in Blender) so any researcher can create workflow and document it. All of that had to play nice with slurm. I also provided a few glue script with a simple config file. Maybe stick to any task runner you like to provide an common interface to link everything.

Nowadays, learning Raku, I think I may use it a lot of needed in the future. Probably Go if I need to ship a binary somewhere. Python if I can’t install anything in the env. I will avoid Bash because I am less confortable with it when things got more complicated. However, for short pipes when you have one entry point and everything follows until the final output, Bash is the easiest.

It sounds like you would benefit from a workflow runtime. I really like snakemake because it is Make-like but allows dropping Python code to fit what’s missing.

Lua and shell script make for amazing glue languages

Just wanted to say that I am super-grateful to everyone who chimed-in. There are a lot of useful ideas here; thank you!

I use bash.

When things get too complex, I do turn to a scripting language. Perl is useful and almost optimal for the task, but some of its structures around abstraction are (were?) lousy. Python is decentish. My gut feel is that Ruby is the right language here, but dinking with ruby libraries is yet another task I have to deal with, and I haven’t mustered the energy to care.

In time, I actually start rewriting the internal components into the scripting language so there’s a cohesive system and without concern about the interop between supervisor and component. Things like “write file, read file” are slow and require essentially a protocol.

I don’t know what your components look like, but the number of different technologies gives me pause; it seems that you’re risking some serious bugs due to the interop issues.

For myself, I have started moving away from the idea of files and pipelines for interesting work and data processing and prefer to use Postgres SQL if feasible. It has guarantees around writing and concurrent access that can really help. That might be more than what you can do due to third party software and so forth.

Ruby is the glue that never sets.

I started using Ruby in 2001 to replace a bunch of shell scripts in a really complex build system and I loved it. I rode it through all the popularity of rails, and the decline as the world became more polyglot, people decided Javascript on the server was ‘good enough’, and Python won in the data space… but for scripting, gluing, command line apps, etc. Ruby is still the first tool I reach for.

Where I work at the moment, the choices are essentially bash or node, with TS or JS. I’d personally be happy with say Python or Ruby but I work in JS because every programmer here knows it.

I despise bash as a programming language and try to minimise the use of it.

For most glue like things things, I use Node with plain JS, but usually make everything synchronous. Editing files is fs.readFileSync() or fs.writeFileSync(). Running commands is child_process.execFileSync().

If anything has to be async, I write async function main() { try { ... } catch(error) { console.error(error); process.exit(1); } main(); because promises are okay but callbacks are hell.

Main reason for not using ts is that the benefits are limited for glue code (so many unstructured bags of bytes) so I’d rather just have the shorter iteration time from not invoking tsc.

I prefer cli one-liners where possible. These can get complex, so I save them in all-purpose notes file relevant to the projects I’m working on. If it is general purpose and not restricted to a project, I’ll see if I can get away with alias/functions. Otherwise, see if I can manage a bash script.

If the above is not suitable, only then I use Python/Ruby. For such cases, I try to avoid shell commands and use built-in features as much as possible.

A different perspective: this may not be a language problem. Of course it depends on what you mean by shooting yourself in the foot. For example, maybe you are accidentally deleting files, or running long-during pipelines that end up crashing because there were silly mistakes in the initial inputs. How would a different language help? You can delete files in any language, and you can skip input validation in any language. Although to be fair perhaps it’s easier to do input validation in another language. Still, what makes Bash so productive in general is maybe also the thing that makes it so hazardous: it’s really easy to compose any set of programs with a few keystrokes. I personally haven’t found a replacement I like more for this purpose.

Some programs offer flags for doing dry runs. What if you added something like that to your pipeline steps?

I usually reach for Bash to start, and if it starts to get complex then migrate into Ruby. Depends on the application or target audience though. If I’m writing a generic cli tool and Bash isn’t the correct fit then I’d likely reach for Go over Ruby, just because deploying it as a single binary is simpler.

Inside applications (at $work mostly) I stick to ruby scripts inside the rails apps, and have been using JS scripts inside the couple of Node apps I’ve touched. Infrastructure repos tend to get bash or ruby, just because I know those folks are on a Mac and have Ruby > 2.5 available with the standard library present.

It depends on what I want to „glue“ together.

a) commands/programs – I use Bash – or a shell in general, but usually it is Bash.

b) C-libraries – I use C++ or Java (especially with JNA it is quite convenient). It provides me type safety (not perfect but reasonably good and much better than dynamic/scripting languages). RAII and exceptions in C++ helps a lot with closing resources and error handling. The same does try-with-resources (AutoCloseable) in Java. The D language also seems to be a good option for this use-case (integration with C or even C++).

One more note on Shell/Bash: this glue work quite well, but there are several pitfalls (see Classic pipeline example). They however arise rather from not-well specified text streams, than the language itself.

For me usually the pain is in the lack of multi-threading (which bash can do if you really try, but it’s clumsy), so when I can’t patch things up with parallel or by wrapping it in make -j, I reach for Node.js.

• it has quick startup,
• it’s portable (and really portable, not just “with some care you can make it run on two Unix flavors, and hope it works on the third.”)
• it’s mainstream enough that other people can be expected to have it or be able to install it,
• it has access to tons of dependencies, so I can do complex things if I need to, and INSTALL instructions don’t get more complicated than running npm i

Around 2013, I’ve started working on Next Generation Shell. Frustrated with bash and Python, which I was using extensively, I looked at the niche of what’s on intersection of modern and for Ops. I’ve seen nothing.

While bash was definitely for Ops, it did not meet any expectations for “modern”: horrible syntax, poor error handling and missing (OK, very limited) data structures. Using jq from bash felt like a symptom.

On the other hand there was a bunch of general purpose programming languages such as Python, Ruby, etc. Well .. they are not specifically for Ops and in practice that means that dealing with files and processes is unnecessary verbose. In NGS for example, if you run a process, it’s an exception if it exits with an error exit code. Note that error exit code is not anything above zero. There are commands such as test where exit code 1 is fine.

Over the time I’ve discovered several projects in that modern-for-Ops niche. Somehow the feeling is that they are trying to improve over existing shell while NGS is based on the idea of creating a programming language for ops … with a shell.

PowerShell was not on my radar for some time but when I did look at it I’ve seen horrible language design choices such as -eq comparison which is not equality comparison.

But if you need to determine whether a variable is $null, you must put $null on the left-hand side of the equality operator. Putting it on the right-hand side does not do what you expect.

(Emphasis mine).

Back to NGS.

Project status: useful programming language which we use at work; starting working on the shell.

I hope that other people will find NGS to be a helpful tool.

For anything parallel which would otherwise be done in bash, (gnu) make is great. With 4.3, it now has support for grouped targets, so stamp files can (mostly) finally die.

For everything else I just use python :)

I have been meaning to learn perl simply because of the vast amount of existing scripts written in it.