Use the .sh (or .bash) extension for your file. It may be fancy to not have an extension for your script, but unless your case explicitly depends on it, you’re probably just trying to do clever stuff. Clever stuff are hard to understand.
Google’s style guide recommends .sh for library files (imports) and no extension for executables, which imo is a good rule and not overly “clever” at all. If you write an executable utility in go, you name it foo, not foo.go, because the client of an executable cares about it’s behavior, not its implementation language.
I agree. It’s also odd to have file endings in any bin directory. Why would you care what language a piece of software is written when you run it. I think if we renamed everything that is a shell script - a lot of software launchers for example it would be really odd. Think /usr/bin/firefox.sh.
It’s even kind of an anti pattern right? Now if you decide to change the implementation language (like replace a large shell script with a Go executable), you’d either need to suffix the go executable “.sh”, or hunt down every use site and change the suffix at every callsite..
I think I don’t really agree with this rule. It’s true that for compiled languages the executable typically doesn’t have an extension on most Unix-y systems, but programs written in interpreted languages, and especially scripts written in traditional Unix-y “scripting languages”, often have had extensions – .sh, .pl, .py, and so on – and I do like the fact that the filename gives me a clue that I’m dealing with a script/scripting language and also what interpreter is likely to be involved in running it.
But I also feel like I’ve lost that fight over the past couple decades, especially as distros started moving to and enforcing the “no extensions” rule, including just renaming scripts when packaging them (which used to cause some issues for Django – the official documentation would tell you to run django-admin.py to start a new project, but on a distro package of Django that would fail because the distro had renamed it to just django-admin). I still typically do put the extensions on scripts in my own projects, though.
Slightly strange that a lot of this is about portability, but still prefers bash to POSIX sh. Even #!/usr/bin/bash scripts could be improved by writing them using as few bashisms as possible, such as using [ or test rather than [[ - all of which are bash builtins.
The TRACE thing seems unnecessarily more complicated than running sh -x script.
Even #!/usr/bin/bash scripts could be improved by writing them using as few bashisms as possible, such as using [ or test rather than [[ - all of which are bash builtins.
Note: this is a different question than “Should portable scripts prefer bash to POSIX shell?” My point is that scripts with bash in the shebang should absolutely use bash features.
I get your point. My thoughts were that when I come across a bash script, I can still run it as a normal shell script like sh script and the fewer bashisms it contains, the easier it is for me to use (I don’t have bash installed).
But that is the diametrical opposite of portability.
Right, but as I said, “Note: this is a different question than ‘Should portable scripts prefer bash to POSIX shell?’ My point is that scripts with bash in the shebang should absolutely use bash features.”
The person I was replying to said “Even #!/usr/bin/bash scripts could be improved by writing them using as few bashisms as possible.” I disagree with that, and I tried to make it clear that I wasn’t talking about portability per se.
I find this idea that something “improves” by using a less capable shell nonsensical. What exactly improves except that you adhere to some standard nobody cares about. This purism buys you nothing. If you know that the target systems have bash, then use all its features.
The first problem is that POSIX shell is an absolutely terrible scripting language, whereas bash is a merely awful one. If you reach the level of complexity where POSIX shell makes it hard to maintain then you are almost certainly at the point where you should ditch any kind of shell and use a programming language.
The second problem is the monoculture. For example, The shell shock vulnerability was so bad because everyone used bash for the scripts that were run from dhcpd and so a malicious user on the same broadcast domain could compromise any system that sent a dhcp request. This would have been far harder to exploit at scale if these scripts used /bin/sh and different systems used different implementations (mainstream systems use at least five different implementations of their default POSIX shell).
The third problem is performance. Bash is definitely not the fastest shell. FreeBSD uses the statically linked version of their POSIX shell for a bunch of things because there’s a noticeable speed up from things that run a load of short-lived shell scripts. I believe this was also part of the motivation for Ubuntu to use dash as /bin/sh instead of bash. If you require a specific implementation with many non-standard quirks then you can’t move to a different implementation.
And I say all of this as someone who uses bash as their interactive shell.
Use bash. Using zsh or fish or any other, will make it hard for others to understand / collaborate. Among all shells, bash strikes a good balance between portability and DX.
I disagree. If you care for portability, and for example resource constrained environments use POSIX shell. If you don’t use something you are familiar with and does the job. Of course that might very well be bash. With zsh, fish and others popping up as main shells people use I think it makes sense to either use a standard for portability (then go for POSIX which is made for portability), but don’t just assume that bash will be the thing that’s installed or most easily available. And I think the trend is certainly going away, though very very slowly. Feel free to use it though, just like you use any other scripting language. It’s just that if you think about portability why not actually go for it?
Just make the first line be #!/usr/bin/env bash, even if you don’t give executable permission to the script file.
I completely agree on this one. It both provides portability and flexibility, for example when a situation arises where you want to for whatever reason need to run it with a different bash. I also think it’s a good indicator of whether the author of a scripts knows what they were doing. There might be exceptions though, similar to other full path situations (so scripting languages), but there should be a good reason. If you don’t have one or when in doubt use #!/usr/bin/env for bash, python, ruby, etc.
Use the .sh (or .bash) extension for your file. It may be fancy to not have an extension for your script, but unless your case explicitly depends on it, you’re probably just trying to do clever stuff. Clever stuff are hard to understand.
I disagree, because that would theoretically mean you’d have to append .sh to a big chunk of what is in /bin/ and /usr/bin. For example /usr/bin/firefox.sh. Also renaming all ./configure to ./configure.sh. Why does the caller need to know what language a piece of software is programmed in?
Use set -o X at the start of your script.
I wished these would become part of POSIX.
Use [[ ]] for conditions in if / while statements, instead of [ ] or test.
I disagree. Don’t reduce portability, especially when you might not need what you gain from it.
I disagree. If you care for portability, and for example resource constrained environments use POSIX shell.
Completely agreed. bash is the default shell on GNU/Linux systems, but not on *BSD, MacOS, or most embedded Linux systems. Worse, on macOS, it is installed, but only the last GPLv2 version, so newer bashisms won’t work. All of the other shells are available as extra packages on most of these systems, so if you’re comfortable writing a shell script that requires someone to install an optional package then zsh is no worse than bash.
As I recall, they install dash as sh, but still have bash installed by default. On BSD systems, bash is an optional install and a lot of folks who want a non-default shell will go with something else.
I have a better shell script best practice - just use and write POSIX /bin/sh compatible scripts - with #!/bin/sh in the first line. They work everywhere. Always.
Nine, twelve, and thirteen are the ones I don’t disagree with. 13 is using long options since they’re easier to read, but they can make one-liners very tangled so you’ve got to case-by-case it.
I’d actually wanna argue for sh (for example dash) over bash or zsh. I use zsh as my interactive shell and so I write zsh functions so I can just source them in. If I write a stand alone shell script, that’s when I’d go for sh first.
I prefer using short options when composing commands interactively, but I prefer long options when composing long lived scripts, as the latter provide better readability as to the intent of arguments. For example grep’s -l vs --files-with-matches
Yes, that’s the general rule, with the exception being things where the short version is just so well known that the long version is almost obfuscating.
Personally, my first rule is “Don’t”.
But the case is that I don’t really write scripts for random people in the internet. I write them for me and my team, so writing logic in our main lang (php in this case) allows more people to contribute, and it’s easier to understand for most.
Even Debian recommends to use python instead if possible.
I do have bash scripts around, but I try to only do it for trivial stuff like aliasing a simple pipe command.
All together a pretty sensible list. I would add shfmt to keep your editing sane. For number (7) another option would be to add a -x <FILE> cli arg in combination with BASH_XTRACEFD and allow a person to write trace output to a file. For number (8) I would perhaps provide some examples of how [[ is more ergonomic and powerful than [, for example:
if [[ -f $f1 && (-f $f2 || -f $f3) ]]; then
echo 'found files'
fi
if [ -f $f1 -a \( -f $f2 -o -f $f3 \) ]; then
echo 'found files'
fi
A good checklist of things to consider. I have a take on this as well, more obsessed with the safety aspects, and I reach many of the same conclusions: Safe ways to do things in bash
Thumbs up for quoting and arrays, and therefore, bash! My take is that posix shell lacks arrays (except the one via set). And you need arrays to be able to quote all the things (there is no such thing as careful use of word splitting, as that also invokes indirect glob expansion). Ergo is posix shell not practically possible to use correctly, so it disqualifies itself.
Of the common advice I’m less convinced of (not that I disagree, but I feel is a distraction on a list of best practices) is the use of [[ ]] conditions: It’s magic syntax. Where it matters, it looks simpler, but is instead yet another topic to learn that you don’t strictly need. What you need anyway is to grasp normal command syntax, and then, you don’t need this language within the language in order to write conditions correctly. It’s a convenience thing, and it has one unique feature – regex (but not globbing, as case also has that, and not logial operators, as command syntax has that).
hese appear to be best practices for writing scripts within the constraints:
The scripts will be used by multiple users for some time.
Tradeoffs dictate the insane expressiveness of scripts over traditional software tools.
Portability within Linux is an issue.
MacOS paused updating Bash on installations over a decade ago (2007; v3.2 IIRC) when Bash moved to an GPL v3.0 license. This is the GPL working as designed: the legal safety of using it in business decreases with the size of the business. In response, Apple moved to the BSD licensed shell zsh. Bash scripts must conform to 2007 interfaces or give up built-in macOS compatibility. If a later version of Bash is expected, that expectation should be checked.
Still, this is a good article and most readers will accept most of its recommendations. There will be disagreements with best practices, though I think all would agree that it is a good idea to have a practice.
Google’s style guide recommends
.shfor library files (imports) and no extension for executables, which imo is a good rule and not overly “clever” at all. If you write an executable utility in go, you name itfoo, notfoo.go, because the client of an executable cares about it’s behavior, not its implementation language.I agree. It’s also odd to have file endings in any bin directory. Why would you care what language a piece of software is written when you run it. I think if we renamed everything that is a shell script - a lot of software launchers for example it would be really odd. Think
/usr/bin/firefox.sh.It’s even kind of an anti pattern right? Now if you decide to change the implementation language (like replace a large shell script with a Go executable), you’d either need to suffix the go executable “.sh”, or hunt down every use site and change the suffix at every callsite..
I think I don’t really agree with this rule. It’s true that for compiled languages the executable typically doesn’t have an extension on most Unix-y systems, but programs written in interpreted languages, and especially scripts written in traditional Unix-y “scripting languages”, often have had extensions –
.sh,.pl,.py, and so on – and I do like the fact that the filename gives me a clue that I’m dealing with a script/scripting language and also what interpreter is likely to be involved in running it.But I also feel like I’ve lost that fight over the past couple decades, especially as distros started moving to and enforcing the “no extensions” rule, including just renaming scripts when packaging them (which used to cause some issues for Django – the official documentation would tell you to run
django-admin.pyto start a new project, but on a distro package of Django that would fail because the distro had renamed it to justdjango-admin). I still typically do put the extensions on scripts in my own projects, though.Slightly strange that a lot of this is about portability, but still prefers bash to POSIX sh. Even
#!/usr/bin/bashscripts could be improved by writing them using as few bashisms as possible, such as using[ortestrather than[[- all of which are bash builtins.The
TRACEthing seems unnecessarily more complicated than runningsh -x script.I disagree—at least about your specific example. If you write shell scripts that specifically point to
bash, then why not take advantages of the differences between[and[[? (For example, word splitting: https://wiki.bash-hackers.org/syntax/ccmd/conditional_expression#word_splitting.)Note: this is a different question than “Should portable scripts prefer
bashto POSIX shell?” My point is that scripts withbashin the shebang should absolutely usebashfeatures.I get your point. My thoughts were that when I come across a bash script, I can still run it as a normal shell script like
sh scriptand the fewer bashisms it contains, the easier it is for me to use (I don’t have bash installed).But that is the diametrical opposite of portability.
Right, but as I said, “Note: this is a different question than ‘Should portable scripts prefer bash to POSIX shell?’ My point is that scripts with bash in the shebang should absolutely use bash features.”
The person I was replying to said “Even #!/usr/bin/bash scripts could be improved by writing them using as few bashisms as possible.” I disagree with that, and I tried to make it clear that I wasn’t talking about portability per se.
You can also see lollipopman’s comment for further arguments in favor of
[[once you are already using bash.“Portability” these days means “works on OS X and Linux”. Might as well get improvements from
bashin that case.Which version of bash? Don’t know if they still do, but macOS used to ship ancient bash releases because of GPLv3
To whom?
What percentage of people who would execute a script of the type described by this article are not Linux or macOS users, do you think?
I mean, I was being snarky, but that seems to be the case; other systems are routinely ignored, for better or worse.
I’d argue portability these days means that it runs on many different systems.
I wouldn’t call anything in bash an “improvement”…
I find this idea that something “improves” by using a less capable shell nonsensical. What exactly improves except that you adhere to some standard nobody cares about. This purism buys you nothing. If you know that the target systems have bash, then use all its features.
The first problem is that POSIX shell is an absolutely terrible scripting language, whereas bash is a merely awful one. If you reach the level of complexity where POSIX shell makes it hard to maintain then you are almost certainly at the point where you should ditch any kind of shell and use a programming language.
The second problem is the monoculture. For example, The shell shock vulnerability was so bad because everyone used bash for the scripts that were run from dhcpd and so a malicious user on the same broadcast domain could compromise any system that sent a dhcp request. This would have been far harder to exploit at scale if these scripts used /bin/sh and different systems used different implementations (mainstream systems use at least five different implementations of their default POSIX shell).
The third problem is performance. Bash is definitely not the fastest shell. FreeBSD uses the statically linked version of their POSIX shell for a bunch of things because there’s a noticeable speed up from things that run a load of short-lived shell scripts. I believe this was also part of the motivation for Ubuntu to use dash as /bin/sh instead of bash. If you require a specific implementation with many non-standard quirks then you can’t move to a different implementation.
And I say all of this as someone who uses bash as their interactive shell.
I disagree. If you care for portability, and for example resource constrained environments use POSIX shell. If you don’t use something you are familiar with and does the job. Of course that might very well be bash. With zsh, fish and others popping up as main shells people use I think it makes sense to either use a standard for portability (then go for POSIX which is made for portability), but don’t just assume that bash will be the thing that’s installed or most easily available. And I think the trend is certainly going away, though very very slowly. Feel free to use it though, just like you use any other scripting language. It’s just that if you think about portability why not actually go for it?
I completely agree on this one. It both provides portability and flexibility, for example when a situation arises where you want to for whatever reason need to run it with a different bash. I also think it’s a good indicator of whether the author of a scripts knows what they were doing. There might be exceptions though, similar to other full path situations (so scripting languages), but there should be a good reason. If you don’t have one or when in doubt use
#!/usr/bin/envfor bash, python, ruby, etc.I disagree, because that would theoretically mean you’d have to append .sh to a big chunk of what is in /bin/ and /usr/bin. For example
/usr/bin/firefox.sh. Also renaming all./configureto./configure.sh. Why does the caller need to know what language a piece of software is programmed in?I wished these would become part of POSIX.
I disagree. Don’t reduce portability, especially when you might not need what you gain from it.
Completely agreed.
bashis the default shell on GNU/Linux systems, but not on *BSD, MacOS, or most embedded Linux systems. Worse, on macOS, it is installed, but only the last GPLv2 version, so newer bashisms won’t work. All of the other shells are available as extra packages on most of these systems, so if you’re comfortable writing a shell script that requires someone to install an optional package thenzshis no worse thanbash.Dash was preferred under the hood by many Debian-based systems. It’s POSIX-compatible but not Bash-compatible.
As I recall, they install dash as sh, but still have bash installed by default. On BSD systems, bash is an optional install and a lot of folks who want a non-default shell will go with something else.
They are.
Although, I prefer
set -euxetc.This was originally on pipefail. Edited the point to mean I agree on the options without pasting each. Oops.
pipefail feels like something that should be in POSIX, but isn’t currently.
I have a better shell script best practice - just use and write POSIX /bin/sh compatible scripts - with #!/bin/sh in the first line. They work everywhere. Always.
Nine, twelve, and thirteen are the ones I don’t disagree with. 13 is using long options since they’re easier to read, but they can make one-liners very tangled so you’ve got to case-by-case it.
I’d actually wanna argue for sh (for example dash) over bash or zsh. I use zsh as my interactive shell and so I write zsh functions so I can just source them in. If I write a stand alone shell script, that’s when I’d go for sh first.
I prefer using short options when composing commands interactively, but I prefer long options when composing long lived scripts, as the latter provide better readability as to the intent of arguments. For example grep’s
-lvs--files-with-matchesYes, that’s the general rule, with the exception being things where the short version is just so well known that the long version is almost obfuscating.
Personally, my first rule is “Don’t”. But the case is that I don’t really write scripts for random people in the internet. I write them for me and my team, so writing logic in our main lang (php in this case) allows more people to contribute, and it’s easier to understand for most.
Even Debian recommends to use python instead if possible.
I do have bash scripts around, but I try to only do it for trivial stuff like aliasing a simple pipe command.
All together a pretty sensible list. I would add shfmt to keep your editing sane. For number (7) another option would be to add a
-x <FILE>cli arg in combination withBASH_XTRACEFDand allow a person to write trace output to a file. For number (8) I would perhaps provide some examples of how[[is more ergonomic and powerful than[, for example:Also you get regex and globbing.
A good checklist of things to consider. I have a take on this as well, more obsessed with the safety aspects, and I reach many of the same conclusions: Safe ways to do things in bash
Thumbs up for quoting and arrays, and therefore, bash! My take is that posix shell lacks arrays (except the one via
set). And you need arrays to be able to quote all the things (there is no such thing as careful use of word splitting, as that also invokes indirect glob expansion). Ergo is posix shell not practically possible to use correctly, so it disqualifies itself.Of the common advice I’m less convinced of (not that I disagree, but I feel is a distraction on a list of best practices) is the use of
[[ ]]conditions: It’s magic syntax. Where it matters, it looks simpler, but is instead yet another topic to learn that you don’t strictly need. What you need anyway is to grasp normal command syntax, and then, you don’t need this language within the language in order to write conditions correctly. It’s a convenience thing, and it has one unique feature – regex (but not globbing, ascasealso has that, and not logial operators, as command syntax has that).hese appear to be best practices for writing scripts within the constraints:
MacOS paused updating Bash on installations over a decade ago (2007; v3.2 IIRC) when Bash moved to an GPL v3.0 license. This is the GPL working as designed: the legal safety of using it in business decreases with the size of the business. In response, Apple moved to the BSD licensed shell zsh. Bash scripts must conform to 2007 interfaces or give up built-in macOS compatibility. If a later version of Bash is expected, that expectation should be checked.
Still, this is a good article and most readers will accept most of its recommendations. There will be disagreements with best practices, though I think all would agree that it is a good idea to have a practice.
[Comment removed by author]