This is why we can’t have good software. This program could literally have been an empty file, a nothing at all, a name capturing the essence perfectly.
I’m not sure I could disagree more strongly. An empty file only has the true behavior because of a bunch of incredibly non-obvious specific Unix behaviors. It would be equally reasonable for execution of this file to fail (like false) since there’s no hashbang or distinguishable executable format to decide how to handle it. At a somewhat higher level of non-obviousness, it’s really weird that true need be a command at all (and indeed, in almost all shells, it’s not—true is a builtin nearly everywhere).
true being implementable in Unix as an empty file isn’t elegant—it’s coincidental and implicit.
I mean, it’s POSIX specified behavior that any file that is executed that isn’t a loadable binary is passed to /bin/sh (”#!” as the first two bytes results in “implementation-defined” behavior), and it’s POSIX specified behavior that absent anything else, a shell script exits true.
It’s no more coincidental and implicit than “read(100)” advances the file pointer 100 bytes, or any other piece of standard behavior. Sure, it’s Unix(-like)-specific, but, well, it’s on a Unix(-like) operating system. :)
It’s precisely specified, yes, but it’s totally coincidental that the specification says what it does. A perfectly-reasonable and nearly-equivalent specification in an alternate universe where Thomson and Ritchie sneezed five seconds earlier while deciding how executables should be handled would have precisely the opposite behavior.
On the other hand, if read(100) did anything other than read 100 bytes, that would be extremely surprising and would not have come about from an errant sneeze.
Black Mirror Episode: The year is 2100 and the world is ravaged by global warming. The extra energy aggregated over decades because non executables went through /bin/sh caused the environment to enter the tipping point where the feedback loops turned on. A time machine is invented, where one brave soul goes back in time with a feather, finds Thomson and makes him sneeze, saving humanity from the brink of extinction. But then finds himself going back to 2100 with the world still ravaged. Learns that it was fruitless because of npm and left-pad.
The current (POSIX) specification is the product of historical evolution caused in part by /bin/true itself. You see, in V7 Unix, the kernel did not execute an empty file (or shell scripts); it executed only real binaries. It was up to the shell to run shell scripts, including empty ones. Through a series of generalizations (starting in 4BSD with the introduction of csh), this led to the creation of #! and kernel support for it, and then POSIX requiring that the empty file trick be broadly supported.
This historical evolution could have gone another way, but the current status is not the way it is because people rolled out of bed one day and made a decision; it is because a series of choices turned out to be useful enough to be widely supported, eventually in POSIX, and some choices to the contrary wound up being discarded.
(There was a time when kernel support for #! was a dividing line between BSD and System V Unix. The lack of it in the latter meant that, for example, you could not make a shell script be someone’s login shell; it had to be a real executable.)
Why isn’t 100 an octal (or a hex or binary) constant? Why is it bytes instead of machine words? Why is read bound to a file descriptor instead of having a record size from an ioctl, and then reading in 100 records?
The opposite isn’t reasonable though. That would mean every shell script would have to explicitly exit 0 or it will fail.
Every. Shell. Script.
And aside from annoying everyone, that wouldn’t even change anything. It would just make the implementation of true be exit 0, instead of the implementation of false be exit 1.
And read(100) does do something besides read 100 bytes. It reads up to 100 bytes, and isn’t guaranteed to read the full 100 bytes. You must check the return value and use only the amount of bytes read.
It’s not obvious to me that an empty file should count as a valid shell script. It makes code generation marginally easier, I suppose. But I also find something intuitive to the idea that a program should be one or more statements/expressions (or functions if you need main), not zero or more.
So if you run an empty file with sh, you would prefer it exits failure. And when you run an empty file with python, ruby, perl, et al., also failures?
Why should a program have one or more statements / expressions? A function need not have one or more statements / expressions. Isn’t top level code in a script just a de facto main function?
It’s intuitive to me that a script, as a sequence of statements to run sequentially, could have zero length. A program with an entry point needs to have at least a main function, which can be empty. But a script is a program where the entry point is the top of the file. It “has a main function” if the file exists.
I think whatever the answer is, it makes equal sense for Perl, Python, Ruby, shell, any language that doesn’t require main().
In my opinion, your last argument begs the question. If an empty program is considered valid, then existing is equivalent to having an empty main. If not, then it isn’t.
In any case, I don’t mean to claim that it’s obvious or I’m certain that an empty program should be an error, just that it seems like a live option.
Exactly. It sounds like arbitrary hackery common in UNIX development. Just imagine writing a semi-formal spec that defines a program as “zero characters” which you pass onto peer review. They’d say it was an empty file, not a program.
I guess true shouldn’t be considered a program. It is definitely tied to the shell it runs in, as you wouldn’t call execv("true", {"/bin/true", NULL}) to exit a program correctly. for example. true has no use outside of the shell, so it makes sense to have it use the shell’s features. That is why now it tends to be a builtin. But having it a builtin is not specified by POSIX. Executing file on the other end, is, and the spec says the default exit code it 0 or “true”. By executing an empty file, you’re then asking the shell to do nothing, and then return true. So I guess it is perfectly fine for true to jist be an empty file. Now I do agree that such a simple behavior has (loke often with unix) way too many ways to be executed, ans people are gonna fight about it for quite some time!
What about these?
alias true=(exit)
alias true='/bin/sh /dev/null'
alias true='sh -c "exit $(expr `false;echo $? - $?`)"'
It depends upon the system. There is IEFBR14, a program IBM produced to help make files in JCL which is similar to /bin/true. So there could be uses for such a program.
It also has the distinction of being a program that was one instruction long and still have a bug in it.
Makes sense. If tied to the shell and unusual, I’d probably put something like this into the interpreter of the shell as an extra condition or for error handling. Part of parsing would identify an empty program. Then, either drop or log it. This is how such things are almost always handled.
That would mean every shell script would have to explicitly exit 0 or it will fail.
I don’t see how that follows.
Once the file is actually passed to the shell, it is free to interpret it as it wishes. No reasonable shell language would force users to specify successful exit. But what the shell does is not in question here; it’s what the OS does with an empty or unroutable executable, for which I am contending there is not an obvious behavior. (In fact, I think the behavior of running it unconditionally with the shell is counterintuitive.)
And read(100) does do something besides read 100 bytes.
You’re being pedantic. Obviously, under some circumstances it will set error codes, as well. It very clearly reads some amount of data, subject to the limitations and exceptions of the system; zero knowledge of Unix is required to intuit that behavior.
You claim the exact opposite behavior would have been equally reasonable. That is, the opposite of an empty shell script exiting true. The precise opposite would be an empty shell script—i.e. a script without an explicit exit—exiting false. This would affect all shell scripts.
Unless you meant the opposite of executing a file not loadable as an executable binary by passing it to /bin/sh, in which case I really would like to know what the “precise opposite” of passing a file to /bin/sh would be.
You’re being pedantic. Obviously, under some circumstances it will set error codes, as well. It very clearly reads some amount of data, subject to the limitations and exceptions of the system; zero knowledge of Unix is required to intuit that behavior.
No. Many people assume read will fill the buffer size they provide unless they are reading the trailing bytes of the file. However, read is allowed to return any number of bytes within the buffer size at any time.
It also has multiple result codes that are not errors. Many people assume when read returns -1 that means error. Did you omit that detail for brevity, or was it not obvious to you?
If a file is marked executable, I think it’s quite intuitive that the system attempt to execute. If it’s not a native executable, the next obvious alternative would be to interpret it, using the default system interpreter.
Saying the behavior is totally (or even partially) coincidental is a bit strong. You’re ignoring the fundamental design constraints around shell language and giving the original designers more credit than they deserve.
Consider this experiment: you pick 100 random people (who have no previous experience to computer languages) and ask them to design a shell language for POSIX. How would all of these languages compare?
If the design constraints I’m talking about didn’t exist, then it would indeed be random and one would expect only ~50% of the experimental shell languages to have a zero exit status for an empty program.
I strongly doubt that is what you would see. I think you would see the vast majority of those languages specifying that an empty program have zero exit status. In that case, it can’t be random and there must something intentional or fundamental driving that decision.
I don’t care about how the shell handles an empty file. (Returning successful in that case is basically reasonable, but not in my opinion altogether obvious.) I’m stating that the operating system handling empty executables by passing them to the shell is essentially arbitrary.
The reason for the existence of human intelligence isn’t obvious either but that doesn’t make it random. A hostile environment naturally provides a strong incentive for an organism to evolve intelligence.
As far as the operating system executing non-binaries with “/bin/sh” being arbitrary, fair enough. Though I would argue that once the concepts of the shebang line and an interpreter exist, it’s not far off to imagine the concept of a “default interpreter.” Do you think the concept of a default is arbitrary?
yes an empty file signifying true violates the principle of least astonishment.However if there were a way to have metadata comments about the file describing what it does, how it works, and what version it is without having any of that in the file we’d have the best of both worlds.
And despite all of this, most shells have true and false as built-in commands anyway…
$ type true
true is a shell builtin
which, imo, gives another twist to the whole debate, since useless changes are made to a “useless” (since to my knowledge the binary itself isn’t used) program. The most it can offer us, is a self-reflective lesson in the Unix ideals and their betrayals.
Shell builtins aren’t used for exec calls and their ilk, no? So if you want to use false as the login shell for a an account, you’d need the “real” /bin/false?
Fair enough, that’s true (pardon the pun). I was only thinking about a shell scripting environment.
On the other hand, who knows how long it will take until systemd or some other do it all system totally takes over account management, making false and true superfluous in that department too.
They’re built-ins on most fully featured interactive shells (bash, zsh, etc.), but on many systems the default shell-script interpreter /bin/sh is a more minimalist shell where they aren’t built-ins. That’s the case on at least Debian (dash) and FreeBSD (ash). So the default for shell scripts on such systems is that true actually does run /bin/true.
The dash on my system (void, and another one running ubuntu and a server running debian(!)) claims that true was a built in command. Before writing my comment, I checked bash, where I already knew it was true, ksh on an OpenBSD system and dash on my laptop. Even the shell on my android phone (I belive the default one in /system/bin/sh) has the two program as built in components.
I haven’t tried ash, but it seems to me that it’s becoming ever rarer, even if a more minimalist shell might theoretically use the binaries directly.
Oops, my mistake, sorry. I had checked before posting that comment, but checked incorrectly. I did this (starting in a zsh shell):
mjn@mjn:~% which true
true: shell built-in command
mjn@mjn:~% ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Jun 28 2017 /bin/sh -> dash
mjn@mjn:~% sh
$ which true
/bin/true
But I didn’t realize that the difference here is due to which, rather than true, going from builtin in zsh to not builtin in dash. Seems that making which a builtin is zsh-specific behavior, and the POSIX way to get info on how a command will be executed is command -V.
I don’t see a lot of value in the changes to to true that Rob complains about.
However, I also don’t see how having your shell scripts depend on an empty file with a particular name, so that you can run that command to get a 0 status code, counts as “good software”.
I don’t suppose there’s a practical problem to doing it that way[0], but imagine you have to explain true to an alien who knows a great deal about programming, but has no background with unix.
[0] I’m tempted to argue that every change in this series of tweets is the predictable consequence of true being a file. So far as you think these changes are bad, you should be bothered by the original decision.
Well, you would have to explain that Alien what unix is and how it works anyway, because true can only be “true” on unix systems.
You could also tell that alien “executing a file on unix will return successfully, unless the program specifies otherwise. An empty file is an empty program and thus does nothing, so it returns successfully”
And he don’t even need his weird alien programming logic ;)
I find it a little disappointing that most of the reactions here are wrapped around the axle on the particulars of true, rather than the (imo) more interesting examination of how complexity creeps into our software design and what that means.
Also, I wonder if people think that they might have something to learn or gain from someone who has a lot experience in our field and has created successful projects! (Note I’m not making an argument from authority – I’m not saying he’s correct or not, just that we might read it charitably and reflect that he might have something valuable to say before reacting strongly!)
If he had something valuable to say, one would hope that his experience would have led him to offer an example of it. Instead we got this, which is essentially “old man yells at cloud.”
This is full documentation of the GNU version of true command:
16.2 ‘true’: Do nothing, successfully
‘true’ does nothing except return an exit status of 0, meaning
“success”. It can be used as a place holder in shell scripts where a
successful command is needed, although the shell built-in command ‘:’
(colon) may do the same thing faster. In most modern shells, ‘true’ is
a built-in command, so when you use ‘true’ in a script, you’re probably
using the built-in command, not the one documented here.
‘true’ honors the ‘–help’ and ‘–version’ options.
Note, however, that it is possible to cause ‘true’ to exit with
nonzero status: with the ‘–help’ or ‘–version’ option, and with
standard output already closed or redirected to a file that evokes an
I/O error. For example, using a Bourne-compatible shell:
$ ./true --version >&-
./true: write error: Bad file number
$ ./true --version > /dev/full
./true: write error: No space left on device
This version of ‘true’ is implemented as a C program, and is thus
more secure and faster than a shell script implementation, and may
safely be used as a dummy shell for the purpose of disabling accounts.
Seems that the binary version of true was probably created so that the system won’t need to load big /bin/sh.
There are no binary files in Unix, and there are no text files either. This is one of the main differentiators between Unix and other systems, especially systems that preceded it, like Multics. In those systems the syntax for dealing with text files is different than the syntax for dealing with binary files. In Unix, there are just files, uninterpreted streams of bytes. Of course there are textual and binary interfaces, but that’s a different thing. The syntax is the same, the semantics differ.
it’s funny how on this website we routinely see people arguing about the inefficient design of fork(), and people who would rather have an empty /bin/true (which would spawn an entire new shell when executed) than implement it in c
This rant makes more sense in context: Rob Pike helped develop and believed in the ‘everything is a file’ paradigm with Plan9 (as borrowed and expanded from UNIX philosophy).
I like being aware of this. Sometimes people try to improve things that are perfectly ok already and when you examine why, it’s usually because they feel that they need to do something. It happens outside of software and engineering too,
Isn’t it better though to have an ELF binary which just returns 0, rather than having to start a shell interpreter every time you want to invoke /bin/true? Also, when every other part of a collection of tools (in this case GNU coreutils) follows a convention (i.e that --version prints version information and --help prints usage information), is it really better that /bin/true is the one binary which doesn’t follow that convention?
This seems like a classic case of making the world a little bit better.
Isn’t it better though to have an ELF binary which just returns 0, rather than having to start a shell interpreter every time you want to invoke /bin/true?
I can see an alternate viewpoint where it just seems like bloat. It’s yet another chunk of source to carry around to a distribution, yet another binary to build when bootstrapping a system, and other holdovers.
Also the GNU coreutils implementation of true is embarrassing. https://github.com/coreutils/coreutils/blob/master/src/true.c is 65 lines of C code and accepts 2 command-line arguments, which means that the binary has to be locale aware in those situations.
Yep, I’d call depending on sh to deal with /bin/true bloat, if only due to the overall extra time to exec() sh in general. Times with a warm cache, not cold. Yes this is golfing to a degree, but this kind of stuff adds up. A minimal binary in my opinion is not worse, its seeing the forest over a single sh tree.
$ time sh -c ''
real 0m0.004s
user 0m0.000s
sys 0m0.003s
$ time /bin/true
real 0m0.001s
user 0m0.000s
sys 0m0.002s
Even that though is gnu true, compared to a return 0 binary its also slow due to the locale etc… stuff you mention:
$ time /tmp/true
real 0m0.001s
user 0m0.001s
sys 0m0.000s
I don’t understand how the C implementation of true is non-portable? I mean, every C compiler and environment I have seen can set main() as an integer, and return 0 to the shell. Can some one explain how the following C code is non-portable?
His complaint is that the executable is not portable, not the source file. The literal file /usr/bin/true on a Linux machine can’t be dropped onto a machine that doesn’t use ELF binaries, for example.
I think this is a really interesting case of Worse is Better, drawn out over years. The empty file solution is clearly Best, but it apparently the business determined it should have comments. Processing comments was not the ideal purpose for the shell; it was too slow! So out comes the C compiler, which of course seems Worse. The Best technical solution is great, but it’s often not great business, especially in the short-term.
I think the real business need was arbitrarily-long comment headers, honestly, and the separate compilation step to avoid the ensuing runtime overhead. Why that need wasn’t laughed aside, I don’t know; I guess you had to be there. They should’ve deleted the comment.
Putting on my “very definitely not a scientist” hat:
$ time for i in `seq 1000`; do true; done
real 0m0.010s
user 0m0.004s
sys 0m0.004s
$ time for i in `seq 1000`; do /bin/true; done
real 0m1.280s
user 0m0.112s
sys 0m0.320s
$ time for i in `seq 1000`; do sh -c ''; done
real 0m1.552s
user 0m0.172s
sys 0m0.292s
$ lsb_release -d
Description: Debian GNU/Linux 8.10 (jessie)
The “very definitely not a scientist” hat has a tag on the inside saying in big red letters “very definitely not for production use”. =) You would indeed need to subshell or something using /bin/time instead of bash’s builtin time, and I made no effort whatsoever to be portable (or to ensure caches were filled or the processor unburdened with background tasks).
As another possibly amusing case:
$ readlink /bin/sh
dash
$ time for i in `seq 1000`; do bash -c ''; done
real 0m2.488s
user 0m0.180s
sys 0m0.332s
Sure, i’ll do a few more for comparison on a SLOW arm box, that’ll make this all the more pronounced. :)
$ for x in ash bash mksh zsh; do
time (for i in `seq 1000`; do ${x} -c ''; done)
done
( for i in `seq 1000`; do; ${x} -c ''; done; ) 0.39s user 0.39s system 2% cpu 35.974 total
( for i in `seq 1000`; do; ${x} -c ''; done; ) 0.08s user 0.71s system 1% cpu 46.198 total
( for i in `seq 1000`; do; ${x} -c ''; done; ) 0.20s user 0.58s system 2% cpu 36.801 total
( for i in `seq 1000`; do; ${x} -c ''; done; ) 0.21s user 0.61s system 1% cpu 44.066 total
$ time (for i in `seq 1000`; do /bin/true; done)
( for i in `seq 1000`; do; /bin/true; done; ) 0.74s user 0.00s system 2% cpu 32.241 total
Lets compare that to a fully static minimal true (no ld.so shenanigans) note, this is a musl libc system not glibc:
$ echo 'int main(){}' > /tmp/true.c
$ make CC=clang CFLAGS=-Oz LDFLAGS=-static /tmp/true
clang -Oz -static /tmp/true.c -o /tmp/true
$ objdump -p /tmp/true | grep NEEDED
$ time (for i in `seq 1000`; do /tmp/true; done)
( for i in `seq 1000`; do; /tmp/true; done; ) 0.74s user 0.00s system 2% cpu 36.887 total
Well that was ostensibly worse, go figure. Lets strip it too to be sure.
$ strip /tmp/true
$ time (for i in `seq 1000`; do /tmp/true; done)
( for i in `seq 1000`; do; /tmp/true; done; ) 0.74s user 0.00s system 2% cpu 36.406 total
¯\_(ツ)_/¯
I would’ve guessed that minimal true would be the fastest of the bunch, but guess not, GO FIGURE. YMMV this is all not scientific at all.
I’m not sure I could disagree more strongly. An empty file only has the
truebehavior because of a bunch of incredibly non-obvious specific Unix behaviors. It would be equally reasonable for execution of this file to fail (likefalse) since there’s no hashbang or distinguishable executable format to decide how to handle it. At a somewhat higher level of non-obviousness, it’s really weird thattrueneed be a command at all (and indeed, in almost all shells, it’s not—trueis a builtin nearly everywhere).truebeing implementable in Unix as an empty file isn’t elegant—it’s coincidental and implicit.I mean, it’s POSIX specified behavior that any file that is executed that isn’t a loadable binary is passed to /bin/sh (”#!” as the first two bytes results in “implementation-defined” behavior), and it’s POSIX specified behavior that absent anything else, a shell script exits true.
It’s no more coincidental and implicit than “read(100)” advances the file pointer 100 bytes, or any other piece of standard behavior. Sure, it’s Unix(-like)-specific, but, well, it’s on a Unix(-like) operating system. :)
It’s precisely specified, yes, but it’s totally coincidental that the specification says what it does. A perfectly-reasonable and nearly-equivalent specification in an alternate universe where Thomson and Ritchie sneezed five seconds earlier while deciding how executables should be handled would have precisely the opposite behavior.
On the other hand, if
read(100)did anything other than read 100 bytes, that would be extremely surprising and would not have come about from an errant sneeze.Black Mirror Episode: The year is 2100 and the world is ravaged by global warming. The extra energy aggregated over decades because non executables went through /bin/sh caused the environment to enter the tipping point where the feedback loops turned on. A time machine is invented, where one brave soul goes back in time with a feather, finds Thomson and makes him sneeze, saving humanity from the brink of extinction. But then finds himself going back to 2100 with the world still ravaged. Learns that it was fruitless because of npm and left-pad.
The current (POSIX) specification is the product of historical evolution caused in part by
/bin/trueitself. You see, in V7 Unix, the kernel did not execute an empty file (or shell scripts); it executed only real binaries. It was up to the shell to run shell scripts, including empty ones. Through a series of generalizations (starting in 4BSD with the introduction ofcsh), this led to the creation of#!and kernel support for it, and then POSIX requiring that the empty file trick be broadly supported.This historical evolution could have gone another way, but the current status is not the way it is because people rolled out of bed one day and made a decision; it is because a series of choices turned out to be useful enough to be widely supported, eventually in POSIX, and some choices to the contrary wound up being discarded.
(There was a time when kernel support for
#!was a dividing line between BSD and System V Unix. The lack of it in the latter meant that, for example, you could not make a shell script be someone’s login shell; it had to be a real executable.)This is true of literally all software specifications, in my experience.
Surely we can agree that it is far more coincidental that an empty executable returns success immediately than that e.g.
read(100)reads 100 bytes?Why isn’t
100an octal (or a hex or binary) constant? Why is it bytes instead of machine words? Why is read bound to a file descriptor instead of having a record size from an ioctl, and then reading in 100 records?Just some examples. :)
Obviously, minor variations are possible. However, in no reasonable (or even moderately unreasonable) world, would
read(100)write 100 bytes.Pass a
mmap’ed pointer toread, and it shall write. :)The opposite isn’t reasonable though. That would mean every shell script would have to explicitly exit 0 or it will fail.
Every. Shell. Script.
And aside from annoying everyone, that wouldn’t even change anything. It would just make the implementation of true be exit 0, instead of the implementation of false be exit 1.
And read(100) does do something besides read 100 bytes. It reads up to 100 bytes, and isn’t guaranteed to read the full 100 bytes. You must check the return value and use only the amount of bytes read.
It’s not obvious to me that an empty file should count as a valid shell script. It makes code generation marginally easier, I suppose. But I also find something intuitive to the idea that a program should be one or more statements/expressions (or functions if you need main), not zero or more.
So if you run an empty file with sh, you would prefer it exits failure. And when you run an empty file with python, ruby, perl, et al., also failures?
Why should a program have one or more statements / expressions? A function need not have one or more statements / expressions. Isn’t top level code in a script just a de facto main function?
It’s intuitive to me that a script, as a sequence of statements to run sequentially, could have zero length. A program with an entry point needs to have at least a main function, which can be empty. But a script is a program where the entry point is the top of the file. It “has a main function” if the file exists.
I think whatever the answer is, it makes equal sense for Perl, Python, Ruby, shell, any language that doesn’t require main().
In my opinion, your last argument begs the question. If an empty program is considered valid, then existing is equivalent to having an empty main. If not, then it isn’t.
In any case, I don’t mean to claim that it’s obvious or I’m certain that an empty program should be an error, just that it seems like a live option.
Exactly. It sounds like arbitrary hackery common in UNIX development. Just imagine writing a semi-formal spec that defines a program as “zero characters” which you pass onto peer review. They’d say it was an empty file, not a program.
I guess
trueshouldn’t be considered a program. It is definitely tied to the shell it runs in, as you wouldn’t callexecv("true", {"/bin/true", NULL})to exit a program correctly. for example.truehas no use outside of the shell, so it makes sense to have it use the shell’s features. That is why now it tends to be a builtin. But having it a builtin is not specified by POSIX. Executing file on the other end, is, and the spec says the default exit code it 0 or “true”. By executing an empty file, you’re then asking the shell to do nothing, and then return true. So I guess it is perfectly fine fortrueto jist be an empty file. Now I do agree that such a simple behavior has (loke often with unix) way too many ways to be executed, ans people are gonna fight about it for quite some time! What about these?The one true
true!It depends upon the system. There is IEFBR14, a program IBM produced to help make files in JCL which is similar to
/bin/true. So there could be uses for such a program.It also has the distinction of being a program that was one instruction long and still have a bug in it.
“That is why now it tends to be a builtin.”
Makes sense. If tied to the shell and unusual, I’d probably put something like this into the interpreter of the shell as an extra condition or for error handling. Part of parsing would identify an empty program. Then, either drop or log it. This is how such things are almost always handled.
I don’t see how that follows.
Once the file is actually passed to the shell, it is free to interpret it as it wishes. No reasonable shell language would force users to specify successful exit. But what the shell does is not in question here; it’s what the OS does with an empty or unroutable executable, for which I am contending there is not an obvious behavior. (In fact, I think the behavior of running it unconditionally with the shell is counterintuitive.)
You’re being pedantic. Obviously, under some circumstances it will set error codes, as well. It very clearly reads some amount of data, subject to the limitations and exceptions of the system; zero knowledge of Unix is required to intuit that behavior.
You claim the exact opposite behavior would have been equally reasonable. That is, the opposite of an empty shell script exiting true. The precise opposite would be an empty shell script—i.e. a script without an explicit exit—exiting false. This would affect all shell scripts.
Unless you meant the opposite of executing a file not loadable as an executable binary by passing it to /bin/sh, in which case I really would like to know what the “precise opposite” of passing a file to /bin/sh would be.
No. Many people assume read will fill the buffer size they provide unless they are reading the trailing bytes of the file. However, read is allowed to return any number of bytes within the buffer size at any time.
It also has multiple result codes that are not errors. Many people assume when read returns -1 that means error. Did you omit that detail for brevity, or was it not obvious to you?
If a file is marked executable, I think it’s quite intuitive that the system attempt to execute. If it’s not a native executable, the next obvious alternative would be to interpret it, using the default system interpreter.
Saying the behavior is totally (or even partially) coincidental is a bit strong. You’re ignoring the fundamental design constraints around shell language and giving the original designers more credit than they deserve.
Consider this experiment: you pick 100 random people (who have no previous experience to computer languages) and ask them to design a shell language for POSIX. How would all of these languages compare?
If the design constraints I’m talking about didn’t exist, then it would indeed be random and one would expect only ~50% of the experimental shell languages to have a zero exit status for an empty program.
I strongly doubt that is what you would see. I think you would see the vast majority of those languages specifying that an empty program have zero exit status. In that case, it can’t be random and there must something intentional or fundamental driving that decision.
I don’t care about how the shell handles an empty file. (Returning successful in that case is basically reasonable, but not in my opinion altogether obvious.) I’m stating that the operating system handling empty executables by passing them to the shell is essentially arbitrary.
The reason for the existence of human intelligence isn’t obvious either but that doesn’t make it random. A hostile environment naturally provides a strong incentive for an organism to evolve intelligence.
As far as the operating system executing non-binaries with “/bin/sh” being arbitrary, fair enough. Though I would argue that once the concepts of the shebang line and an interpreter exist, it’s not far off to imagine the concept of a “default interpreter.” Do you think the concept of a default is arbitrary?
laughs That’s really taking an axe to the sum basis of knowledge, isn’t it?
yes an empty file signifying true violates the principle of least astonishment.However if there were a way to have metadata comments about the file describing what it does, how it works, and what version it is without having any of that in the file we’d have the best of both worlds.
But isn’t this in some sense exactly living up to the “unix philosophy”?
No.
Why is it weird that
trueneed be a command at all?To me, the issue is whether it is prone to error. If it is not, it is culture building because it is part of the lore.
I actually like the OpenBSD version; http://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/src/usr.bin/true/true.c?rev=1.1&content-type=text/plain&only_with_tag=MAIN
vs the GNU version; https://github.com/coreutils/coreutils/blob/master/src/true.c
The GNU version also contains –help and –version handling.
And despite all of this, most shells have
trueandfalseas built-in commands anyway…which, imo, gives another twist to the whole debate, since useless changes are made to a “useless” (since to my knowledge the binary itself isn’t used) program. The most it can offer us, is a self-reflective lesson in the Unix ideals and their betrayals.
Shell builtins aren’t used for
execcalls and their ilk, no? So if you want to usefalseas the login shell for a an account, you’d need the “real”/bin/false?Fair enough, that’s true (pardon the pun). I was only thinking about a shell scripting environment.
On the other hand, who knows how long it will take until systemd or some other do it all system totally takes over account management, making
falseandtruesuperfluous in that department too.They’re built-ins on most fully featured interactive shells (bash, zsh, etc.), but on many systems the default shell-script interpreter
/bin/shis a more minimalist shell where they aren’t built-ins. That’s the case on at least Debian (dash) and FreeBSD (ash). So the default for shell scripts on such systems is thattrueactually does run/bin/true.The dash on my system (void, and another one running ubuntu and a server running debian(!)) claims that
truewas a built in command. Before writing my comment, I checked bash, where I already knew it was true, ksh on an OpenBSD system and dash on my laptop. Even the shell on my android phone (I belive the default one in/system/bin/sh) has the two program as built in components.I haven’t tried ash, but it seems to me that it’s becoming ever rarer, even if a more minimalist shell might theoretically use the binaries directly.
Oops, my mistake, sorry. I had checked before posting that comment, but checked incorrectly. I did this (starting in a zsh shell):
But I didn’t realize that the difference here is due to
which, rather thantrue, going from builtin in zsh to not builtin in dash. Seems that makingwhicha builtin is zsh-specific behavior, and the POSIX way to get info on how a command will be executed iscommand -V.I don’t see a lot of value in the changes to to true that Rob complains about.
However, I also don’t see how having your shell scripts depend on an empty file with a particular name, so that you can run that command to get a 0 status code, counts as “good software”.
I don’t suppose there’s a practical problem to doing it that way[0], but imagine you have to explain true to an alien who knows a great deal about programming, but has no background with unix.
[0] I’m tempted to argue that every change in this series of tweets is the predictable consequence of
truebeing a file. So far as you think these changes are bad, you should be bothered by the original decision.Not to mention that performance is another reason true and false were moved to a builtin.
Well, you would have to explain that Alien what unix is and how it works anyway, because
truecan only be “true” on unix systems.You could also tell that alien “executing a file on unix will return successfully, unless the program specifies otherwise. An empty file is an empty program and thus does nothing, so it returns successfully” And he don’t even need his weird alien programming logic ;)
I find it a little disappointing that most of the reactions here are wrapped around the axle on the particulars of
true, rather than the (imo) more interesting examination of how complexity creeps into our software design and what that means.Also, I wonder if people think that they might have something to learn or gain from someone who has a lot experience in our field and has created successful projects! (Note I’m not making an argument from authority – I’m not saying he’s correct or not, just that we might read it charitably and reflect that he might have something valuable to say before reacting strongly!)
If he had something valuable to say, one would hope that his experience would have led him to offer an example of it. Instead we got this, which is essentially “old man yells at cloud.”
This is full documentation of the GNU version of
truecommand:Seems that the binary version of
truewas probably created so that the system won’t need to load big/bin/sh.You’re all missing the chance to debate a real philosophical question: is a zero-length /bin/true a binary file or a text file?
There are no binary files in Unix, and there are no text files either. This is one of the main differentiators between Unix and other systems, especially systems that preceded it, like Multics. In those systems the syntax for dealing with text files is different than the syntax for dealing with binary files. In Unix, there are just files, uninterpreted streams of bytes. Of course there are textual and binary interfaces, but that’s a different thing. The syntax is the same, the semantics differ.
Screenshot for those who don’t want to go to twitter: https://u.teknik.io/1tID2.png
https://threadreaderapp.com/thread/966896123548872705.html
Unrolled for people who don’t want to read from screenshots.
Even better.
it’s funny how on this website we routinely see people arguing about the inefficient design of fork(), and people who would rather have an empty /bin/true (which would spawn an entire new shell when executed) than implement it in c
This rant makes more sense in context: Rob Pike helped develop and believed in the ‘everything is a file’ paradigm with Plan9 (as borrowed and expanded from UNIX philosophy).
I like being aware of this. Sometimes people try to improve things that are perfectly ok already and when you examine why, it’s usually because they feel that they need to do something. It happens outside of software and engineering too,
Isn’t it better though to have an ELF binary which just returns 0, rather than having to start a shell interpreter every time you want to invoke /bin/true? Also, when every other part of a collection of tools (in this case GNU coreutils) follows a convention (i.e that
--versionprints version information and--helpprints usage information), is it really better that /bin/true is the one binary which doesn’t follow that convention?This seems like a classic case of making the world a little bit better.
I can see an alternate viewpoint where it just seems like bloat. It’s yet another chunk of source to carry around to a distribution, yet another binary to build when bootstrapping a system, and other holdovers.
Also the GNU coreutils implementation of
trueis embarrassing. https://github.com/coreutils/coreutils/blob/master/src/true.c is 65 lines of C code and accepts 2 command-line arguments, which means that the binary has to be locale aware in those situations.Yep, I’d call depending on sh to deal with /bin/true bloat, if only due to the overall extra time to exec() sh in general. Times with a warm cache, not cold. Yes this is golfing to a degree, but this kind of stuff adds up. A minimal binary in my opinion is not worse, its seeing the forest over a single sh tree.
Even that though is gnu true, compared to a return 0 binary its also slow due to the locale etc… stuff you mention:
I don’t understand how the C implementation of
trueis non-portable? I mean, every C compiler and environment I have seen can setmain()as an integer, and return0to the shell. Can some one explain how the following C code is non-portable?His complaint is that the executable is not portable, not the source file. The literal file
/usr/bin/trueon a Linux machine can’t be dropped onto a machine that doesn’t use ELF binaries, for example.That makes way more sense, thank you.
It does make more sense, but what doesn’t (to me) is why Pike of all people complains about binary vs. source compatibility.
Main must have exactly two parameters (int and char **) or void. Anything else is implementation defined territory.
Good call!
Saved you some bytes, leaving out the include works fine, no?
Saved you some bytes, leave that to your compilet, he knows his job!
Fun thread.
I think this is a really interesting case of Worse is Better, drawn out over years. The empty file solution is clearly Best, but it apparently the business determined it should have comments. Processing comments was not the ideal purpose for the shell; it was too slow! So out comes the C compiler, which of course seems Worse. The Best technical solution is great, but it’s often not great business, especially in the short-term.
I think the real business need was arbitrarily-long comment headers, honestly, and the separate compilation step to avoid the ensuing runtime overhead. Why that need wasn’t laughed aside, I don’t know; I guess you had to be there. They should’ve deleted the comment.
Really I’d say the root problem is not being able to have metadata comments about a file that aren’t in the file.
So how long till we see a benchmark between the different implementations of “true”?
Putting on my “very definitely not a scientist” hat:
Beware of constructs like this if you write production code and intend it to be portable.
Also: true is not guaranteed to be a builtin, : is.
The “very definitely not a scientist” hat has a tag on the inside saying in big red letters “very definitely not for production use”. =) You would indeed need to subshell or something using
/bin/timeinstead of bash’s builtintime, and I made no effort whatsoever to be portable (or to ensure caches were filled or the processor unburdened with background tasks).As another possibly amusing case:
Anybody want to try with
zsh? ;)Sure, i’ll do a few more for comparison on a SLOW arm box, that’ll make this all the more pronounced. :)
Lets compare that to a fully static minimal true (no ld.so shenanigans) note, this is a musl libc system not glibc:
Well that was ostensibly worse, go figure. Lets strip it too to be sure.
I would’ve guessed that minimal true would be the fastest of the bunch, but guess not, GO FIGURE. YMMV this is all not scientific at all.
Any slowness in the script version could probably be linked to some other bloat in that program.
Careful, the troll police is hawk-eyed and doesn’t appreciate truths like that.