1. 36

    This is why we can’t have good software. This program could literally have been an empty file, a nothing at all, a name capturing the essence perfectly.

    I’m not sure I could disagree more strongly. An empty file only has the true behavior because of a bunch of incredibly non-obvious specific Unix behaviors. It would be equally reasonable for execution of this file to fail (like false) since there’s no hashbang or distinguishable executable format to decide how to handle it. At a somewhat higher level of non-obviousness, it’s really weird that true need be a command at all (and indeed, in almost all shells, it’s nottrue is a builtin nearly everywhere).

    true being implementable in Unix as an empty file isn’t elegant—it’s coincidental and implicit.

    1. 7

      I mean, it’s POSIX specified behavior that any file that is executed that isn’t a loadable binary is passed to /bin/sh (”#!” as the first two bytes results in “implementation-defined” behavior), and it’s POSIX specified behavior that absent anything else, a shell script exits true.

      It’s no more coincidental and implicit than “read(100)” advances the file pointer 100 bytes, or any other piece of standard behavior. Sure, it’s Unix(-like)-specific, but, well, it’s on a Unix(-like) operating system. :)

      1. 16

        It’s precisely specified, yes, but it’s totally coincidental that the specification says what it does. A perfectly-reasonable and nearly-equivalent specification in an alternate universe where Thomson and Ritchie sneezed five seconds earlier while deciding how executables should be handled would have precisely the opposite behavior.

        On the other hand, if read(100) did anything other than read 100 bytes, that would be extremely surprising and would not have come about from an errant sneeze.

        1. 16

          Black Mirror Episode: The year is 2100 and the world is ravaged by global warming. The extra energy aggregated over decades because non executables went through /bin/sh caused the environment to enter the tipping point where the feedback loops turned on. A time machine is invented, where one brave soul goes back in time with a feather, finds Thomson and makes him sneeze, saving humanity from the brink of extinction. But then finds himself going back to 2100 with the world still ravaged. Learns that it was fruitless because of npm and left-pad.

          1. 6

            it’s totally coincidental that the specification says what it does.

            This is true of literally all software specifications, in my experience.

            1. 6

              Surely we can agree that it is far more coincidental that an empty executable returns success immediately than that e.g. read(100) reads 100 bytes?

              1. 5

                Why isn’t 100 an octal (or a hex or binary) constant? Why is it bytes instead of machine words? Why is read bound to a file descriptor instead of having a record size from an ioctl, and then reading in 100 records?

                Just some examples. :)

                1. 5

                  Obviously, minor variations are possible. However, in no reasonable (or even moderately unreasonable) world, would read(100) write 100 bytes.

                  1.  

                    Pass a mmap’ed pointer to read, and it shall write. :)

            2. 6

              The current (POSIX) specification is the product of historical evolution caused in part by /bin/true itself. You see, in V7 Unix, the kernel did not execute an empty file (or shell scripts); it executed only real binaries. It was up to the shell to run shell scripts, including empty ones. Through a series of generalizations (starting in 4BSD with the introduction of csh), this led to the creation of #! and kernel support for it, and then POSIX requiring that the empty file trick be broadly supported.

              This historical evolution could have gone another way, but the current status is not the way it is because people rolled out of bed one day and made a decision; it is because a series of choices turned out to be useful enough to be widely supported, eventually in POSIX, and some choices to the contrary wound up being discarded.

              (There was a time when kernel support for #! was a dividing line between BSD and System V Unix. The lack of it in the latter meant that, for example, you could not make a shell script be someone’s login shell; it had to be a real executable.)

              1. 5

                The opposite isn’t reasonable though. That would mean every shell script would have to explicitly exit 0 or it will fail.

                Every. Shell. Script.

                And aside from annoying everyone, that wouldn’t even change anything. It would just make the implementation of true be exit 0, instead of the implementation of false be exit 1.

                And read(100) does do something besides read 100 bytes. It reads up to 100 bytes, and isn’t guaranteed to read the full 100 bytes. You must check the return value and use only the amount of bytes read.

                1.  

                  It’s not obvious to me that an empty file should count as a valid shell script. It makes code generation marginally easier, I suppose. But I also find something intuitive to the idea that a program should be one or more statements/expressions (or functions if you need main), not zero or more.

                  1.  

                    That would mean every shell script would have to explicitly exit 0 or it will fail.

                    I don’t see how that follows.

                    Once the file is actually passed to the shell, it is free to interpret it as it wishes. No reasonable shell language would force users to specify successful exit. But what the shell does is not in question here; it’s what the OS does with an empty or unroutable executable, for which I am contending there is not an obvious behavior. (In fact, I think the behavior of running it unconditionally with the shell is counterintuitive.)

                    And read(100) does do something besides read 100 bytes.

                    You’re being pedantic. Obviously, under some circumstances it will set error codes, as well. It very clearly reads some amount of data, subject to the limitations and exceptions of the system; zero knowledge of Unix is required to intuit that behavior.

                    1. 5

                      I don’t see how that follows.

                      You claim the exact opposite behavior would have been equally reasonable. That is, the opposite of an empty shell script exiting true. The precise opposite would be an empty shell script—i.e. a script without an explicit exit—exiting false. This would affect all shell scripts.

                      Unless you meant the opposite of executing a file not loadable as an executable binary by passing it to /bin/sh, in which case I really would like to know what the “precise opposite” of passing a file to /bin/sh would be.

                      You’re being pedantic. Obviously, under some circumstances it will set error codes, as well. It very clearly reads some amount of data, subject to the limitations and exceptions of the system; zero knowledge of Unix is required to intuit that behavior.

                      No. Many people assume read will fill the buffer size they provide unless they are reading the trailing bytes of the file. However, read is allowed to return any number of bytes within the buffer size at any time.

                      It also has multiple result codes that are not errors. Many people assume when read returns -1 that means error. Did you omit that detail for brevity, or was it not obvious to you?

                  2.  

                    Saying the behavior is totally (or even partially) coincidental is a bit strong. You’re ignoring the fundamental design constraints around shell language and giving the original designers more credit than they deserve.

                    Consider this experiment: you pick 100 random people (who have no previous experience to computer languages) and ask them to design a shell language for POSIX. How would all of these languages compare?

                    If the design constraints I’m talking about didn’t exist, then it would indeed be random and one would expect only ~50% of the experimental shell languages to have a zero exit status for an empty program.

                    I strongly doubt that is what you would see. I think you would see the vast majority of those languages specifying that an empty program have zero exit status. In that case, it can’t be random and there must something intentional or fundamental driving that decision.

                    1.  

                      I don’t care about how the shell handles an empty file. (Returning successful in that case is basically reasonable, but not in my opinion altogether obvious.) I’m stating that the operating system handling empty executables by passing them to the shell is essentially arbitrary.

                      1.  

                        The reason for the existence of human intelligence isn’t obvious either but that doesn’t make it random. A hostile environment naturally provides a strong incentive for an organism to evolve intelligence.

                        As far as the operating system executing non-binaries with “/bin/sh” being arbitrary, fair enough. Though I would argue that once the concepts of the shebang line and an interpreter exist, it’s not far off to imagine the concept of a “default interpreter.” Do you think the concept of a default is arbitrary?

                    2.  

                      It’s precisely specified, yes, but it’s totally coincidental that the specification says what it does.

                      laughs That’s really taking an axe to the sum basis of knowledge, isn’t it?

                  3.  

                    yes an empty file signifying true violates the principle of least astonishment.However if there were a way to have metadata comments about the file describing what it does, how it works, and what version it is without having any of that in the file we’d have the best of both worlds.

                    1.  

                      true being implementable in Unix as an empty file isn’t elegant—it’s coincidental and implicit.

                      But isn’t this in some sense exactly living up to the “unix philosophy”?

                      1.  

                        No.

                      2.  

                        To me, the issue is whether it is prone to error. If it is not, it is culture building because it is part of the lore.

                      1. 14

                        I don’t see a lot of value in the changes to to true that Rob complains about.

                        However, I also don’t see how having your shell scripts depend on an empty file with a particular name, so that you can run that command to get a 0 status code, counts as “good software”.

                        I don’t suppose there’s a practical problem to doing it that way[0], but imagine you have to explain true to an alien who knows a great deal about programming, but has no background with unix.

                        [0] I’m tempted to argue that every change in this series of tweets is the predictable consequence of true being a file. So far as you think these changes are bad, you should be bothered by the original decision.

                        1.  

                          Not to mention that performance is another reason true and false were moved to a builtin.

                        1. 28

                          In the Hacker News thread about the new Go package manager people were angry about go, since the npm package manager was obviously superior. I can see the quality of that now.

                          There’s another Lobster thread right now about how distributions like Debian are obsolete. The idea being that people use stuff like npm now, instead of apt, because apt can’t keep up with modern software development.

                          Kubernetes official installer is some curl | sudo bash thing instead of providing any kind of package.

                          In the meantime I will keep using only FreeBSD/OpenBSD/RHEL packages and avoid all these nightmares. Sometimes the old ways are the right ways.

                          1. 7

                            “In the Hacker News thread about the new Go package manager people were angry about go, since the npm package manager was obviously superior. I can see the quality of that now.”

                            I think this misses the point. The relevant claim was that npm has a good general approach to packaging, not that npm is perfectly written. You can be solving the right problem, but writing terribly buggy code, and you can write bulletproof code that solves the wrong problem.

                            1.  

                              npm has a good general approach to packaging

                              The thing is, their general approach isn’t good.

                              They only relatively recently decided locking down versions is the Correct Thing to Do. They then screwed this up more than once.

                              They only relatively recently decided that having a flattened module structure was a good idea (because presumably they never tested in production settings on Windows!).

                              They decided that letting people do weird things with their package registry is the Correct Thing to Do.

                              They took on VC funding without actually having a clear business plan (which is probably going to end in tears later, for the whole node community).

                              On and on and on…

                              1.  

                                Go and the soon-to-be-official dep dependency managment tool manages dependencies just fine.

                                The Go language has several compilers available. Traditional Linux distro packages together with gcc-go is also an acceptable solution.

                                1.  

                                  It seems the soon-to-be-official dep tool is going to be replaced by another approach (currently named vgo).

                                2.  

                                  I believe there’s a high correlation between the quality of the software and the quality of the solution. Others might disagree, but that’s been pretty accurate in my experience. I can’t say why, but I suspect it has to do with the same level of care put into both the implementation and in understanding the problem in the first place. I cannot prove any of this, this is just my heuristic.

                                  1. 8

                                    You’re not even responding to their argument.

                                    1.  

                                      There’s npm registry/ecosystem and then there’s the npm cli tool. The npm registry/ecosystem can be used with other clients than the npm cli client and when discussing npm in general people usually refer to the ecosystem rather than the specific implementation of the npm cli client.

                                      I think npm is good but I’m also skeptical about the npm cli tool. One doesn’t exclude the other. Good thing there’s yarn.

                                      1.  

                                        I think you’re probably right that there is a correlation. But it would have to be an extremely strong correlation to justify what you’re saying.

                                        In addition, NPM isn’t the only package manager built on similar principles. Cargo takes heavy inspiration from NPM, and I haven’t heard about it having a history of show-stopping bugs. Perhaps I’ve missed the news.

                                    2. 8

                                      The thing to keep in mind is that all of these were (hopefully) done with best intentions. Pretty much all of these had a specific use case… there’s outrage, sure… but they all seem to have a reason for their trade offs.

                                      • People are angry about a proposed go package manager because it throws out a ton of the work that’s been done by the community over the past year… even though it’s fairly well thought out and aims to solve a lot of problems. It’s no secret that package management in go is lacking at best.
                                      • Distributions like Debian are outdated, at least for software dev, but their advantage is that they generally provide a rock solid base to build off of. I don’t want to have to use a version of a python library from years ago because it’s the only version provided by the operating system.
                                      • While I don’t trust curl | sh it is convenient… and it’s hard to argue that point. Providing packages should be better, but then you have to deal with bug reports where people didn’t install the package repositories correctly… and differences in builds between distros… and… and…

                                      It’s easy to look at the entire ecosystem and say “everything is terrible” but when you sit back, we’re still at a pretty good place… there are plenty of good, solid options for development and we’re moving (however slowly) towards safer, more efficient build/dev environments.

                                      But maybe I’m just telling myself all this so I don’t go crazy… jury’s still out on that.

                                      1.  

                                        Distributions like Debian are outdated, at least for software dev,

                                        That is the sentiment that seems to drive the programming language specific package managers. I think what is driving this is that software often has way too many unnecessary dependencies causing setup of the environment to build the software being hard or taking lots of time.

                                        I don’t want to have to use a version of a python library from years ago because it’s the only version provided by the operating system.

                                        Often it is possible to install libraries at another location and redirect your software to use that though.

                                        It’s easy to look at the entire ecosystem and say “everything is terrible” but when you sit back, we’re still at a pretty good place…

                                        I’m not so sure. I forsee an environment where actually building software is a lost art. Where people directly edit interpreted files in place inside a virtual machine image/flatpak/whatever because they no longer know how to build the software and setup the environment it needs. And then some language specific package manager for distributing these images.

                                        I’m growing more disillusioned the more I read Hacker News and lobste.rs… Help me be happy. :)

                                        1.  

                                          So like squeak/smalltalk images then? Whats old is new again I suppose.

                                          http://squeak.org

                                          1.  

                                            I’m not so sure. I forsee an environment where actually building software is a lost art. Where people directly edit interpreted files in place inside a virtual machine image/flatpak/whatever because they no longer know how to build the software and setup the environment it needs. And then some language specific package manager for distributing these images.

                                            You could say the same thing about Docker. I think package managers and tools like Docker are a net win for the community. They make it faster for experienced practitioners to setup environments and they make it easier for inexperienced ones as well. Sure, there is a lot you’ve gotta learn to use either responsibly. But I remember having to build redis every time I needed it because it wasn’t in ubuntu’s official package manager when I started using it. And while I certainly appreciate that experience, I love that I can just install it with apt now.

                                          2.  

                                            I don’t want to have to use a version of a python library from years ago because it’s the only version provided by the operating system.

                                            Speaking of Python specifically, it’s not a big problem there because everyone is expected to work within virtual environments and nobody runs pip install with sudo. And when libraries require building something binary, people do rely on system-provided stable toolchains (compilers and -dev packages for C libraries). And it all kinda works :-)

                                            1.  

                                              I think virtual environments are a best practice that unfortunately isn’t followed everywhere. You definitely shoudn’t run pip install with sudo but I know of a number of companies where part of their deployment is to build a VM image and sudo pip install the dependencies. However it’s the same thing with npm. In theory you should just run as a normal user and have everything installed to node_modules but this clearly isn’t the case, as shown by this issue.

                                              1. 5

                                                nobody runs pip install with sudo

                                                I’m pretty sure there are quite a few devs doing just that.

                                                1.  

                                                  Sure, I didn’t count :-) The important point is they have a viable option not to.

                                                2.  

                                                  npm works locally by default, without even doing anything to make a virtual environment. Bundler, Cargo, Stack etc. are similar.

                                                  People just do sudo because Reasons™ :(

                                              2.  

                                                It’s worth noting that many of the “curl | bash” installers actually add a package repository and then install the software package. They contain some glue code like automatic OS/distribution detection.

                                                1.  

                                                  I’d never known true pain in software development until I tried to make my own .debs and .rpms. Consider that some of these newer packaging systems might have been built because Linux packaging is an ongoing tirefire.

                                                  1.  

                                                    with fpm https://github.com/jordansissel/fpm it’s not that hard. But yes, using the Debian or Redhat blessed was to package stuff and getting them into the official repos is def. painful.

                                                    1.  

                                                      I used the gradle plugins with success in the past, but yeah, writing spec files by hand is something else. I am surprised nobody has invented a more user friendly DSL for that yet.

                                                      1.  

                                                        A lot of difficulties when doing Debian packages come from policy. For your own packages (not targeted to be uploaded in Debian), it’s far easier to build packages if you don’t follow the rules. I like to pretend this is as easy as with fpm, but you get some bonus from it (building in a clean chroot, automatic dependencies, service management like the other packages). I describe this in more details here: https://vincent.bernat.im/en/blog/2016-pragmatic-debian-packaging

                                                      2.  

                                                        It sucks that you come away from this thinking that all of these alternatives don’t provide benefits.

                                                        I know there’s a huge part of the community that just wants things to work. You don’t write npm for fun, you end up writing stuff like it because you can’t get current tools to work with your workflow.

                                                        I totally agree that there’s a lot of messiness in this newer stuff that people in older structures handle well. So…. we can knowledge share and actually make tools on both ends of the spectrum better! Nothing about Kubernetes requires a curl’d installer, after all.

                                                      1. 1

                                                        Reposing my comment here for more engaging feedback: “One idea I’ve had for a while has been smart contextual notifications that take into account what you are doing and remind you if/when the work you do is somewhat related or at least when you’re procrastinating or resting. For example, you go to Amazon and a notification says “ah, you wanted to check this thing out a week ago remember?”. Or “oh you’re on Facebook, how about you answer that email…”. It’d need quite a bit of autonomy and AI of course, but that’s the gist of it.”

                                                        1. 1

                                                          That’s a cool idea. Regarding the “oh, you’re on facebook” thing, I use https://gist.github.com/unhammer/01c65597b5e6509b9eea to notify me about when long compiles complete, because I tend to alt-tab and forget about what I was doing when wait times get too long.

                                                          But other than that I just use really dumb notifications that arbitrarily remind me of random TODO’s dredged up from the depths of my old notes.

                                                          OTOH, I do make Emacs turn off very visible notifications (like from IRC etc.) during working hours.

                                                          1. 2

                                                            What do you use for notifications in Emacs? I use org mode, but have a bad habit of forgetting to check my agenda frequently enough.

                                                            1. 3

                                                              For scheduled agenda items, I just use the built-in appt thing

                                                              (use-package appt
                                                                  :config
                                                                  (add-hook 'org-finalize-agenda-hook #'org-agenda-to-appt)
                                                                  (appt-activate 1)
                                                                  ;; Won't work without display-time:
                                                                  (display-time))
                                                              

                                                              and to ensure it’s run at least once a day I have

                                                              (add-hook 'midnight-hook (defun my-org-agenda-on-midnight ()
                                                                                         (org-agenda nil "p")
                                                                                         (delete-other-windows)
                                                                                         (org-agenda-goto-today)
                                                                                         (recenter)))
                                                              

                                                              (which also means my agenda shows up in the morning)

                                                              But for other things I’ve started using https://github.com/jwiegley/alert (I could maybe get appt to use alert too, but haven’t bothered yet).

                                                              https://github.com/akhramov/org-wild-notifier.el might be relevant if you want more control (I haven’t tried it yet)

                                                        1. 24

                                                          “There are a lot of CAs and therefore there is no security in the TLS CA model” is such a worn out trope.

                                                          The Mozilla and Google CA teams work tirelessly to improve standards for CAs and expand technical enforcement. We remove CAs determined to be negligent and raise the bar for the rest. There seems to be an underlying implication that there are trusted CAs who will happily issue you a google.com certificate: NO. Any CA discovered to be doing something like this gets removed with incredible haste.

                                                          If they’re really concerned about the CA ecosystem, requiring Signed Certificate Timestamps (part of the Certificate Transparency ecosystem) for TLS connections provides evidence that the certificate is publicly auditable, making it possible to detect attacks.

                                                          Finally, TLS provides good defense in depth against things like CVE-2016-1252.

                                                          1. 13

                                                            Any CA discovered to be doing something like this gets removed with incredible haste.

                                                            WoSign got dropped by Mozilla and Google last year after it came to light that they were issuing fraudulent certificates, but afaict there was a gap of unknown duration between when they started allowing fraudulent certs to be issued and when it was discovered that they were doing so. And it still took over six months before the certificate was phased out; I wouldn’t call that “incredible haste”.

                                                            1. 2

                                                              I’m not sure where the process is, but if certificate transparency becomes more standard, I think that would help with this problem.

                                                            2. 5

                                                              TLS provides good defense in depth against things like CVE-2016-1252.

                                                              Defense in depth can do more harm than good if it blurs where the actual security boundaries are. It might be better to distribute packages in a way that makes it very clear they’re untrusted than to additionally verify the packages if that additional verification doesn’t actually form a hard security boundary (e.g. rsync mirrors also exist and while rsync hosts might use some kind of certification, it’s unlikely to follow the same standards as HTTPS. So a developer who assumed that packages fed into apt had already been validated by the TLS CA ecosystem would be dangerously mislead)

                                                              1. 5

                                                                This is partly why browsers are trying to move from https being labeled “secure” to http being labeled “insecure” and displaying no specific indicators for https.

                                                                1. 1

                                                                  e.g. rsync mirrors also exist and while rsync hosts might use some kind of certification, it’s unlikely to follow the same standards as HTTPS

                                                                  If you have this additional complexity in the supply chain then you are going to need additional measures. At the same time, does this functionality provide enough value to the whole ecosystem to exist by default?

                                                                  1. 5

                                                                    If you have this additional complexity in the supply chain then you are going to need additional measures.

                                                                    Only if you need the measures at all. Does GPG signing provide an adequate guarantee of package integrity on its own? IMO it does, and our efforts would be better spent on improving the existing security boundary (e.g. by auditing all the apt code that happens before signature verification) than trying to introduce “defence in depth”.

                                                                    At the same time, does this functionality provide enough value to the whole ecosystem to exist by default?

                                                                    Some kind of alternative to HTTPS for obtaining packages is vital, given how easy it is to break your TLS libraries on a linux system through relatively minor sysadmin mistakes.

                                                              1. 2

                                                                In what other languages would it be possible?

                                                                I guess everything with properties (functions disguised as fields) so D, C#, etc.

                                                                Afaik not with C, C++, or Java.

                                                                1. 26
                                                                  #define a (++i)
                                                                  int i = 0;
                                                                  
                                                                  if (a == 1 && a == 2 && a == 3)
                                                                      ....
                                                                  
                                                                  1. 1

                                                                    Isn’t that undefined behavior? Or is && a sequence point?

                                                                    1. 3

                                                                      && and || are sequence points. The right expression may never happen depending on the result of the left, so it would make things interesting if they weren’t.

                                                                  2. 10

                                                                    This is very easy to do in C++.

                                                                    1. 5

                                                                      You can also do it with Haskell.

                                                                      1. 3

                                                                        Doable with Java (override the equals method), and as an extension, with Clojure too:

                                                                        (deftype Anything []
                                                                          Object
                                                                          (equals [a b] true))
                                                                        
                                                                        (let [a (Anything.)]
                                                                          (when (and (= a 1) (= a 2) (= a 3))
                                                                            (println "Hello world!")))
                                                                        

                                                                        Try it!

                                                                        Or, inspired by @zge above:

                                                                        (let [== (fn [& _] true)
                                                                              a 1]
                                                                          (and (== a 1) (== a 2) (== a 3)))
                                                                        
                                                                        1. 3

                                                                          Sort of. In Java, == doesn’t call the equals method, it just does a comparison for identity. So

                                                                           a.equals(1) && a.equals(2) && a.equals(3); 
                                                                          

                                                                          can be true, but never

                                                                           a == 1 && a == 2 && a == 3;
                                                                          
                                                                        2. 3

                                                                          perl can do it very simply

                                                                          my $i = 0;
                                                                          sub a {
                                                                          	return ++$i;
                                                                          }
                                                                          
                                                                          if (a == 1 && a == 2 && a == 3) {
                                                                          	print("true\n");
                                                                          }
                                                                          
                                                                          1. 2

                                                                            Here is a C# version.

                                                                            using System;
                                                                            
                                                                            namespace ContrivedExample
                                                                            {
                                                                                public sealed class Miscreant
                                                                                {
                                                                                    public static implicit operator Miscreant(int i) => new Miscreant();
                                                                            
                                                                                    public static bool operator ==(Miscreant left, Miscreant right) => true;
                                                                            
                                                                                    public static bool operator !=(Miscreant left, Miscreant right) => false;
                                                                                }
                                                                            
                                                                                internal static class Program
                                                                                {
                                                                                    private static void Main(string[] args)
                                                                                    {
                                                                                        var a = new Miscreant();
                                                                                        bool broken = a == 1 && a == 2 && a == 3;
                                                                                        Console.WriteLine(broken);
                                                                                    }
                                                                                }
                                                                            }
                                                                            
                                                                            1. 2

                                                                              One of the ‘tricks’ where all a’s are different Unicode characters is possible with Python and Ruby. Probably in Golang too.

                                                                              1. 7

                                                                                In python, you can simply create class with __eq__ method and do whatever you want.

                                                                                1. 4

                                                                                  Likewise in ruby, trivial to implement

                                                                                  a = Class.new do
                                                                                    def ==(*)
                                                                                      true
                                                                                    end
                                                                                  end.new
                                                                                  
                                                                                  a == 1 # => true
                                                                                  a == 2 # => true
                                                                                  a == 3 # => true
                                                                                  
                                                                              2. 2

                                                                                In Scheme you could either take the lazy route and do (note the invariance of the order or ammount of the operations):

                                                                                (let ((= (lambda (a b) #t))
                                                                                       (a 1))
                                                                                  (if (or (= 1 a) (= 2 a) (= 3 a))
                                                                                      "take that Aristotle!"))
                                                                                

                                                                                Or be more creative, and say

                                                                                (let ((= (lambda (x _) (or (map (lambda (n) (= x n)) '(1 2 3)))))
                                                                                        (a 1))
                                                                                    (if (or (= 1 a) (= 2 a) (= 3 a))
                                                                                        "take that Aristotle!"))
                                                                                

                                                                                if you would want = to only mean “is equal to one, two or three”, instead of everything is “everything is equal”, of course only within this let block. The same could also be done with eq?, obviously.

                                                                                1. 1

                                                                                  Here is a Swift version that uses side effects in the definition of the == operator.

                                                                                  import Foundation
                                                                                  
                                                                                  internal final class Miscreant {
                                                                                      private var value = 0
                                                                                      public static func ==(lhs: Miscreant, rhs: Int) -> Bool {
                                                                                          lhs.value += 1
                                                                                          return lhs.value == rhs
                                                                                      }
                                                                                  }
                                                                                  
                                                                                  let a = Miscreant()
                                                                                  print(a == 1 && a == 2 && a == 3)
                                                                                  
                                                                                1. 1

                                                                                  I tried to reproduce his observation and noticed something strange.

                                                                                  When I use !google on DuckDuckGo then his music article is the first hit. When I search using google.com his article doesn’t show up at all.

                                                                                  1. 1

                                                                                    The article is a few days old, and was highly rated on HN for awhile. So Google probably indexed it, followed his link to the music article, and thereby “fixed” the problem for now.

                                                                                  1. 30

                                                                                    All of them:

                                                                                    The fact that they exist at all. The build spec should be part of the language, so you get a real programming language and anyone with a compiler can build any library.

                                                                                    All of them:

                                                                                    The fact that they waste so much effort on incremental builds when the compilers should really be so fast that you don’t need them. You should never have to make clean because it miscompiled, and the easiest way to achieve that is to build everything every time. But our compilers are way too slow for that.

                                                                                    Virtually all of them:

                                                                                    The build systems that do incremental builds almost universally get them wrong.

                                                                                    If I start on branch A, check out branch B, then switch back to branch A, none of my files have changed, so none of them should be rebuilt. Most build systems look at file modified times and rebuild half the codebase at this point.

                                                                                    Codebases easily fit in RAM and we have hash functions that can saturate memory bandwidth, just hash everything and use that figure out what needs rebuilding. Hash all the headers and source files, all the command line arguments, compiler binaries, everything. It takes less than 1 second.

                                                                                    Virtually all of them:

                                                                                    Making me write a build spec in something that isn’t a normal good programming language. The build logic for my game looks like this:

                                                                                    if we're on Windows, build the server and all the libraries it needs
                                                                                    if we're on OpenBSD, don't build anything else
                                                                                    build the game and all the libraries it needs
                                                                                    if this is a release build, exit
                                                                                    build experimental binaries and the asset compiler
                                                                                    if this PC has the release signing key, build the sign tool
                                                                                    

                                                                                    with debug/asan/optdebug/release builds all going in separate folders. Most build systems need insane contortions to express something like that, if they can do it at all,

                                                                                    My build system is a Lua script that outputs a Makefile (and could easily output a ninja/vcxproj/etc). The control flow looks exactly like what I just described.

                                                                                    1. 15

                                                                                      The fact that they exist at all. The build spec should be part of the language, so you get a real programming language and anyone with a compiler can build any library.

                                                                                      I disagree. Making the build system part of the language takes away too much flexibility. Consider the build systems in XCode, plain Makefiles, CMake, MSVC++, etc. Which one is the correct one to standardize on? None of them because they’re all targeting different use cases.

                                                                                      Keeping the build system separate also decouples it from the language, and allows projects using multiple languages to be built with a single build system. It also allows the build system to be swapped out for a better one.

                                                                                      Codebases easily fit in RAM …

                                                                                      Yours might, but many don’t and even if most do now, there’s a very good chance they didn’t when the projects started years and years ago.

                                                                                      Making me write a build spec in something that isn’t a normal good programming language.

                                                                                      It depends on what you mean by “normal good programming language”. Scons uses Python, and there’s nothing stopping you from using it. I personally don’t mind the syntax of Makefiles, but it really boils down to personal preference.

                                                                                      1. 2

                                                                                        Minor comment is that the codebase doesn’t need to fit into ram for you to hash it. You only need to store the current state of the hash function and can handle files X bytes at a time.

                                                                                      2. 14

                                                                                        When I looked at this thread, I promised myself “don’t talk about Nix” but here I am, talking about Nix.

                                                                                        Nix puts no effort in to incremental builds. In fact, it doesn’t support them at all! Nix uses the hashing mechanism you described and a not terrible language to describe build steps.

                                                                                        1. 11

                                                                                          The build spec should be part of the language, so you get a real programming language and anyone with a compiler can build any library.

                                                                                          I’m not sure if I would agree with this. Wouldn’t it just make compilers more complex, bigger and error prone (“anti-unix”, if one may)? I mean, in some cases I do appriciate it, like with go’s model of go build, go get, go fmt, … but I wouldn’t mind if I had to use a build system either. My main issue is the apparent nonstandard-ness between for example go’s build system and rust’s via cargo (it might be similar, I haven’t really ever used rust). I would want to be able to expect similar, if not the same structure, for the same commands, but this isn’t necessarily given if every compiler reimplements the same stuff all over again.

                                                                                          Who knows, maybe you’re right and the actual goal should be create a common compiler system, that interfaces to particular language definitions (isn’t LLVM something like this?), so that one can type compile prog.go, compile prog.c and compile prog.rs and know to expect the same structure. Would certainly make it easier to create new languages…

                                                                                          1. 2

                                                                                            I can’t say what the parent meant, but my thought is that a blessed way to lay things out and build should ship with the primary tooling for the language, but should be implemented and designed with extensibility/reusability in mind, so that you can build new tools on top of it.

                                                                                            The idea that compilation shouldn’t be a special snowflake process for each language is also good. It’s a big problem space, and there may well not be one solution that works for every language (compare javascript to just about anything else out there), but the amount of duplication is staggering.

                                                                                            1. 1

                                                                                              Considering how big compilers/stdlibs are already, adding a build system on top would not make that much of a difference.

                                                                                              The big win is that you can download any piece of software and build it, or download a library and just add it to your codebase. Compare with C/C++ where adding a library is often more difficult than writing the code yourself, because you have to figure out their (often insane) build system and integrate it with your own, or figure it out then ditch it and replace it with yours

                                                                                            2. 8

                                                                                              +1 to all of these, but especially the point about the annoyance of having to learn and use another, usually ad-hoc programming language, to define the build system. That’s the thing I dislike the most about things like CMake: anything even mildly complex ends up becoming a disaster of having to deal with the messy, poorly-documented CMake language.

                                                                                              1. 3

                                                                                                Incremental build support goes hand in hand with things like caching type information, extremely useful for IDE support.

                                                                                                I still think we can get way better at speeding up compilation times (even if there’s always the edge cases), but incremental builds are a decent target to making compilation a bit more durable in my opinion.

                                                                                                Function hashing is also just part of the story, since you have things like inlining in C and languages like Python allow for order-dependent behavior that goes beyond code equality. Though I really think we can do way better on this point.

                                                                                                A bit ironically, a sort of unified incremental build protocol would let compilers avoid incremental builds and allow for build systems to handle it instead.

                                                                                                1. 2

                                                                                                  I have been compiling Chromium a lot lately. That’s 77000 mostly C++ (and a few C) files. I can’t imagine going through all those files and hashing them would be fast. Recompiling everything any time anything changes would probably also be way too slow, even if Clang was fast and didn’t compile three files per second average.

                                                                                                  1. 4

                                                                                                    Hashing file contents should be disk-io-bound; a couple of seconds, at most.

                                                                                                    1. 3

                                                                                                      You could always do a hybrid approach: do the hash check only for files that have a more-recent modified timestamp.

                                                                                                    2. 1

                                                                                                      Do you use xmake or something else? It definitely has a lot of these if cascades.

                                                                                                      1. 1

                                                                                                        It’s a plain Lua script that does host detection and converts lines like bin( "asdf", { "obj1", "obj2", ... }, { "lib1", "lib2", ... } ) into make rules.

                                                                                                      2. 1

                                                                                                        Codebases easily fit in RAM and we have hash functions that can saturate memory bandwidth, just hash everything and use that figure out what needs rebuilding. Hash all the headers and source files, all the command line arguments, compiler binaries, everything. It takes less than 1 second.

                                                                                                        Unless your build system is a daemon, it’d have to traverse the entire tree and hash every relevant file on every build. Coming back to a non-trivial codebase after the kernel stopped caching files in your codebase will waste a lot of file reads, which are typically slow on an HDD. Assuming everything is on an SSD is questionable.

                                                                                                      1. 33

                                                                                                        Side topic, this story may explain an odd story from two weeks ago about how the Intel CEO sold all the shares he could. If he doesn’t have rock-solid documentation that the trade was planned before he learned this, that’s probably insider trading. (Hat tip to @goodger prompting me to look up the SEC rule in the chat.)

                                                                                                        ETA: here’s the form 4 he filed. I’ve got to step out the door, but if anyone can figure out if this was reported to Intel before Nov 29 that would be interesting.

                                                                                                        1. 18

                                                                                                          From the project zero blog post:

                                                                                                          We reported this issue to Intel, AMD and ARM on 2017-06-01

                                                                                                          1. 6

                                                                                                            Good find. Looks like some press has it now, too. And a yc news commenter notes it’s not in their 10-Q, so that’s probably a couple counts in an indictment and a shareholder lawsuit.

                                                                                                          2. 1

                                                                                                            Even if he knew, I think it matters whether this is a recurring event. If he always sells his shares at the end of the year, it would be insane to demand that he doesn’t do it.

                                                                                                            Otherwise people could just start shorting as soon as they see an executive not selling stock, because they can infer now that there is some bad news incoming.

                                                                                                            1. 9

                                                                                                              It’s public information, there’s no need to speculate. He doesn’t.

                                                                                                              1. 1

                                                                                                                Matt Levine is relaying Intel comments that are the opposite of what you’re saying.

                                                                                                                1. 3

                                                                                                                  That article is pretty misleading. It’s true that the November sale was “pursuant to a pre-arranged stock sale plan with an automated sale schedule,” but that stock sale plan was pre-arranged only in October, months after Google had notified Intel of these vulnerabilities.

                                                                                                                  1. 3

                                                                                                                    I thought these all had to be disclosed on Form 4s. Maybe there’s another reporting vehicle I’m unaware of, but “Krzanich’s plan seems to involve getting stock grants at the beginning of each year and then selling as much as he can in the fourth quarter, which he has done consistently for a few years.” is not an accurate description of the record in the linked form 4s. His sales happen in every quarter and this is the only time he’s sold down to Intel’s minimum (eyeballing rather than making a running total, but it seems clear).

                                                                                                            1. 2

                                                                                                              Someone needs to trash cargo, because it seems like a lot of people really like it. However, it’s a build system, so it must be bad.

                                                                                                              1. 1

                                                                                                                I don’t think they solved the Debian integration yet? Building without network access, using shared libraries, etc.

                                                                                                              1. 2

                                                                                                                On a modern computer, RAM does not go unused.

                                                                                                                I have a computer with 32 GB of RAM. (It was on sale.) Most of it goes unused. It turns out (shocker!) that 32 GB is actually a shitton of data, and unless you’re allocating extreme amounts to applications (in which case swap isn’t useful), chugging through huge amounts of on-disk data (in which case cache mostly isn’t useful) or going for an uptime award, you’re very unlikely to pagecache anything approaching that much.

                                                                                                                1. 6

                                                                                                                  …or running a web browser for any length of time.

                                                                                                                  I also have 32GB in my desktop, which I don’t reboot often. I wish I could find out how large the page cache would grow over time, but instead I routinely end up with Chromium & Firefox devouring dozens of GBs.

                                                                                                                  I hate web browsers.

                                                                                                                  1. 2

                                                                                                                    I’ve never seen Firefox grow beyond a few gigabytes. (Granted, this is still a ton, but not compared to the total amount of RAM in that system.) My habits may contribute; I rarely have more than a dozen tabs open, and I rarely have a browser process going for more than a week or two.

                                                                                                                    Chromium I understand to be worse. Also, perhaps ironically, most of that memory usage is cache; one of the arguments for swap is, essentially, that much of that cache will be weeks-old pages you’ll never look at again, and swap lets the kernel get that junk out of memory even though it can’t actually free it. (That suggests to me that better memory-management APIs may be in order, which allow applications to indicate to the OS that certain areas of memory are transient and may be reclaimed if needed.)

                                                                                                                    1. 3

                                                                                                                      I recently released 11 gigs of memory by closing Chrome. I typically have >= 50 tabs, and often over 100 open across 2,3,4 browser windows.

                                                                                                                    2. 1

                                                                                                                      Depending on OS, you might be able to limit the amount it can take up so you can kill it early if it’s doing it. You’ll at least consistently know you have all that extra space available.

                                                                                                                    3. 1

                                                                                                                      With that much, you should consider exploring RAM disks for any apps or projects that you want speed on with a utility syncing them to real disk every so often. There used to be hard disks you could buy that would internally use RAM with flash as backup. USPS actually used them on desktops or something to make apps go crazy fast. I used to use them for both speed boosts and ephemeral data

                                                                                                                      I haven’t had one in a while so no links to current options or strategies.

                                                                                                                      1. 3

                                                                                                                        I do this. Syching 4GB on every boot is the downside due to waiting times. But it works good and fast: browser cache and so on.

                                                                                                                        First I used a tmpfs, but found that it didn’t support extended file attributes (xattr; it is supported now). Played a bit around, and found out it’s pretty easy to mount a tmpfs, create (truncate) an empty file in it of a certain size (4GiB), write ext4 to that file and mount it.

                                                                                                                        Then I discovered zram, which allocates a certain amount of ram to a block device and compresses the data (algo lzo and lz4). There’s a little bit of a trade-off in terms of speed, but fun to play with on a dull hour, and useful in multiple occasions.

                                                                                                                        I agree - 32G is a big shoe to fill.

                                                                                                                        1. 2

                                                                                                                          Even with zfs, to add to that. Which has an all-you-can eat ram coupon pretty much. Although that can be tweaked, to be fair. And inspite of all its ram use there’s surpisingly little benefit in terms of transfer speed gains. Not that zfs is slow, it’s just a bit sluggish perhaps.

                                                                                                                          1. 2

                                                                                                                            “Syching 4GB on every boot is the downside due to waiting times.”

                                                                                                                            My strategy for that was causing the boot or sync to happen before I get up. It will just have been running a while. One at night. Basically, try to eliminate the whole waiting phase of permanent to RAM store. Incremental ones during the day as much as you’re willing to avoid losing work. ECC memory a must.

                                                                                                                          2. 2

                                                                                                                            The Sun prestoserve!

                                                                                                                        1. 2

                                                                                                                          Only worked at one company, primarily on one app.

                                                                                                                          There are 156 tables in that app.

                                                                                                                          Edit: Sorry, I’m new here, didn’t know the formatting would change “156.” to “1.”