1. 17

    And the corollary: The first line of your shell script should be

    #!/bin/sh
    

    If you need bashism for something complex that needs to be maintainable, use #!/usr/bin/env bash as the article suggests, but 99% of ‘bash’ scripts that I’ve seen are POSIX shell scripts and work just fine with any POSIX shell.

    1. 16

      And the corollary: The first line of your shell script should be

      I disagree, enormously. There are some specific, isolated, and extremely uncommon situations where it’s important you target sh; the rest of the time, you are constraining yourself for no appreciable benefit.

      In particular, if you did go and change the interpret to /bin/sh, you also need to change the second line, since (at least) pipefail is a bashism.

      1. 3

        I think being constrained to a smaller and simpler set of features is the appreciable benefit.

        This applies doubly if somebody is going to have to maintain this code.

        1. 2

          One does not necessarily follow the other. I’ve seen horrific apparently-posix-sh scripts with the most unmaintainable contortions to avoid using a widely known bash feature that would make the code far more legible. Eg bash arrays.

          1. 1

            I don’t disagree; often there is no nice way in posix sh to do something that a bash array could. But I’ve found that once you begin to feel a desire for any kind of data structure in your program, it’s probably time to switch to a more robust programming language than (ba)sh.

            1. 2

              I generally agree. When I look back at my professional career (16 years and counting), I’ve spent a remarkable amount of it essentially writing bash scripts. Which is surprise.

              Most recently, for containers which can’t have a larger interpreter inside them for space reasons, ruling out Python etc

        2. 1

          In particular, if you did go and change the interpret to /bin/sh, you also need to change the second line, since (at least) pipefail is a bashism.

          Thanks. I have little experience with non-BASH shells and I didn’t want to assume that this would work in on a non-BASH script. Good to learn that it doesn’t.

        3. 14

          Good POSIX shell reference here:

          http://shellhaters.org/

          1. 5

            Fair point. I am trying to target the 100%. It feels to me that’s it’s best to program against bash than risk in failing in a corner case on a shell which I never tested my script against.

            1. 2

              For 99% of shell scripts its a non issue. A lot of people use things like arrays in bash when simple quoted strings work on every shell. Writing portable across multiple bourne shell interpreters is easier than you think it is. Most of my scripts run under any of ksh/zsh/bash/sh with zero changes. And testing wise… there isn’t much to test tbh, outside of -o pipefail only works for zsh and bash. Generally the -u catches those cases anyway.

              What kind of corner case are you thinking of? Also as yule notes, run shellcheck on your scripts when writing them. That will reduce your edge cases significantly.

              1. 4

                The annoying thing is that bash doesn’t disable bash extensions when invoked as sh. A couple weeks ago, I had to debug why a script written by a colleague didn’t work. It turned out that he used &> to redirect both stderr and stdout to a file. I didn’t even know this existed, and he didn’t even know it wasn’t portable. The nasty part is that this doesn’t cause an error. Instead, foo &> bar is parsed as foo & (not sure what happens to the > bar bit; file is created but not hooked up to stdout of foo), which causes race conditions with whatever comes next relying on foo having already completed.

                Of course, with #! /usr/bin/env bash in the script, this wouldn’t have been a problem on my machine, but I maintain that it would be more productive if bash disabled all of its non-POSIX extensions when invoked as sh. More commonly, you see people use [[ ... ]] instead of [ ... ]. I’m still not even sure how [[ is different from [.

                1. 1

                  I’m still not even sure how [[ is different from [.

                  I vaguely know that and I switch out Bash for Python as soon as even simple string or array manipulation kicks in. My reasoning is that programs always become more complex over time. You start with simple string manipulation and soon end up with edge case handling and special cases. Best to switch to a language like Python which supports that.

                  1. 1

                    I think (among other things) [[ disables word splitting, i.e. you can use variables without needing to quote them. At least that’s how it works in ksh.

              2. 4

                Are they though? POSIX shell is so extremely limited that most scripts end up with some bashisms in them, if only to send a string to a program’s stdin. foo <<< "$var" is so much nicer than printf "%s" "$var" | foo (and no, you can’t use echo because POSIX is so vague that you can’t correctly tell echo to treat an argument as a string and not an option in a portable way)

                I’ve made the mistake of thinking my script is POSIX, using #!/bin/sh, and then hearing from Ubuntu users that it doesn’t work in Dash. (I’ve also had problems where my script actually is POSX, but dash is so riddled with bugs that it still didn’t work on Debian-based systems.)

                1. 6

                  That’s why it’s a good idea to always lint your scripts with shellcheck. It will catch all bashisms when you set shellbang to /bin/sh

                2. 3

                  In which case -o pipefail is undefined, at least according to shellcheck.

                  In any case I find running shellcheck on shell scripts to be really valuable. You don’t have to follow it blindly, but it has some very good hints.

                  1. 1

                    shellcheck is my favorite linter for bash scripts as well. It sometimes feels a bit aggressive but I prefer that as opposed to making a mistake.

                  2. 2

                    I respectfully disagree, using the /usr/bin/env method requires bash, so you can use all the bashisms you want, and require users of your scripts to have bash installed.

                    For 95% (wild-ass guess) of Unix users, this is the case. For the rest, if you’ve stated the requirements clearly, they’ll either accept the requirement or form a large enough crowd with pitchforks to force you to change.

                    1. 2

                      *BSD, Solaris, and Android don’t have bash installed by default and it isn’t part of their base systems so it isn’t available until /usr/local is mounted and can’t be used anywhere in early boot. Ubuntu uses dash as /bin/sh, but does install bash by default. Embedded Linux systems typically use busybox or similar and so don’t have bash.

                      95% of Unix users probably covers all Android users, so that’s not a particularly useful metric. The interesting metric is the set of people you expect to use your script. If it’s GNU/Linux users, that’s fine. If it’s people running large desktop / server *NIX systems (macOS, *BSD, Linux), bash is probably available easily but it may not be installed on any given system so you’ve added a dependency.

                      By all means, if you depend on bashism then use bash. In general, I’ve found that my shell scripts fall into two categories:

                      • Simple things, where the POSIX shell is completely adequate and so there’s no reason to use anything other than /bin/sh.
                      • Complex things where I actually want a rich scripting language. Bash is a pretty terrible scripting language, but it’s a much lighter weight dependency than something like Python so I’ll often use bash for these things.

                      I use bash as my interactive shell on FreeBSD, so I don’t really mind having it as a dependency for things, but I consider it somewhat impolite to add dependencies in things I distribute when I don’t really need them.

                    2. 2

                      Why #!/usr/bin/env bash as opposed to #!/bin/bash tho? I’ve seen that done for ruby scripts, and I figured it’s because maybe ruby is in /usr/local/bin, but isn’t bash always in /bin/bash?

                      1. 10

                        Not on all systems. e.g. on FreeBSD and OpenBSD it’s /usr/local/bin/bash.

                        1. 7

                          Or Nix and Guix, which have them in some entirely different place (e.g. on Guix is something like /gnu/store/.../).

                        2. 4

                          in the BSDs it should be under /usr/local/bin or /usr/pkgs/bin. MacOS ships an old version of bash and one might want to install a newer version from homebrew, which goes under /usr/local/bin

                          1. 1

                            Using env bash let me both use a newer homebrew bash on Macs, and keep the script working on Linux machines, and possibly even Windows via WSL. On Macs the newer homebrew bash was needed for a switch statement, if memory serves.

                            For ruby, chruby lets one install multiple versions on one system, and flip between them.

                        1. 7

                          Definitely a recommended read!

                          So you have to take a different approach. If you’re using Markdown, you can either:

                          1. Allow them to only enter pure Markdown, and convert that to HTML on render (many Markdown libraries allow raw HTML by default; be sure to disable that). This is the most secure option, but also more restrictive.
                          2. Allow them to use HTML in the Markdown, but only a whitelist of allowed tags and attributes, such as <a href=”…”> and <img src=”…”>. Both Stack Exchange and GitHub take this second approach.

                          I would like to add a word of caution here for thinking twice before attempting the whitelisting approach. For HTML there might be good tools around already, but in other situations it might be even more risky.

                          Even if you are careful, things can go horribly wrong. Doing things differently is often more work, but the outcome is more predictable.

                          1. 1

                            I have a couple issues with this article. Most notably, the way the author portrayed the way time.Parse works.

                            In real life, this would actually look something more like:

                            currentTime, err := time.Parse("2006-01-02 15:04:05", timestamp)
                            
                            // or, if dealing with some form of standardized timestamp
                            
                            currentTime, err := time.Parse(time.RFC3339, timestamp)
                            

                            I can understand the weird order of arguments, but it’s not quite as confusing if your vars are named correctly. In my opinion, this makes dates and times much easier to parse and deal with rather than looking up whether I should be using %H or %h in man strptime

                            The compiler is another story. While I do agree that whining about what seem to be just “tidiness” issues in the code is annoying in the development process, I think the reasoning behind throwing an error when you have a dangling import or unused variable boils down to compile-time optimization. If the compiler doesn’t need to import something that you’re not using, you can save time and speed up the compilation process. I’m sure it’s not a big change with one import, but what about 50? That could seriously change things. All that being said, I still think this is an improvement that can be made within the toolchain of the language, but I believe it’s the job of gofmt or gopls to handle manipulation of your code before it gets to the compiler to avoid these errors. Something where it would automatically comment out an import when you comment out code that uses it, and removing the import if you delete the code that uses it. I would love to see something like this in Go, it would definitely improve the development experience.

                            The rest of the article is opinions, and the author is certainly entitled to them, so those aren’t what I have an issue with.

                            1. 2

                              Of course you wouldn’t parse a string constant in real code, except maybe in a unit test for the time.Parse() function itself. ;) All I did is choose a humorous way of showing that the format string looks similar to the timestamps it is supposed to parse. I don’t think the order of arguments is a concern.

                              Ad compiler: I believe for development a quick debug cycle between writing code and running the binary to try it is quite important, that’s why I find those errors annoying. I still think being notified about unused code in an automated way is great by itself. When writing Python code I regularly run pyflakes to find issues like that, but on my schedule, when I deem it useful (usually before committing a diff). That’s what I meant with the tool working for the programmer, not the other way around.

                              Hope this clears things up.

                            1. 7

                              Go hit 1.0 eight years ago. Can we please stop circulating these thinkpieces written by people who have spent all of one weekend with the language that just whinge about how it’s not a clone of something else?

                              the timestamp example is exceptionally contrived because in reality you would never call time.Parse with two string literals. If you are writing the string literal into the source, you already know the value of the timestamp; you’d never start with a string literal and then parse it with another string literal because you’re turning what can be a compile-time error into a runtime error; you would just construct the value directly with time.Date, which cannot give you an error. I have been programming Go professionally since 2012 and this has never once been a problem for me.

                              the XML thing is a quagmire because XML itself is rife with redundancies and ambiguities. Try using an XML parser in any other statically typed language and compare them. They’re all a nightmare. Go sucks here but the alternatives suck more so I don’t understand why anyone would feel the need to belabor this point beyond using it to bring traffic to their blog.

                              The Go grammar definition contains semicolons, but you don’t see them much in idiomatic Go code.

                              The semicolon is a statement separator, every for loop delimits its statements with the semicolon for the same reason. Every if err := fn(); err != nil { invocation is a standard usage of the semicolon. So no, not really.

                              Taking shots at javascript for semicolon insertion in (checks watch) 2020? Yawn.

                              To me Go doesn’t really seem to be designed as a general purpose programming language

                              how could you possibly conclude such a thing from this tiny example? did you try to use it for any other domain? Talk about that. What’s established does not come anywhere close to being sufficient evidence to support such a conclusion.

                              Go also seems to suffer from ignorance towards well established patterns that are proven to work

                              this is where I just, quite frankly, lose it. Without this comment I would not have bothered to respond to this piece.

                              You can’t just hand waive and say they’re ignorant of “well established patterns” without naming which patterns specifically. I’m willing to bet for the majority of patterns you can name, somebody on the Go team has written publicly about how the Go team has thought about that problem. It will never cease to amaze me that the Go team can, for example, write what now must be hundreds of pages of explanations as to why generics aren’t there and people will still stand back and say “the Go team hasn’t thought about generics”. There are a lot of things that the Go team has thought about and intentionally decided to omit. They didn’t decide to do what you wanted them to do. You can talk about why you disagree with their conclusions but if you’re going to just make a blanket statement that they haven’t thought about it, you’re just making unsubstantiated accusations of incompetence.

                              The Go team has made some very opinionated choices and some of those choices I don’t agree with. There are aspects of the language that I find absolutely irritating (the lack of sum types is particularly aggravating to me). Go is not a perfect tool. It has its problems. But to say that the Go team is ignorant is something that I will always stand up to oppose. The Go team is absolutely exemplary when it comes to explaining their design philosophy, their strategy, and design problems that are works in progress.

                              the getopt thing does suck though lol

                              1. 2

                                I don’t understand where the hate is coming from. I used Go to write software and made some observations while doing so. Then I shared them with other people on a platform whose purpose is to share links. What’s wrong about that?

                                Yes, the timestamp example is contrived. I did that to make a point that the format and the value to be parsed look the same.

                                the XML thing is a quagmire because XML itself is rife with redundancies and ambiguities. Try using an XML parser in any other statically typed language and compare them. They’re all a nightmare. Go sucks here but the alternatives suck more so I don’t understand why anyone would feel the need to belabor this point beyond using it to bring traffic to their blog.

                                I praised the XML parser in Go: “Parsing XML in Go is relatively easy, and I have to admit rather neat. Definitely more comfortable to use than Python’s ElementTree.” The personal attack was completely uncalled for.

                                Go also seems to suffer from ignorance towards well established patterns that are proven to work

                                this is where I just, quite frankly, lose it. Without this comment I would not have bothered to respond to this piece.

                                Damn, I chose some unfortunate wording there… Let me repeat a comment here that I wrote in response to someone else complaining about the same issue:

                                Yes, datetime parsing and flag handling is what I was refering to.

                                Go is developed by smart people, I never assumed they are not aware of how timestamp formats look like in other languages. I assumed they did it differently anyway ignoring the established practices, that’s why i wrote ‘ignorance’. Maybe not the most fortunate wording, didn’t mean to offend.

                                1. 4

                                  I had the same reaction to your blog post as scraps. On seeing your response and reflecting, I think it’s just a matter of being so tired of surface-level criticisms of Go that seem to sweep up the rankings on hacker news etc. Which is 100% not your fault; of course you’re entitled to write about your experiences.

                                  While your article was pretty balanced in pro/anti sentiments, the fact that it was sandwiched between a contrived and sarcastic example at the start, and closing remarks that used “ignorance” to mean “ignoring”, “disregarding”, or “choosing differently”, meant that my brain put your article into the “sigh, another one of these; why are they always so popular?” category :-)

                                  (fwiw, I agree with scraps that sum types are the biggest obvious hole in Go)

                                  1. 3

                                    this is exactly right.

                              1. 6

                                I thought this was a really good, terse article. I apologize in advance that this comment only focuses on disagreements (I find disagreements more interesting, but people often misinterpret them as defensiveness or something; I need to find a better way to communicate).

                                To me Go doesn’t really seem to be designed as a general purpose programming language, but rather targeted towards a specific niche (although a big one at that) — web services. All of the necessary tools are there: Template rendering, parsing common data formats like XML and JSON, HTTP client and server implementations, hash algorithms, encryption, rudimentary SQL support, and all text is unicode.

                                What’s your standard for a general purpose programming language? Does it have to run on bare metal? Or have a GUI toolkit and/or scientific computing modules in the standard library?

                                Go seems pretty vanilla to me. It’s used for a lot of web service things, but also for a lot of cloud/systems-y things like Docker, Kubernetes, and just about everything Hashicorp has touched since Vagrant. Further, there’s no GUI toolkit, but that’s probably because building a cross-platform (desktop) GUI toolkit worth using is a monumental task, there still isn’t a clear “right way” to do this (a lot of movement in this space), and it’s not clear whether native desktop apps are even the future. Last I checked, Go’s scientific computing story was actually moving right along, but I also haven’t checked in in a few years.

                                To pick a couple more nits, “text” doesn’t have to be unicode (you can put anything in a string), but string literals are always encoded as utf-8. Also, Go doesn’t have anything in the std lib for authentication, which is more important for modern web services than things like templating or XML.

                                1. 2

                                  Glad you liked the article, and thank you for the insights!

                                1. 5

                                  the compiler presents those warnings as errors, meaning the build stops and you don’t get a runnable binary. That’s quite annoying when developing or debugging a program: commenting out a block of code, and now suddenly you have to modify much more code because an import is not used anymore

                                  Hook your editor up to goimports or gopls.

                                  1. 1

                                    Yes, that would alleviate the problem. I just hoped it wouldn’t be necessary, because I like to avoid modifying my editor for a specific programming language. (I run vim with ctrlp and vimwiki being the only plugins.)

                                    1. 1

                                      Language-server protocols are a huge boost to code comprehension, IMO. Well worth the extra configuration effort.

                                  1. 7

                                    Pretty poorly informed piece. Hilarious that it ends with “Go also seems to suffer from ignorance towards well established patterns that are proven to work” and then fails to provide a single example.

                                    1. 4

                                      The datetime parsing and command line flag handling mentioned in the articles are the examples.

                                      1. 5

                                        They’re not examples of ignorance. The Go team was well aware of how date parsing has been done historically but decided to try something more intuitive. The way they wrote the example was carefully chosen to make it look confusing. See https://golangcode.com/parsing-dates/ for how this is usually used.

                                        1. 4

                                          I don’t have enough experience with Go to have an opinion about the specifics I just wanted to clarify what seemed like the intentions of the author to me. I get that you don’t agree with the examples but I think that’s something entirely different from claiming that they aren’t there.

                                          1. 2

                                            Yes, datetime parsing and flag handling is what I was refering to.

                                            Go is developed by smart people, I never assumed they are not aware of how timestamp formats look like in other languages. I assumed they did it differently anyway ignoring the established practices, that’s why i wrote ‘ignorance’. Maybe not the most fortunate wording, didn’t mean to offend.

                                            1. 0

                                              That’s, uh, not what “ignorance” means…

                                    1. 5

                                      The time example is kind of silly, usually one argument is a variable, the other is a constant format string so you can easily deduce which is the format string.

                                      1. 1

                                        The ‘silliness’ is exactly why I picked that example. :) I did that to make a point that the format and the value to be parsed look the same.

                                        1. 1

                                          That’s the GP’s point: in real world code, they don’t! This is typical code:

                                          time.Parse(ExpectedFormat, value)

                                          e.g. here’s code from my own project:

                                          https://github.com/contribsys/faktory/blob/master/util/util.go#L144

                                      1. 7

                                        I’m returning to a go project at work and feel a little more sensitive to these comments (so take with a grain of salt).

                                        • Timestamp parsing - I never do timestamp parsing without looking at the documentation (Oracle’s vs Java’s vs C#‘s vs SQL Server vs Javascript), so i feel that Go’s formatting being based on a magical date is “quirky”, but you can easily do date formatting/parsing just by referring to the one magical date ( 01/02 03:04:05PM ’06 -0700 ) which seems pretty cheap for the large amount of flexibility.

                                        • XML - I’ve dealt with more of this than I’d like and ran into awkward parts when dealing with multiple namespaces (canonicalization is a pain) but for straightforward xml, Go is pretty delightful to deal with. I haven’t needed anything faster or more complicated.

                                        • Flags - Again, Go is fairly idiomatic (based upon very old-school Unix command line sensibilities), but the Flags package gets so much stuff absolutely correct (built in help, connects to custom object parsing, default values). I’ve never used it but I believe the Go community really likes Steve Francia’s cobra, if you want/need more complicated stuff.

                                        • The compiler is one of the things that has gotten more complicated (dealing with multiple modes for dynamic objects, etc), but everyone appreciates it being:

                                          • cross platform
                                          • fast
                                          • produces fairly fast code
                                          • debuggable

                                          and the Go team spends lots of time keeping those in mind.

                                        • Modules - My last usage of Go is actually before modules landed so I need to get caught up on them. I never really liked some of the awkwardness of GOPATH (especially since there were multiple tools that handled pinning versions).

                                        • The last comment is… interesting. I’m pretty confident that the Go team isn’t ignorant of “well established patterns that are proven to work”, but have strong (idiomatic) design opinions. I happen to like the design aesthetic of the language, but realize it’s not for everyone.

                                        1. 3

                                          I’m a bit curious about the magical date you mentioned, how does go deduce from that whether it is the 1st of february or the 2nd of january?

                                          1. 6

                                            It’s not that magical. It’s basically just the same as “classic” string parsing with strptime() and %Y and whatnot, but instead of %Y you use 2006, and instead of %d you use 01, etc.

                                            The idea is that instead of %Y-%m-%d you have a more human-readable 2006-01-02. It’s quite easy to memorize too, since 2006 is the year, and after that it just increases from high to low. Also see some examples from the docs.

                                            In other words, it’s a lot less magical than strptime(), IMHO. For example for this comment I looked up if it was %m or %M for day-of-month, and then it turns out it’s actually %d.

                                            1. 1

                                              The idea is that instead of %Y-%m-%d you have a more human-readable 2006-01-02.

                                              One can only imagine the ease-of-use if they had picked an order that was used in more than one country on this planet. Feels a bit like the usual jingoism.

                                              It’s quite easy to memorize too, since 2006 is the year, and after that it just increases from high to low.

                                              It’s timezone > year > second > minute > hour > day > month. There is nothing logical about it, except that it looks the way some American’s prefer to write their date.

                                              1. 1

                                                I think the other part is that each portion has a different number (the month is always 01, the day is always 02, etc).

                                                1. 1

                                                  Yes and that is ordered (“intuitively”) timezone > year > second > minute > hour > day > month.

                                            2. 4

                                              If you work with go dates regularly you end up memorizing the order pretty quickly.

                                              That said, I think it’d have been nice if they’d used the 13th December instead since 2006-12-13 is less ambiguous.

                                              Unfortunately, that would’ve meant not having a nice way to make _2 indicate no leading zero. No easy tradeoffs.

                                              1. 2

                                                You have a better memory than me. I’ve been with Go since 1.3 and I still have to look up the documentation every time. I think that if they wanted to do dates like this, they should have gone with the ISO format 2000-01-02 03:04:05. That would have been way more intuitive to way more people.

                                                1. 1

                                                  It’s less the quality of my memory, and more that I was working with date parsing on a regular basis for a couple of weeks. If you don’t do it often there’s no way you would memorize it.

                                              2. 2

                                                It is a bit American centric, but the date in documentation is also written

                                                Mon Jan 2 15:04:05 MST 2006

                                              3. 3

                                                Thank you for the feedback!

                                                Timestamp parsing - I never do timestamp parsing without looking at the documentation.

                                                To me %m just seems more intuitive than 01 for referencing the month, because ‘m’ is a mnemonic for month, but ‘01’ doesn’t tell me much. One thing it does tell me though, is that the number to be formatted is 0-padded, which is admittedly a nice touch.

                                                […] the Flags package gets so much stuff absolutely correct […]

                                                Yes, agreed. I think the API is great, I just criticized the perceived quirkiness of the resulting command interface.

                                                Go is not a bad language (far from it), I just believe some of the bold design decisions to not be the best choice.

                                              1. 10

                                                Speaking specifically about the flag anomaly, I much prefer Go’s package because it removes ambiguity. In a getopt(1) program, when I see -abc, this could mean two separate things: either there are three flags -a -b -c written in a shortened form, or there is a single flag -a which takes a string argument bc. Go’s flag removes this ambiguity entirely.

                                                1. 8

                                                  It doesn’t, because you as an end user don’t know if the program is written in Go, or if the author wrote their own argument handling because they didn’t like the flag package.

                                                  1. 2

                                                    Still, clarity is something to strive towards as we develop software.

                                                    There are other reasons I prefer flag as well. For example, it is possible to provide boolean flags with true defaults, and to disable them with -name=false. This is in contrast to getopt(1)-style flag handling where individual character flags can only be toggled on.

                                                    1. 1

                                                      I personally am a fan of having both — short and long options — around.

                                                      I use the short options when typing in a shell, because it’s convenient. I also usually bunch together multiple options, e.g. ls -ltr, just because of typing efficiency. (I also type python3 -mvenv env without space between the option and the argument, sue me!) For shell scripts on the other hand long options might make code more readable/self-explanatory, especially if co-workers also have to maintain it.

                                                      That’s why I like having the single-dash prefix reserved for short options and double-dash for long options, because both variants have their place.

                                                1. 6

                                                  Note that both -flag and --flag work with Go’s flag package, although the single - is more conventional.

                                                  1. 3

                                                    People will use whatever -h tells them is available. And it shows the single dash options.

                                                    1. 1

                                                      Interesting, I didn’t know that. Thanks for pointing it out!

                                                    1. 14

                                                      I think the only other occurence of long options with single-dash prefix I have ever seen is in X11 tools. Why break with the de-facto standard way of doing things?

                                                      find(1) would also be a popular example, but I’m quite sure I have seen a few more.

                                                      1. 4

                                                        Oh yeah, I forgot about that one, even though I use it almost evey day. ;)

                                                        1. 2

                                                          Also another example I remembered would be qemu.

                                                        2. 1

                                                          and gcc

                                                          1. 3

                                                            That’s tricky, because most of gcc’s flags are of the form -[one letter key][rest of the word is the value], which is kind of another thing again?

                                                        1. 15

                                                          Upvoted because interesting topic and because it’s authored by the poster. I appreciate to see that on lobsters!

                                                          Assuming that the author is reading the comment, I have some feedback though:

                                                          Content

                                                          There is not much new I learned from the article, but to be honest, I have already read a lot of stuff about syntax highlighting and color schemes. I used vim without syntax highlighting for many years (out of conviction), but semi-recently switched back to having it turned on without looking back.

                                                          What I never found is hard data on what color scheme / mechanism of syntax highlight (or lack thereof) is objectively most readable, if that is even possible. The linked article is no exception in stating something for a fact without supporting evicence.

                                                          I like the idea of having a color scheme that doesn’t overdo the highlighting. Which theme are you using, or did you assemble your own theme?

                                                          Usability of the website
                                                          • The link is not a deeplink to the article, when you open the page, all the articles are collapsed. It took a bit of mental effort to realize that this is the case. I almost closed the tab again, without reading the article.
                                                          • On mobile the picture is very small and it can’t be zoomed. It would already have helped to put it in a link tag that points to the image file.
                                                          1. 8

                                                            Re: Usability, I actually did close the tab and only reopened it when I saw your comment explaining what was going on. I thought the site was just broken, the mouse cursor never changed into anything that looked clickable (like it usually does for links).

                                                            1. 7

                                                              The linked article is no exception in stating something for a fact without supporting evicence

                                                              true, these are just my musings.

                                                              Which theme are you using, or did you assemble your own theme?

                                                              overdone colors (almost everywhere) have driven me to assemble my own monochrome themes for every program I use:

                                                              you can find the relevant configuration files here, the vim color scheme is a fork of another popular “no nonsense” color scheme.

                                                              Usability of the website

                                                              noted. ill make changes. thank you for your feedback!

                                                              1. 1

                                                                I’d love to see a Sublime Text theme. Oh, and a Nova one too, once Panic releases it.

                                                                Nice concept!

                                                            1. 4

                                                              Upvoted this article, because the topic is interesting, but not sure I concur with the conclusion and proposed solution of the author. I feel like there are good reasons to prefer ints over floats for time durations: easier to understand, less edge cases and maybe other reasons.

                                                              When using std::chrono::minutes you basically sign up for time durations with a resolution of one minute. Not that suprising that adding seconds to that won’t work out nicely. Good API decision? - Debatable, but certainly not as horrible and unusable as made out by the author. The article seems quite arrogant to me.

                                                              I personally would take ints over floats most of the time. (pun intended)

                                                              1. 5

                                                                I agree. My understanding of std::chrono‘s types is basically that they’re type-safe time units, C++-style. Of course you can’t add seconds to minutes, because they’re different types. Of course casting seconds to minutes backfires for this reason, just like char c += static_cast<char>(int{ 300 }) would. If you want typesafe units in C++, std::chrono is exactly the kind of API you come up with.

                                                                I think I’d agree with the author that, more due to C++‘s unfortunate arithmetic behavior than anything else, std::chrono arguably has less intuitive interval results than, I dunno, Joda Time. But it’s worth noting that—again, if you think of std::chrono in terms of type-safe units—it’s not that bad, either. m += std ::chrono::duration_cast<std::chrono::minutes>(std::chrono::seconds{ 1 }); does fail because of rounding, but reversing the calculation to s += std ::chrono::duration_cast<std::chrono::seconds>(std::chrono::minutes{ 1 }); has the expected result. It might be nice if the first example didn’t compile, but, as I noted, this is basically normal C++ casting truncation.

                                                                I dunno. I’m not going to say that std::chrono is the best API I’ve ever seen, but I’m entirely with you that the author’s rant seems misplaced.

                                                              1. 7

                                                                Turning it into a static array in the data segment and then stripping the resulting binary gives you a ~7k difference and a compile time of under a second:

                                                                $ time c99 -Os foo.c
                                                                
                                                                real	0m0.539s
                                                                user	0m0.515s
                                                                sys	0m0.023s
                                                                
                                                                $ strip -s a.out
                                                                
                                                                $ ls -l a.out pg2162.txt 
                                                                -rwxrwxr-x. 1 baron baron 443064 Dec 20 12:19 a.out
                                                                -rw-rw-r--. 1 baron baron 431888 Dec 20 12:03 pg2162.txt
                                                                

                                                                I didn’t check the assembly but I’m assuming that when using the method in TFA, actual move instructions are generated for each assignment and those aren’t particularly small. Putting it in the data segment just plops the chars in there unmolested.

                                                                EDIT

                                                                Yeah, it’s generating move instructions. GCC (I didn’t test Clang) attempts to optimize for size by packing everything into 64-bit quantities and then performing the move all at once for each block of eight characters rather than moving one character at a time.

                                                                1. 2

                                                                  how did you move that data array into data segment?

                                                                  Overall I find this quite interesting, I wonder what web assembly would look like for this. It is sort of like the challenge building a prepackaged/compressed website.

                                                                  1. 2

                                                                    Essentially, changed the declaration to static const and changed the way it was populated from a bunch of assignments like

                                                                       foo[0] = 65
                                                                    

                                                                    to a static array initializer like

                                                                       foo[39393] = {
                                                                          65,
                                                                          38,
                                                                          41,
                                                                          ...
                                                                    
                                                                    1. 2

                                                                      ah, got it. thanks.

                                                                  2. 2

                                                                    I was pretty sure compilers wouldn’t just zip up the contents of a static array. That’s why I put it in code, because that’s where I know compilers perform optimizations.

                                                                    1. 2

                                                                      I had assumed as much, sorry. Wasn’t trying to steal any thunder, I was just testing for my own curiosity.

                                                                      What’s neat, IMHO, was the massive difference between no optimization and Os: almost three MB for the unoptimized version, and like 900k for the optimized one. That was pretty neat.

                                                                      1. 2

                                                                        Heh, no thunder was stolen :) I agree, the difference between unoptimized and Os is quite significant!

                                                                  1. 2

                                                                    recently I had some problems with my dance pad: getting those “Fantastics” was just too damn hard.

                                                                    This read to me at first as though it used to work and something changed. The rest of the write up suggests that it never had the appropriate polling rate and the transplant was the necessary fix, is that right?

                                                                    1. 1

                                                                      Yes, that’s right. I was just not going for accuracy before. ;)

                                                                    1. 11

                                                                      These icons look terrible to me (not an icon designer…) It seems what they did was try a few designs and ask around in some kind of association game until no one was able to associate the icon with anything and then define them to be protein and fat. The approach at least seems a good one, ask many people around the world for feedback to avoid the obvious mistakes, but maybe they took it a little too far.

                                                                      “We have accomplished our mission: keep the information simple, easy to understand, language-free and top line.”

                                                                      The “Calories” icon sure has a lot of text for being “language-free” /s

                                                                      1. 3

                                                                        I think if you judge it on “do I know at a glance what this icon means”, I’d agree.

                                                                        However, I doubt that’s their primary goal: I think McDonalds is trying to find universally acceptable (i.e., not evoking negative images) iconography so they can have one homogeneous icon set used around the world with local translations to satisfy local labeling requirements.

                                                                        In other words: it’s cheaper to do the design and layout once for all their cups, containers, etc. and a spot on the menu that says (three circles) = “salt” (in whatever language).

                                                                        1. 2

                                                                          From the five final icons, I would have only guessed the calories one correctly. The “universal icons” clearly don’t seem to be working for me.

                                                                        1. 54

                                                                          Ugh. I’m pretty happy sticking with Python 2, but this post is so bad I’m tempted to switch to Python 3. Even as a joke the Turing complete section is just stupid.

                                                                          1. 23

                                                                            I couldn’t tell whether the Turing Complete section was a joke or just profoundly confused, to be honest. Conflating the language with the bytecode, claiming that one VM can’t run the bytecode generated by another language (or maybe complaining that the bytecode changed? or that there’s no py2 compiler targeting py3 bytecode?), and trying to tie that to a fundamental property that even “languages” like SQL and Excel have ….

                                                                            It was all very muddle-headed.

                                                                            Don’t get me wrong, I know he’ll claim that it was a joke, later. That’s not in question. I’m just not sure if it actually is a joke.

                                                                            1. 20

                                                                              I don’t think this is meant as a joke. The “post” is a chapter in his book Learn Python the Hard Way.

                                                                              Difficult To Use Strings

                                                                              Wait.. what? Strings are a mess in Python 2. They have been cleaned up in Py3.

                                                                              Python 3 Is Not Turing Complete

                                                                              No comment.

                                                                              Purposefully Crippled 2to3 Translator

                                                                              It took me about one day of work to port a 70k lines django application to Py3 with the 2to3 tool. That’s was one year ago. Since then I’ve only found two bugs caused by the conversion. Doesn’t seem that bad to me.

                                                                              Too Many Formatting Options

                                                                              Yes, I can agree with that. This is the only valid criticism in that chapter.

                                                                              1. 15

                                                                                I agree, but just as a data point, it’s taken about 3 people-weeks for us to port a large (~200kloc) Django application (over the course of several months).

                                                                                The points that made this take a while:

                                                                                • We tried to maximize the amount of changes that were Py2 + Py3 compatible. This meant pulling things in over time, heavy usage of six, and catching a lot of bugs early on. Highly recommended for easy reviewing by third parties. For example: a couple changesets that were just “use six for more imports”.

                                                                                • we deal with many different encodings, and a lot of old text-to-bytes conversion code was pretty brittle. Py3 forced us to fix a lot of this

                                                                                • imports! 2to3 generated way too many false positives for our taste (hard to review the real changes), so it took a while to find the right solution (for us: a monkey-patched six that would minimize changes)

                                                                                • changes of standard API from lists to iterators generated a decent amount of busy work. Changes for the better, though.

                                                                                • Handling binary files. Here, too, we were often doing the wrong thing in Py2, but it would “just work” before.

                                                                                • Lots of dependencies that we needed to upgrade to get the Python 3 support.

                                                                                • Pretty minor things around bytes’s defautl __str__ method. For example, checking the output of a process call, we would do if output == "'0'", and that would fail because `“b'0'” != “‘0’” but that turned out to cause more issues down the road.

                                                                                • issues around pickling + celery. Our solution mostly centered around lowering usage of pickle even more (dangerous)

                                                                                • deployment issues. Juggling Python 2 tooling and Python 3 tooling in the CI pipeline would sometimes mess things up.

                                                                                1. 4

                                                                                  I can only recommend https://pypi.python.org/pypi/modernize

                                                                                  Instead of translating x.iteritems() to x.items(), it translates it to six.iteritems(x), and adds the six import. Fixes 80% of the boring stuff and you only need to focus on unicode/bytes, etc.

                                                                                  1. 2

                                                                                    The idea of Python 3 was to make iteration cleaner the easier to read and understand. Now we have to insert calls to six in every loop and for every string. The result is hideous code that’s harder to read, not easier.

                                                                                    1. 2

                                                                                      I was writing this because

                                                                                      We tried to maximize the amount of changes that were Py2 + Py3 compatible

                                                                                      If that is not your objective, you can just use 2to3 and be ready.

                                                                                      By the way, I really do not understand, why the CPython devs haven’t kept iteritems() as an alias to items() in Python 3 with a deprecation warning to be removed in Python 4. I cannot imagine that it would have been a massive maintenance effort. But on the other hand, making a function call instead of a method call is not rendering code unreadable. I have never heard a Python dev complain about calling str(foo).

                                                                                      Essentially, Python 3 adoption has not been slow because of “readability”, but because Python 3 bundles fairly boring changes (iteritems->items) with this massive unicode change (requiring quite a few changes to code bases) and few breathtaking new features that weren’t available in 2.7. This changes at the moment with Async, matrix multiplication operators, etc.

                                                                                2. 3

                                                                                  Python 3 Is Not Turing Complete

                                                                                  No comment.

                                                                                  Did you read the note?

                                                                                  1. 13

                                                                                    If anything, the note make it seems like he’s really serious about his Turing FUD.

                                                                                3. 9

                                                                                  Joke or not, the fact that we even have to ask that question significantly harms the credibility of the article.

                                                                                4. 5

                                                                                  Just out of curiosity, why are you sticking with Python2 for now? At this point all of the major libraries have been ported to be 2/3 compatible. In addition, while there aren’t any huge new features in Python3 (besides the byte/Unicode separation) there has been an accumulation of small quality of life improvements that from my point of view make it very much worth it to switch.

                                                                                  1. 1

                                                                                    Not sure about the comment author, but I’ve personally moved over some of my open source libraries to support both python 2 and 3. Moving larger project is tricky though as it involves updating the language, all dependencies and forking the ones that don’t support python 3.

                                                                                  2. -7

                                                                                    haha

                                                                                  1. 8

                                                                                    Personally, I wouldn’t mind removing voting on comments entirely. Most threads are small enough that if I click through, I read all of the comments anyways. Flagging/hiding is probably enough. Maybe with enough hides, it can propagate to other users to discourage lightweight snark and trolling, but I would hesitate on that.

                                                                                    As far as community norms, I have been pretty happy with what gets posted so far. There isn’t much trolling, the comments are generally good, and the topics are interesting. I am not sure what people want to change or enforce here.

                                                                                    1. 4

                                                                                      Maybe not so much change as define what they are for new users and the scope of moderator involvement.

                                                                                      1. 3

                                                                                        In that case, I’d strongly prefer to keep it as lightweight as possible. I’d like this community to trust its members as much as is practical. And I’d like to trust the moderators to act as benevolent dictators.

                                                                                      2. 4

                                                                                        As far as community norms, I have been pretty happy with what gets posted so far. There isn’t much trolling, the comments are generally good, and the topics are interesting. I am not sure what people want to change or enforce here.

                                                                                        I fully agree and I also think upvotes/downvotes are one of the reasons this is the case.

                                                                                        1. 2

                                                                                          My first internet communities were on Usenet, and some of the best discussion I remember was on there. Individual killfiles and moderator removal of persistent trolls seemed to be enough.

                                                                                          Nostalgia may be tinging my memories, but I suspect the lack of pressure for approval could help increase comment quality.

                                                                                          1. 1

                                                                                            It’s really hard to assess that sort of thing … a community can only count on feedback, in general, from people who actively participate in it. When people look in but find a place too hostile to want to get to know, so they leave without ever posting… you’re lucky to ever hear about it. And feedback from people who don’t actively participate is generally ignored, even when it is given.

                                                                                            1. 1

                                                                                              I’m not sure why you immediately jumping to the assumption of hostility. Looking at some groups I was on (eg, comp.compilers, https://groups.google.com/forum/#!forum/comp.compilers), it definitely isn’t the case.

                                                                                              1. 3

                                                                                                Killfiles, and any ignore mechanism, seem to imply it to me. I do realize that they aren’t necessarily a sign of an unhealthy community; just a particular type of community. But please read my above remark as agnostic to the specific nature of why newcomers might be turned off.

                                                                                                I was on Usenet during that time period, and I can’t say I ever felt welcomed enough to participate more than minimally, but I don’t think it was for any reason now under discussion, so I’m just noting it to say that we do have a little shared context here.

                                                                                        2. 1

                                                                                          Most threads are small enough that if I click through, I read all of the comments anyways.

                                                                                          That’s a good point. While the community is low in number, it’s very easy to just scroll through everything. Doing several Lobsters threads is like one on some other sites. I can sometimes do the whole site that way as there’s more people reading stuff than commenting on it. Probably A Good Thing.

                                                                                        1. 9

                                                                                          This article could be like two paragraphs. It repeats itself all over. Multiple times.