1. 9

    Writing good shell scripts takes a lot of discipline and effort if you ask me. On the long run, after some practice it becomes easier though. I disagree with using bash idioms everywhere though. POSIX sh is sufficiently complex to do everything you need, whill keeping things simple enough so you don’t get surprised by bash’s « cool features ».

    An example would be rhe [[ ]] constructs, which can be very misleading:

    chmod 0400 file.txt
    # format %a returns permission in octal mode
    mode=$(stat -c %a file.txt)
    if [[ "$mode" -eq 0400 ]]; then
        echo "File is read-only!"
    fi
    

    I think that all the bash features turn shell scripting into a full language, which it is not. Shell scripting should only be some glue between actual programs, where the actual logic is. Another advice I would give is to try to keep every shell script under 100 lines, so they remain manageable and auditable. More lines usually means that the programs you are wrapping are lacking some features, or are hardly composable.

    1. 1

      Another advice I would give is to try to keep every shell script under 100 lines, so they remain manageable and auditable. More lines usually means that the programs you are wrapping are lacking some features, or are hardly composable.

      Yes and no. My current project has several tens of thousands lines of bash, with some scripts in several thousand line long. And those are just calling external programs and piping them forward, doing simple if statements occasionally.

      1. 1

        An example would be rhe [[ ]] constructs, which can be very misleading:

        chmod 0400 file.txt
        # format %a returns permission in octal mode
        mode=$(stat -c %a file.txt)
        if [[ "$mode" -eq 0400 ]]; then
            echo "File is read-only!"
        fi
        

        Not sure what you mean by [[ and ]] being misleading in this case? It’s not the fault of bash that you are comparing 0400 (an octal number) with 400 (a decimal number)?

        1. 1

          I mean this:

          bash-5.0.3$ [[ '0400' -eq '400' ]] && echo decimal || echo octal
          octal
          bash-5.0.3$ [ '0400' -eq '400' ] && echo decimal || echo octal
          decimal
          

          Note: I know and understand what’s going on here. But this is definitely misleading as some people tend to use [[ ]] without even thinking about it, or understanding how an if block is supposed to work in shell. For example, I have seen this a huge number of times:

          grep -q 'foobar' file.txt
          if [[ $? -eq 0 ]]; then
              # do stuff
          fi
          

          This could be (in fully POSIX shell):

          if grep -q 'foobar' file.txt; then
              # do stuff
          fi
          
          1. 2

            I would argue that [[ and ]] is doing the least surprising thing here because it is actually evaluating the arguments when in an arithmetic context. It’s the same with $((0400 + 1)) evaluating to 257 and not 401.

            Interesting, in zsh, $((0400)) evaluates to 400, even though $((0x400)) evaluates to 1024. Now that’s surprising and kinda buggy! But that’s why I don’t write shell scripts in zsh ;)

            1. 2

              That’s indeed surprising of zsh because the arithmetic expansion $(( )) is the only place (according to POSIX) where octal numbers are to be interpreted by the shell.

              Interestingly, mksh does the same as zsh here, and express it explicitely in the manpage:

              Arithmetic expressions
              […]
              Prefixing numbers with a sole digit zero (“0”) does not cause interpretation as octal (except in POSIX mode, as required by the standard), as that’s unsafe to do.

              As for [[ ]] vs. [ ], this makes sense because [[ ]] is a shell construct, while [ ] is a program (see test(1)). If the shell where expanding numbers with a leading 0 as octal before passing them to programs, we would be fucked.

              1. 1

                Thanks for looking that up on mksh. Explains alot. But I wonder why it is unsafe?

            2. 1

              Note: I know and understand what’s going on here. But this is definitely misleading as some people tend to use [[ ]] without even thinking about it, or understanding how an if block is supposed to work in shell. For example, I have seen this a huge number of times:

              grep -q 'foobar' file.txt
              if [[ $? -eq 0 ]]; then
                  # do stuff
              fi
              

              Yes, this is absolutely horrible and stems from the fact that people do not know that all if is supposed to do is execute its argument and respond to the return value.

              Some trivia for geeks: if [ 42 -eq 41 ]; then ... is actually just executing /bin/[ with the four arguments "42", "-eq", "41" AND "]". /bin/[ simply ignores ] as syntactic sugar. Example:

              sh-5.0$ /bin/[ 0 -eq 1 ]; echo $?
              1
              
        1. 14

          I really dislike websites that don’t even pretend to work without javascript enabled. This site gives you a 100% blank page with JS disabled. They don’t even bother to tell you, hey, we hate your non-JS ways, so you can’t play here. Who cares about accessibility or sanity.

          1. 8

            I imagine being a website for javascript developers, they don’t bother considering users that don’t have js enabled. Not that I agree with that sentiment.

            1. 8

              It’s just a discourse server, and a relatively vanilla one it seems at that. Whether nojs works is probably just however much discourse is designed to support it, and not a decision of npm.

              EDIT: Discourse supports nojs fine, zie’s browser is just misconfigured

              1. 4

                Just to wrap this up, I agree, my browser is sort of misconfigured, I used uBlock Origin with the global setting, and it does not work, nor does imgur, and a bunch of other websites.

                I went digging around the various extensions/addons for other options, and I settled on LibreJS[0].

                similar to noscript, it allows some JS to run, but unlike noscript it doesn’t automatically trust random corporate websites just to make users lives easier. Plus the UI is a lot easier than noscript, and allows one to easily see the source of the JS(both the URL and the actual JS source code) before whitelisting it..

                So I think this is maybe the best of both worlds. imgur and other sites basically work, you can easily whitelist specific things as needed, and by and large most every JS 0day exploit will be a non problem.

                Thanks everyone for helping me better understand the crazy insanity that is browsers and JS these days.

                0: https://addons.mozilla.org/en-US/firefox/addon/librejs/

              2. 4

                This reminds me of the brief time I was working for a media company. Developers needed to interface with ads servers and nothing worked until they disables ad blockers… :D

              3. 3

                You must be one of those wierdos that prefer plain text email.

                (I’m genuinely not being an ass, I’m a wierdo just like you)

                1. 3

                  It’s almost like the content of the email is what I’m there for :)

                  1. 1

                    Maybe I am :) hehe

                  2. 2

                    That’s strange, it worked fine for me https://i.imgur.com/lmC3FID.png

                    1. 2

                      imgur doesn’t work without JS either :P

                      1. 2

                        I linked you directly to the image, there is no html let alone javascript on that URL. You can curl it if your browser is misconfigured to just block all imgur domains.

                        Relevant headers:

                        HTTP/2.0 200 OK
                        last-modified: Fri, 28 Jun 2019 15:07:43 GMT
                        content-type: image/png
                        date: Fri, 28 Jun 2019 16:08:22 GMT
                        server: cat factory 1.0
                        

                        Cat Factory 1.0, great name for a web server.

                        1. 4

                          Imgur has been redirecting direct image links to the main site for a while now (see https://minimaxir.com/2014/02/moved-temporarily/.

                          I couldn’t reproduce the redirect using curl, but clicking that link on the browser will go with the redirected version.

                          1. 2

                            It depends on your user agent and accepts request headers.

                            1. 2

                              From FF, it redirects to the full page, and the image doesn’t display(assuming you have JS totally disabled). Which makes imgur totally broken, the entire point of using a site like imgur is the showing an image.

                              AKA imgur is totally unusable without JS for anyone in a browser.

                              The noscript FF extension and others actually do allow some JS to run, by default, to fix broken sites like imgur.

                              But uBlock Origin, which totally disables JS(if you set it as a global default) imgur doesn’t serve it’s primary purpose of showing an image.

                              Unless you re-think their primary purpose is to make money, not show images, and then maybe they are doing just fine.

                              1. 2

                                From FF

                                You’re already using an adblocker I recommend you try UA switching.

                                Unless you re-think their primary purpose is to make money

                                This is the goal of every business, for worse or, well, for worse.

                      2. 2

                        It seems to be a difference between disabling JS and just blocking scripts. zie, if you’re uMatrix, you can enable ’Spoof <noscript> tags` in the ‘…’ menu to be able to see the noscript block of the page.

                        Edit: <noscript> was getting eaten by the markdown parser.

                        1. 1

                          disabling JS and just blocking scripts

                          OK I think what you are saying here is 1st party JS vs 3rd party JS? Or are you trying to say something else?

                          I’m not really up to speed on JS and browsers, as I basically just leave it disabled all the time unless I really need to see some site that requires it.

                          I use Ublock Origin, and it’s settings near as I can tell is just all on or all off.

                          1. 2

                            In HTML there is a <noscript> tag where content is shown if the web browser doesn’t support javascript. For some reason your blocker is blocking the noscript tags LOL

                            1. 3

                              Not that it’s blocking it specifically, more that js is “enabled”, but just prevented from loading, so the browser hides the <noscript> blocks as it it normally would.

                              Weirdly though it sounds like uBO should make the browser show noscript tags: https://github.com/gorhill/uBlock/issues/308#issuecomment-417812479 Maybe “that new per-site switch” isn’t what you’re using, zie?

                              1. 2

                                Right, I disable JS globally, not using a per-site switch. the per-site switch requires you to at least load the JS once, to even get access to the switch. That is a terrible security position. This story[0] is a good reason why.

                                0: https://lobste.rs/s/vi8ybf/i_was_7_words_away_from_being_spear_phished

                      3. 1

                        Yeah. E.g. every time I read the docker documentation i get a headache. I run noscript

                        1. 1

                          noscript by default, actually allows running of JS. both on every single global page, and on a bunch of domains that come pre-trusted.

                      1. 21

                        It would also be a way better email if you dropped all the HTML shenanigans.

                        Composing better emails? Plain text should be number 1 on the list.

                        1. 14

                          Personally disagree on this. A proper HTML link is almost always cleaner than “(see link below)”. You can’t underline stuff (you can “put asterisks around stuff” but…). Sometimes you want to just reference an image inline!

                          There’s a reason that word processors are a big business. Laying out a message nicely aesthetically is valuable for human consumption! The answer to “people always misuse HTML layout” isn’t to get rid of HTML layouts, it’s to teach people how to use it nicely!

                          1. 2

                            I totally get what you mean, but emails are not meant for rich text.

                            Link to a shared document (or an HTML page!) if you need to convey some information that requires media.

                            1. 1

                              I will admit to preferring plain text, but if HTML is used sparingly, it’s fine. That said…

                              A proper HTML link is almost always cleaner than “(see link below)”.

                              This is true, but practically no one does this. Certainly in the 10+ years I’ve been working in corporate environments, no one makes the effort to do a proper link.

                              The answer to “people always misuse HTML layout” isn’t to get rid of HTML layouts, it’s to teach people how to use it nicely!

                              That time has come and gone. The only way this is likely feasible is if you change the tools in some way. Teaching people to use something “correctly” when there are a myriad of (easy) ways to use it incorrectly is a losing battle.

                            2. 10

                              Sorry to be blunt, but I don’t think it’s good that plain text email is such a shibboleth that you can say the equivalent of “I prefer plain text email” without giving any justification or discussion, and it will be the top comment on an article.

                              There are good arguments against HTML email, there are good arguments that we should support some form of formatting, whether or not it’s HTML (see the sibling post by rtpg). Whichever view is right, I don’t think it should just be assumed without any attempt to argue for your opinion.

                              1. 2

                                I’m all for formatting. I regularly use markdown-style formatting in my plain text mails, and I’m an avid user of references[1] for links.

                                HTML formatting in emails is an abomination. Period. It’s a hack. It causes all kinds of issues; it enables phishing, automatic “read notifications” that you did not approve of and difficulties for people with a need for screen readers, just to name a few. Not to mention the security vulnerabilities in clients that have resulted from trying to support this crap.

                                [1] Like this.

                              2. 7

                                I agree but this is not realistic in a world where everyone usees Outlook. I swam against the tides and ran mutt at work for years, until one day I missed a critical update from my manager that used rich text to denote something in red.

                                IMO This is a lost cause, but feel free to rage against the dying of the light :)

                                1. 2

                                  Up voted for poetry. :)

                                  1. 2

                                    Alternatively, you can go work somewhere where the managers use mutt. :) https://boards.greenhouse.io/wikimedia/jobs/1623040

                                    1. 1

                                      Yup it’s all about choices.

                                      I’m willing to run Outlook as my mailer and deal with a bit of large corporate white noise because the value I derive from working here far FAR outstrips those minor annoyances.

                                      Everybody has to do their own cost/value curve calculations though.

                                    2. 2

                                      IMO This is a lost cause, but feel free to rage against the dying of the light :)

                                      Don’t worry; I will! https://p.hagelb.org/line.jpg

                                      1. 2

                                        +10 for an entirely apropos ST:TNG

                                      2. 1

                                        It is of course unfortunate that you missed such an important update. This is where it would make sense to use the Subject header to emphasize the importance of the message, such as the use of [URGENT] or [CRITICAL].

                                        I totally agree that this is a lost cause, but yeah, I will continue to fight for the cause :)

                                        1. 2

                                          It is of course unfortunate that you missed such an important update. This is where it would make sense to use the Subject header to emphasize the importance of the message, such as the use of [URGENT] or [CRITICAL].

                                          It had that, but as I said in the post, it was like:

                                          Blahblahblah

                                          <SUPER CRITICAL STUFF IN RICH TEXT COLORED RED THAT MUTT CAN’T SEE>

                                          blahBLAHblahblah.

                                          So yeah, no hope at all other than “Don’t use rich text.”

                                          In my work environment, I know of literally maybe 2 people in a team of 150 who use mutt. I don’t know what the stats are for the wider company, but I know it’s a TINY fraction.

                                          Expecting my manager to cater to my needs and preferences to this extent is unreasonable in my book.

                                          1. 1

                                            maybe 2 people in a team of 150 who use mutt

                                            OK, but how many are colorblind?

                                            1. 3

                                              You’re preaching to the choir.

                                              I personally think leaving critical information to the vagaries of color is a mistake, but it wasn’t my call. I just need to roll with the punches and deal with the technology environment I’m given.

                                              Yes, I know, I could go work over at $PERFECT_COMPANY and all manner of things would be well, but having to run Outlook and deal with rich text in my E-mail isn’t enough to blunt what is otherwise a really compelling value prop for me in this job.

                                    1. 3

                                      A long, long time ago, I had a boss who would say “This Linux thing will never catch on because no one knows how to pronounce it. LIE-nux, LEE-nooks… DEEB-ian, deb-EE-an… it’s doomed.”

                                      He may have been right for the desktop… (-:

                                      1. 1

                                        I consistently say “Mac Oh-Es Ex” but apparently the consensus is “Mac Oh-Es Ten”?

                                        1. 2

                                          “Tencode Ecks Beta Six”

                                          1. 1

                                            I say it Os-Es-Ex, and no has corrected me.

                                            1. 2

                                              If you walk around Apple HQ saying “oss ecks” (with the “oss” being the same sound as in “hoss” or “cross”), people get really mad at you. I’ve also been saying “eye oss” for so long I’ve forgotten it was initially a joke, and have probably weirded out a few coworkers after I switched jobs.

                                              1. 3

                                                Thank goodness they dropped it and it is plain old macOS now ;-)

                                            2. 1

                                              Not just consensus, that’s how Apple employees pronounced it in keynotes.

                                              Now, I know that’s basically the “GIF argument”, but “OS Ten comes after OS Nine” actually makes sense, unlike “Jraphics Interchange Format” :)

                                              (and now it’s pronounced mac-O-S anyway)

                                          1. 6

                                            I’ve found that the Canonicalize-family of options are my best friends.

                                            CanonicalizeHostname yes
                                            CanonicalizeFallbackLocal no
                                            CanonicalizeMaxDots 0
                                            

                                            This allows me to do stuff like

                                            Host *.domain.tld
                                                User ansible
                                                IdentityFile ~/.ssh/ansible
                                            
                                            Host host1 host2 host3
                                                HostName %h.domain.tld
                                            

                                            Such a setup allows me to not rely on the system resolver for hostname lookups. It works by canonicalizing hostnames and then re-reading the configuration file. You can see how this is done with -v and how options are applied. But most importantly, it allows me to easily overwrite the default options for some hosts. Something that does not “just works” without the CanonicalizeHostname option.

                                            Host gerrit.domain.tld
                                                User birkelund
                                                Port 29418
                                                KexAlgorithms +diffie-hellman-group1-sha1
                                                IdentityFile ~/.ssh/id_rsa
                                            
                                            1. 1

                                              WTF? You have to sign in to read medium articles now? Do they think this will make me read more articles? Sigh, another domain for the shit list.

                                              (Nobody else complaining out of politeness? Or you all signed up already? Or did I just win the A/B lottery?)

                                              (So it seems to come and go. When I’m “lucky”, it says “you’ve already read one article this month. Sign in to read more. Sign in with Google. Sign in with Facebook. Sign in…” but there’s no way to avoid signing in. Switch browsers, no popup.)

                                              1. 4

                                                I’m for this change, it will make it much easier to not read Medium.

                                                1. 1

                                                  I’m not signed in, but I remember that Medium bugged be once to log in. I do believe that I had the option to choose “go away and don’t bug me again”.

                                                  1. 1

                                                    It doesn’t ask me to sign in (Safari on iPhone), FWIW.

                                                    1. 1

                                                      Works for me on mobile

                                                      1. 1

                                                        I’m on desktop Edge. It’s not asking me to sign in. Perhaps it knows you have an account? Try clearing your cookies.

                                                      1. 6

                                                        I suggested a title change for the link to add (2014) to it…

                                                        1. 2

                                                          Yes, the bug is from 2014, which makes it all the more “interesting” that it’s still not closed and got the “serious” tag in June 2017.

                                                        1. 7

                                                          They have a StartCom SSL cert on ipfs.io.

                                                          1. 3

                                                            I was really hoping you were joking. But no.

                                                            1. 6

                                                              Be careful who you get your advice on juking nation-state adversaries from.

                                                          1. 17

                                                            My research group runs our entire grid computing (that is, a “cloud” for you young ones) infrastructure from a LackRack in an old bathroom. We’re poor I guess.

                                                            1. 3

                                                              Now to wait for the sidebar patch to be finished for 1.6…

                                                              1. 6

                                                                Good news - the neomutt project has you covered.

                                                                1. 2

                                                                  Wow - did not know about that one. Thanks. Looks good!

                                                              1. 6

                                                                karelzak’s patched version of mutt adds notmuch integration making it stupidly fast, even with huge mailboxes. It’s based on the development version, so I think everything in upstream v1.6 is included.

                                                                1. 3

                                                                  Copenhagen, Denmark.

                                                                  1. 1

                                                                    What is this feature? I don’t see it plugged in to anything, but maybe I suck at searching:

                                                                    OpenBSD

                                                                    OpenSSH

                                                                    EDIT: Now that the patch went out I can see why my search wasn’t working. s/UseRoaming/use_roaming/

                                                                    1. 1

                                                                      It’s undocumented. edit: oh, I misunderstood your question, sorry!

                                                                      1. 1

                                                                        I can’t even find it being used in the code.

                                                                    1. 2

                                                                      This is extremely interesting! I’d like to take a look at the diff–anyone have a link? Given the succinctness of the plege interface, I assume its minimal, but strategic.

                                                                      1. 3

                                                                        You can find the patches in the ports tree. Check the patches directory. The commit lists the files involved.

                                                                        1. 1

                                                                          Wow! Thanks for the direct pointers!

                                                                      1. 2

                                                                        Someone please explain this one. I’m going crazy.

                                                                        edit: BTW - same behaviour on python3 (3.4.4).

                                                                        1. 2

                                                                          This is a great showcase for httpd(8) actually. I don’t think I’m the only one who has fought a fight with nginx or apache to get cgit running nicely.

                                                                          1. 5

                                                                            Very nice. The lobste.rs interface isn’t too bad on mobile already, but I’d love a native implementation. Great work and I look forward to testing out the beta.

                                                                            1. 6

                                                                              FWIW there is also Pinchy.

                                                                              1. 2

                                                                                Last commit to Pinchy was on May 5, 2014. Maybe abandoned?

                                                                                1. 9

                                                                                  It might actually just be “finished” :)

                                                                                  1. 6

                                                                                    It would be the first piece of software in history to be such then ;)

                                                                                  2. 4

                                                                                    I actually pushed an update for it recently enough (June 2nd), after someone emailed me with a request. I guess I forgot to push the source to github though.

                                                                                    I swapped to iOS recently, so haven’t been actively developing, because I don’t use it. But I’m happy to update it if people have suggestions

                                                                                    Edit: pushed latest to dev branch ….

                                                                                2. 1

                                                                                  Thanks! The invites are on their way!

                                                                                1. 8

                                                                                  Learn You Some Erlang for Great Good is also worth mentioning here then :)

                                                                                  1. 3

                                                                                    I’ve found a couple of good papers via lobste.rs, so I thought I’d share a paper I chose for a reading group in case others are interested.

                                                                                    1. 4

                                                                                      Agreed. Good papers seem to surface on lobste.rs from time to time and this is a great submission. Keep it up!