Threads for jamesog

  1. 23

    Maintenance cost of bash scripts is lower

    I would strongly disagree with this. Unless the script is very simple - only single commands, few variables - shell scripts can have many pitfalls that many people don’t realise until things go wrong. Pipelines and subshells and their side effects are not widely understood, for example.

    c.f. https://lobste.rs/s/5ldmtp/pipefail_how_missing_shell_option_slowed -> https://lobste.rs/s/wl6kem/simple_script_is_still_someone_s_bad_day

    Don’t get me wrong, shell scripts do have their place, but for anything even vaguely complex they’re generally the wrong choice. Use the right tool for the job.

    Oh and lastly:

    Every machine has bash installed

    As a BSD user, this is not true. :-)

    1. 4

      Unless the script is very simple

      You can do a lot of useful stuff with simple bash scripts. I probably write one per week. Lets look at an example:

      https://github.com/no-gravity/git-retroamend

      4 lines of code. Saves me a lot of hassle every time I need it.

      Pipelines and subshells and their side effects are not widely understood

      They can even be indeterministic:

      https://www.gibney.org/the_output_of_linux_pipes_can_be_indeter

      1. 2

        You can do a lot of useful stuff with simple bash scripts.

        I’m not disputing that. I’ve been writing shell scripts for a very long time, and I use them where they’re appropriate. What I am disputing is your statement that the maintenance cost of shell scripts is lower compared to other languages. If you’re only ever writing simple scripts this is often true; but if you’re comparing shell to other languages there was probably a need to use the other language in the cases they were used.

        They can even be indeterministic:

        Indeed. You’re kind of making my argument for me. :-)

        1. 5

          With “maintenance cost” I do not mean the cost to change the functionality. I mean that from time to time you have to change your script because the language changes. I expect Bash to have less of these breaking changes than most other languages.

          1. 3

            from time to time you have to change your script because the language changes

            What? I mean, there was python 2 to 3 and Ruby 1.8 to 1.9, but I don’t think breaking language changes are common?

            1. 5

              Python in my opinion is an example for high maintenance cost. They sometimes do backwards-incompatible changes within a major version. For example Python 3.7 introduced “async” as a reserved keyword, breaking code that used “async” as a variable name.

              If you follow each version update of Python, you will probably recognize all breaking changes as deprecations in earlier versions. But if I would just write a script, leave it alone for 10 years and then try to run it with the latest Python version, I would not bet on it running without errors. Whereas for bash scripts I would assume they still work.

              But for bash scripts it totally depends on which programs you call.

              1. 1

                The needless breaking changes are my least favorite parts of Python and Node.

        2. 2

          You can put this directly in your .gitconfig. I have the following in mine:

            # "commit fix <commit>" - Add index files to given commit without changing
            # its message.  For more control, create a temp commit, then "rebase -i" and
            # use fixup and edit (which lets you change commits msgs)
            cfix =  "!f() { \
              local committofix=${1:-HEAD}; \
              git commit --fixup="$committofix"; \
              git rebase -i --autosquash "$committofix"^; \
            }; f"
          
        3. 0

          Also NixOS.

        1. 10

          I do something similar, but in a more generic way. I have two “search engines” defined in Chrome, one called Jira and the other called JQL.

          Jira is defined as https://company.atlassian.net/secure/QuickSearch.jspa?searchString=%s JQL is defined as https://company.atlassian.net/issues/?jql=%s

          The Jira one means I can do jira<space>project foo which activates Jira’s “smart” search and, if project is a valid Jira project, searches for foo in that project, or otherwise does a more generic text search for project foo; and jira<space>project-1234 also takes you right to ticket PROJECT-1234 as in this blog post.

          For those that use Alfred on macOS you can also create a similar search there, using {query} instead of %s

          1. 4

            Yes, Alfred is what I use and it’s extremely useful for this. My shortcut is jira and putting in a ticket number will go straight to it. I also created a workflow to take the full URL of a Jira ticket from my clipboard and replace it with just the ticket. All this saves a great deal of tedium.

            If you’re on macOS I can’t recommend it enough. https://www.alfredapp.com

            1. 2

              Yes the QuickSearch one is great, and that’s what I have j aliased to in Firefox. You can type in a ticket ID or a search term or whatever.

              One really frustrating thing is that Jira then redirects you to a combination search/ticket view and puts the first result’s ID in the URL bar. This makes it that much harder for people to reverse-engineer the search URL!

              1. 1

                Mine are very similar, I have ji for ‘JIRA issue’ and js for ‘JIRA search’.

              1. 1

                I think I’m missing something. Is English your native language, or Portuguese? Some things in the post aren’t connecting up for me.

                1. 1

                  Portuguese is my first language.

                  1. 3

                    Sorry, I guess I’m still confused then. For you to work in your native language requires empathy… on your part? Someone else’s?

                    The best I could figure is that you were looking to ask people to be empathetic to people who are speaking their non-native language, but that’s not what the title said, so I’m a bit lost.

                    1. 1

                      The article is directed towards native-English speakers, so My native language == your non-native. I understand the confusion, tho.

                      Regarding empathy, I think both sides have to be empathic. Working in a second language can be exhausting, and people often feel like they’re less valued, less intelligent or treated differently for speaking “bad English”, so I’m making a case to say that you (non-native speaker) are not alone, and it’s okay to make mistakes. For native speakers, on the other hand, I ask for empathy because you’re an expert on your language, so your very presence can be intimidating to others. Using simpler language, for example, is a way to help foreigners.

                      1. 4

                        I’m all for using simpler language, not just for helping people who are less fluent in a language, but also to reduce the cognitive overhead required to understand what is being said. I’m not a native English speaker either (I’m Dutch), but oftentimes I get annoyed by the horrible ways people use English (even native speakers).

                        For me, I approach it a bit like a programming language. It’s always good to have empathy towards your fellow developer (they might be tired, overworked or just new to the language), and if people are continually making mistakes it doesn’t make sense to chew them out, because it’s clear they’re at a language level below full mastery. But I still kind of expect a professional who has been in this business for several years to have mastered both English and the programming language at hand.

                        1. 4

                          I’m all for using simpler language, not just for helping people who are less fluent in a language, but also to reduce the cognitive overhead required to understand what is being said

                          Simpler doesn’t always help. A lot of the complicated words in English are lifted directly from French or Italian and so might be more familiar to a native speaker of another romance language than the more common English terms (I recall a French friend had a habit of just putting French words in his PhD thesis and anglicising the word ending. It worked 90% of the time and was very confusing the few times when it didn’t).

                          I once tried playing Balderdash with a group of Italian friends and it was no fun at all. You’re presented with an obscure English word and you’re all supposed to make up plausible definitions. You get one point for every person who believes that your definition is the right one. If you write the correct definition then yours is excluded from the ones people guess and you just get two points. Every round, all of them would write the correct definition because the obscure English words were everyday Italian ones.

                          Even for native speakers, there are a lot of weird dialect variations, especially where one has dropped a word from a common phrase. For example, American dropped the ‘to’ from phrases such as ‘one through to five’ and just says ‘one through five’, which sounds to an English speaker like a sequence that starts at one, continues through five, and then probably stops somewhere later, but to an American means something that stops at five. In contrast, after people in England stoped wearing pantaloons, they stopped using the abbreviation ‘pants’ and so dropped the ‘under’ prefix from ‘underpants’. Americans kept the word ‘pants’ as a synonym for ‘trousers’ and so cause hilarity to English speakers (we’re easily entertained) whenever they refer to their pants. Pants is definitely a simpler word than ‘trousers’, but saying it increases the cognitive load for folks outside of the US, who first have to (by law in the UK) smirk, and then mentally translate.

                          1. 3

                            My experience as an English person is that native English speakers are often worse at speaking our own language than non-native speakers. Working with non-native speakers for my entire career has actually helped me realise where native speakers get lazy in their use of language, or use idioms that don’t translate well (or at all) - sometimes even across different variations of “native English” such as English vs Scottish vs American.

                            1. 4

                              I’m very used to Scandinavians going “sorry, I’m not a native speaker” and delivering perfect English. (Well, it’s perfect until it isn’t - sometimes you can get very interesting mispronunciations.

                          2. 2

                            The article is directed towards native-English speakers, so My native language == your non-native. I understand the confusion, tho.

                            Ahhh, I think I see now. Thanks.

                    1. 4

                      IRC, dict pop3 and whois. Whois is terrible, we really need a structured alternative. But there it is.

                      1. 3

                        RDAP is the proposed replacement for WHOIS.

                        1. 2

                          Yes, which not all TLDs support

                      1. 2

                        Despite everything, 2021 was reasonably good for me.

                        • January: Approached by a recruiter at a great company, started interview process.
                        • February: Got an offer from said company, glad to finally resign from a company I’d been resenting for too long.
                        • March: Finished at old job, took a few weeks off.
                        • April: Start at new job, mind very much blown.
                        • May-November: Thoroughly enjoying new company and job, but realise towards the end of the period I’ve taken no time off during the year so took 2 weeks off.
                        • December: Towards end of my 2 weeks’ leave I finally got myself an e-bike to try and get out more and get more exercise. Immediately have an accident, seriously fractured shoulder joint and had to have surgery.
                        • Now: still in a sling :-(

                        As for next year, well I’ll still be recovering from the accident for most of January and will need physio for a while. So in my first year of a new job I’ll have had to take 2 months off! However the company and team are great and I look forward to keep growing at this company.

                        1. 2

                          It is a good year! I am also starting to prep for interviewing as I have stayed too long with my current employer. I failed to get a promo to staff level earlier in the year despite working my ass off. Felt depressed and burnt out after that debacle. But, in rear sight, I learned that I need to look ahead beyond this company as in “I have a business that is providing my skills for whoever could pay me decent”.

                          So leetcode and system design I go!

                          1. 1

                            Sorry to hear about your accident. I’ve been wanting to get an ebike as well.. which one did you get, do you recommend it?

                            1. 2

                              It’s made by a UK chain called Halfords. It’s a Carrera Impel im-3. From the brief amount of time I was able to use it, it’s nice. Comfortable to ride, the electric assist is great.

                              1. 1

                                awesome. I’ll check it out

                          1. 2

                            The author pronounces it [aɡe̞], like the Italian “aghe”.

                            Does the author mean aghi? Aghe is not an Itailan word.

                            Now I am confused. Is the pronunciation as ah-gee (as you would say aghi in Italian) or ah-geh (as you would pronounce aghe in Italian if that were a word)?

                            1. 3

                              It seems unlikely to me that the majority of people who encounter this library at its point of use will think to investigate how it’s pronounced. I expect most will assume it’s the English word. Naming things is hard; there are many pitfalls.

                              1. 3

                                I’m also confused. It links to google translate which translates it to “needles”, but I’ve never heard the word pluralized like that. I’m guessing it comes from FiloSottile’s dialect.

                                1. 1

                                  I think I just got it. The link to google translate is there so you can play the pronunciation and not to translate it to English. I guess that’s helpful for everyone that is not Italian, lol.

                                2. 2

                                  The latter. I’m sure he used to describe it as pronounced the Japanese way, but perhaps even fewer people understand that :-)

                                  1. 1

                                    I also thought was pronounced like in chicken karaage. However, I now suspect I pronounce that wrong also since I say “ah-hey” rather than “ah-geh”

                                    1. 3

                                      Heh yeah, for the record you’re pronouncing it wrong - a mora consisting of a g followed by any vowel is always a hard G sound in Japanese.

                                      So it’s kah-rah-ah-geh (more or less, in a standardish American accent, although with no aspiration because those Hs are just there to steer you towards the right vowel sound, and with the vowel sounds held for a somewhat shorter period of time than you might default to)

                                1. 4

                                  I had no idea about Lunar but it sounds like just what I need! Thanks for this really great write-up on an interesting problem and for Lunar :-)

                                  1. 2

                                    Glad you enjoyed it! If you experience any issues with Lunar, feel free to reach me and Lunar’s community on our Discord channel listed on the lunar.fyi website.

                                  1. 1

                                    Just say no to cleaning up git history people! You don’t look at it enough to pay off.

                                    Unfortunately, I don’t have data to back up this claim. :/

                                    1. 13

                                      I don’t have the data either, but I do have the experience. Bad commit messages and dirty history are the bane of my life. The advice in this post is excellent.

                                      1. 5

                                        I look at it easily more than 10 times a day. I do think you are on the right track as to why a large population of developers don’t take the time to write useful commit messages. They treat think of it as a write only medium. If I used the git command line or the GitHub web UI to navigate history I wouldn’t check the VC history so often.

                                        1. 4

                                          If people looked more often, perhaps they would care more about their commits.

                                          I have a gutter with commit messages for each line/chunk in my editor for much of the day since it gives me some context about why a line/function looks as it does.

                                          1. 1

                                            Right, I am totally for writing good commit messages! They should contain a description of the changes and a link to the ticket. That way you get requirement + architecture idea. But I never look at the graph structure.

                                          2. 3

                                            I use our git history constantly (we have a clean, well-organized one). I work at a Very Large Enterprise, too. Could your experience be related to not working within a space where the history is clean enough to be reliably usable, rather than it being worthless?

                                          1. 2

                                            There’s one small omission in this, particularly relating to footnote 1, that you don’t have to use pq.Array. As long as you don’t mind using a non-standard type, lib/pq has types like [StringArray[(https://pkg.go.dev/github.com/lib/pq#StringArray) which handles reading a postgres text[] column into a (wrapped) []string - so you wouldn’t necessarily need to use pq.Array when using pgx.

                                            Similarly, there’s a pgtype package which can be used with either pgx or pq in database/sql mode to provide similar types. pgtype has many more types available.

                                            1. 3

                                              Do people use fzf at the CLI?

                                              Not by typing fzf but I started using its zsh bindings so it’s acting as my ctrl-R (history search) and I’ve been getting into the habit of using its alt-C instead of typing cd. The ctrl-T behaviour can be useful, but zsh’s menu completion is usually good enough that I don’t need it. Overall I love the fzf shell bindings.

                                              Other fun tools I use are fd, the find replacement, and k9s for interacting with Kubernetes clusters.

                                              1. 2

                                                Do people use fzf at the CLI?

                                                Not by typing fzf

                                                I actually do that with fzy, because I am lazy to study the fzf bindings for Bash (and find out whether they work for Korn shell, then create mine), but I am also tired of the imprecise algorithm of fzf, and rather use that of fzy.

                                                But, obviously, I am the exception, not the rule.

                                              1. 2

                                                I was cheering until unsafe was used. Come on, don’t play stupid games to avoid one allocation.

                                                1. 1

                                                  Yeah, this was the one part of the post I didn’t agree with.

                                                  I was also curious there was no mention of strings.Builder.

                                                1. 1

                                                  I used mutt exclusively for more than 15 years. Last year I migrated my mail from self-hosted over to Fastmail and I found their web interface so good it’s all I use now.

                                                  (That, and I somewhat rely on the “Report Spam” and “Report Phishing” buttons in the UI, which, although can be sort of replicated in an IMAP client with filter rules, it’s less convenient.)

                                                  1. 2

                                                    Same, 90%[1] of the time: you can use a keyboard combination to hard wrap plain text, I pipe raw messages into hg or git to get patches.

                                                    Decent standard configuration and also good settings if you go deeper. Also, it’s ridiculously fast, I’ll never use imap with a desktop client and I’m the kind of person that doesn’t like to use web apps.

                                                    [1]: The last 10% are occasional email that I send from a terminal when it’s easier to do so. I never read from there.

                                                  1. 17

                                                    One note with the “Passing request to Consul services” section: when you use a DNS name in proxy_pass, that name is only ever resolved at startup. If the IP address the domain name points to could ever change you should define an upstream. Nginx only does normal DNS TTL / refresh in upstreams.

                                                    1. 6

                                                      This is rarely mentioned and you learn it the hard way. @alexdzyoba I think it’s a good thing to add to your great article :)

                                                      1. 4

                                                        I have previously documented a workaround for this (although, really the open source version should just support this): https://tenzer.dk/nginx-with-dynamic-upstreams/.

                                                        1. 1

                                                          Ah yes, I had seen the variable hack once before! Both the variable hack and upstreams need the resolver set, so both will do the trick.

                                                        2. 2

                                                          Thanks, everyone! I’ve updated my post. Put the link to this thread and the post by Tenzer.

                                                          1. 2

                                                            Thanks for pointing out! But what about “valid” option in resolver directive? It should control the TTL for DNS cache.

                                                            1. 2

                                                              It does, but proxy_pass doesn’t use DNS TTL, it only ever resolves once, unless you use a variable or an upstream. The Nginx docs aren’t very clear, but the resolver setting says it’s used for upstreams.

                                                          1. 8

                                                            One fun incompatibility I ran in to a while ago is that GNU libc will try the first namserver first, and then move on to the second if the first doesn’t work. Musl libc however will just always pick a random nameserver from the list.

                                                            So if you do something like:

                                                            nameserver 127.0.0.53    # Nameserver with some special resolving rules.
                                                            nameserver 8.8.8.8       # Fallback in case the above doesn't work.
                                                            

                                                            You’re going to run in to trouble on musl-based systems.

                                                            1. 5

                                                              One fun incompatibility I ran in to a while ago is that GNU libc will try the first namserver first, and then move on to the second if the first doesn’t work. Musl libc however will just always pick a random nameserver from the list.

                                                              I was curious what musl’s documented behavior is, so I went searching, evidently it issues queries to all your DNS servers in parallel and then uses the first response it gets back in an effort to reduce latency for lookups.

                                                              1. 1

                                                                Ah yes, you’re right; I remember it now. I actually ran in to this about 5 years ago and misremembered the details. I ran in to this when I was working on my DNS-based proxy/filter program, which worked great on my OpenBSD laptop that I initially developed it on while I was away for a month, but had weird intermittent problems on my Void Linux musl desktop system after I got home. It took me quite a while to track it down to this 😅

                                                                1. 1

                                                                  I can imagine, it definitely wouldn’t be the behavior I would expect out of the box if you asked me to predict how musl’s lookups would work.

                                                              2. 2

                                                                Which distributions use musl? The behavior you describe for musl is what I remember in whatever we were using in production circa 2003, either suse or redhat.

                                                                1. 5

                                                                  Alpine Linux for one. Alpine is commonly used with Docker setups too (because the images are smaller it means you can pack more images per gigabyte of production storage); not to mention Alpine recently added Tailscale as a package so we at Tailscale really have to make it Just Work™️.

                                                                2. 1

                                                                  glibc (and most other libcs I’ve encountered) also lets you do this with options rotate (among other things). Granted that’s not default behaviour in most places, but I’ve used that in the past where I wanted some internal systems to do more of a round-robin between internal nameservers.

                                                                1. 3

                                                                  In-kernel TLS finally lands! This is very cool. I’m curious to see how straightforward it’ll be to use it.

                                                                  1. 3

                                                                    It’s disappointing that it supports TLS 1.0 & 1.1. Nobody should be using those. https://datatracker.ietf.org/doc/rfc8996/

                                                                    1. 2

                                                                      My offhand guess is that netflix falls back to old tls versions due to old TV software (or similar devices) out there that end users can’t update.

                                                                      1. 1

                                                                        I hope old protocols are at least disabled by default and surrounded by scary warnings.

                                                                  1. 2

                                                                    I’ve always assumed Stripe’s docs, like most other API references, are generated by using Swagger/OpenAPI or similar. Docuowl looks nice but maintaining docs separately is often problematic. Using Swagger or ReDoc to generate docs from a spec is much more effective than manually maintaining them.

                                                                    That said, this looks nicely styled and I can see it being useful for smaller projects.

                                                                    1. 1

                                                                      Hi there! Thank you for the feedback! Using Swagger/OpenAPI is indeed better for bigger projects, and I completely agree with you that it may become problematic. In our case, we will for sure have extra work to maintain everything updated, but I think it is worth it, since using Docuowl over Swagger/OpenAPI can lead to a better (visual) result. Maybe in the future we can try to find a way to transform ReDoc into a Docuowl site! :D

                                                                      1. 2

                                                                        Docuowl definitely looks nicer than the default Swagger UI :-) I chose ReDoc for my last project because it had a nicer UI, although customising it was a bit tricky.

                                                                        I’ll definitely consider this for some personal projects where Swagger is too much :-)

                                                                    1. 13

                                                                      Why does Fn belong on the left of Control?

                                                                      I feel the opposite. Doubly so if you’re running not-macOS. I use control much more frequently than Fn and it’s much easier to hit if it’s all the way at the end.

                                                                      1. 2

                                                                        Control being left of Fn means you need less reach to press it along with most other keys, which is particularly important a) for people with small hands (like me) and b) on a laptop where there isn’t always a control key on the right.

                                                                        I don’t mind needing to use two hands to use Fn + another key, but that would be annoying with control.

                                                                        1. 2

                                                                          Honestly, Control belongs immediately outboard of Space: it’s the most commonly-used modifier, so it should be typed with the strongest fingers. Then Alt, then Super and finally Hyper, with Function past that if present.

                                                                          1. 5

                                                                            On a Mac Command is used more frequently, and that is right next to space, but I agree on non-Macs it’d make sense for Control to be there.

                                                                        2. 1

                                                                          I prefer it on the right. It may be to do with hand size/shape or just bring used to it. What Lenovo gets right is making it configurable, so the placement is not really an issue.

                                                                        1. 1

                                                                          This is a pretty great set of examples.

                                                                          One thing: the article says

                                                                          First, there is no convenient CLI tool like gh to communicate with the JIRA API

                                                                          True, there isn’t an official one, but go-jira exists and has been very useful. You can create custom templates and commands for its output so you could easily set up a jira command with output suitable for piping to fzf.

                                                                            1. 2

                                                                              I also use x, it just feels right

                                                                            1. 6

                                                                              ssh(1): Prefix keyboard interactive prompts with “(user@host)” to make it easier to determine which connection they are associated with in cases like scp -3, ProxyJump, etc. bz#3224

                                                                              Ah, now this is a useful fix!