1. 2

    This might not be a popular opinion but as someone who spends most of their day on the phone, I strongly dislike mechanical keyboards and the culture that has recently sprung up around them. I’m a consultant and my normal interactions with customers is over the phone. I can usually tell within 24 hours on a project who I have to mute by default on any new project. Especially with remote meeting software that emphasizes “call using my computer”. The sound of someone taking notes with a loud mechanical keyboard has disrupted my meetings so often that I can’t even count the occurrences. For clients I know very well, I’ll usually start the call saying “Hey [name], if you have something to say, make sure you unmute yourself” because I start the call with that person muted.

    I know there are mechanical keyboards that are quieter, but there are also a lot that are so loud no one can hear the conversation over the sound of the person taking notes on the call. Please be aware of the sound of your keyboard if you’re on a conference call.

    1. 13

      I find it incredibly odd that “push-to-talk” isn’t the cultural default almost anywhere. It’s so much nicer for every participant at so little cost to the individual.

      1. 6

        Yeah this is more of a broken software than anything else. I wish more systems were like mumble. Super low latency, crystal clear, push to talk. Instead we’re trying to cram 30 pointless video streams onto everyone’s screen.

        1. 5

          Your comment could just as well have been written by myself. Not only are these 30 pointless video streams crammed onto everyone’s screen – they’re also choking everyone’s network connection, causing further latencies in the audio, meaning conversations require explicit handoff to other people.

          I really wish I could convince my company to switch to something like Mumble. The low latency, crystal clear audio would make conversations flow so much more naturally and feel less forced. But no. We have to look at each others badly lit outlines of faces. That’s worth so much more than fluent communication.

          1. 4

            It’s madness. In the pandemic it has shown itself to be a deeply irritating technology. In 99% of cases the sole purpose of the video feed is to see what people’s houses look like.

            Mumble cracked the ‘how to we do natural conversation’ issue 15 years ago. Everything I have used since then feels like a step backwards.

          2. 3

            Mumble is especially good with RNNoise, that seems to “learn” filtering out sounds such as keyboard or clicking noises, improving over time. I have a not-so-quiet keyboard and a friend of mine has a more-loud-than-not mechanical keyboard and unless you’re typing and speaking, nobody notices either of us.

            Sadly it has to be enabled, as it’s not turned on by default.

            1. 1

              Wow, that was really cool!

        2. 9

          Back when folks were in the office, I didn’t mind using my mechanical keyboard around others. If they get to talk loudly on the phone, wandering around with wireless headsets, often times about topics that aren’t even vaguely work-related, then they get to listen to my clacking. Seems fair.

          1. 7

            If everyone thought like that then there would be no peace in the world. Well, maybe there isn’t… but still, I don’t think others acting poorly means you should too.

          2. 3

            I’m a fan of mechanical keyboards, my first one was a Sun UNIX style buckling spring model with the command and caps lock key functions swapped (so that CTRL is on the home row where it should be for programmers). Still, I think you’re undoubtedly right that they are best for people who work in private offices and non-collaborative work environments instead of open plan offices and people who do a lot of conferencing.

            1. 9

              Not to be facetious. But that sounds like a problem for management.

              If you want to stick me in an open office and then complain that my work is too loud; that’s on you. (Yeah, I know keyboards are a preference but so is working in an open landscape)

              1. 5

                (Note: writing this comment turned out more aggressive than it ought to be. Rest assured that I have no quarrel with you, I just hate open plans, to the point I’d consider turning down offers over them.)

                private offices and non-collaborative work environments instead of open plan offices

                I’m not sure I agree with the implication that anything “not open plan” is not collaborative. Like many people here I suddenly started to work remotely this last few months, and the amount of practical collaboration within my own team doesn’t seem to have significantly decreased, despite the higher friction of instant messaging with microphones compared to our shared office. I’ve also worked in an actual open plan office, with over 50 people on the same completely floor. We collaborated all right, but boy, the noise.

                Let’s not kid ourselves, what is so often sold as a way to increase collaboration is mostly cost cutting, surveillance, and showing off. Discovered that last one pretty recently: open plan offices are great at showing the sheer mass of people buzzing & working together to executives and clients. Lots of people at their desk doing whatever hermetic magic technical people do, a couple group here and there on a Scrum meeting, or just discussing obscure schematics on a whiteboard, honestly it’s beautiful.

                Me, I yearn for a cubicle. I don’t even require a full wall, I just want less noise, less visual distraction, and a wall behind my back. Seriously, leaving your back open to a room full of people you barely know, some of which you may even dislike a little? Nobody wants that. Why do you think the higher ups end up near the corners of the open floor? Why do you think the last hire, juniors, or interns, end up with the one office with their back to the door?

                Cubicles however are horrible to show. Everyone looks isolated. You don’t hear as much buzzing activity, the floor is now closer to an oppressive maze than a green field, you don’t see as many faces…

                I tried to put up walls on my desk. 90cm tall, some foam to dampen the sound, all around my desk (80cm deep, 180cm wide). Very effective at attenuating the sound, much less distractions. Despite prior authorization to try it out by ones of the higher ups, the first higher up to actually see it instantly vetoed it. And here’s the thing: one thing they worried about was that everyone would do something similar, and the whole office would start looking like a slum. So they knew on some level that many people might want this. But they were reluctant to give it to them because it wouldn’t look nearly as good.

                Lesson learned: outwards appearances are more important than internal well being.

              1. 2

                It seems to imply that it is 66.94%, with mainly Safari doing its own thing. Am I reading it wrong?

                1. 1

                  It’s 66.94% of users, but 35% of the browsers.

                  1. 2

                    But is that really a useful distinction when many of those browsers are at less than 1% market share (and possibly no longer receive updates)? This is not a feature critical to the functioning of a website, so adopting it won’t break anything other than the presence of an icon for a minority of users. It’s still a choice to be made of course, I just think it’s a perfectly valid choice either way. It’s very different to adopting some new JavaScript syntax with the potential to completely break your site for many users, or CSS changes without a fallback that break the layout when not supported.

              1. 36

                There are good reasons for the complexity of text rendering. Developers don’t just add complexity for no reason. In general, if one says “why not just”, one is probably oversimplifying the problem.

                1. 7

                  This article is arguing against “Why not just assume everyone has a high-DPI display since then we can ignore existing solutions and focus on the one that requires everyone to replace the equipment they already have which works fine.”

                  1. 14

                    It doesn’t require you to replace anything. I use vector fonts on a non-hidpi display just fine and, I figure I’m not the only one who has been doing this for at least a couple decades? I am however looking forward to upgrading to a hidpi display, not because I need to but because it’s simply better.

                  2. 3

                    Well, cause the problem used to be simpler. Using ‘why not just’ also calls into questioning the presuppositions and the nature of issues. The font rendering for typesetting is much less difficult as it is more predictable. Yes, fonts are weird and there are edge cases, but code is still mostly written in pure ASCII set.

                    Also, it is a rant and not a technical text for a reason.

                    1. 9

                      The problem was never simpler, the solutions were merely incomplete, incorrect, and exclusionary.

                      1. 1

                        I think you’re being uncharitable. But, @lich also has a lot of unstated assumptions here. I think they are talking about font usage in a text-only programming environment in a terminal emulator, on a low-DPI display. So yeah, that’s a pretty narrow use case compared to, say, a web browser for use with every possible combination of typographic style and language in the Unicode universe, which is what @mwcampbell ‘s link is about. But it’s a valid enough use case, and relevant in this forum, where people are interested in OS development and simpler solutions in general. The modern typographic rendering libraries are not being threatened here, and of course they’re not going away. But they are so massive and complex as to be exclusionary in their own way.

                    2. 2

                      Developers don’t just add complexity for no reason.

                      HAHAHAHA AHAHAHAHA AAAAHAHAHAHA! ahhhhgh…

                    1. 6

                      I found WebP to be significantly better for my usage, but I had to experiment a bit, and initially found it worse due to the smoothing out of details. What I found to work well, were these settings (with GraphicksMagick, and I was testing with 1024x1024 images at the time; I also saw a comment for this article on HN which claimed libvips has better results for WebP too):

                      # JPEG for comparison
                      gm convert <input_file> -quality 80 <output_file>.jpg
                      # WebP
                      gm convert <input_file> \
                        -define webp:emulate-jpeg-size=true \
                        -define webp:filter=sharpness=7 \
                        -quality 80 \
                        <output_file>.webp
                      

                      Even with default settings WebP looked cleaner to my eyes (I strongly dislike JPEG compression artifacts; and I personally didn’t like the results from MozJPEG which also seems to destroy fine detail), but the loss of detail didn’t seem worth it. The webp:filter=sharpness=7 made a noticeable difference in retaining detail in the compressed image, and I personally think it looks a lot better, certainly nothing like the examples in this article. Apparently WebP support is finally landing in Safari 14, so the format finally will be supported by all major browsers (that doesn’t mean I won’t be providing JPEG fallbacks, but it’s nice to know).

                      I also found that WebP’s lossless mode compressed an image of one of my app’s logos to 10KB compared to 38KB with PNG.

                      In the long term, I do believe JPEG XL will be the prevailing format. It takes on features from experimental formats like FLIF/FUIF and PIK, and is royalty free. For now though, I disagree with this article’s conclusion, I think WebP has some clear benefits.

                      1. 1

                        The use of web fonts makes the website look the same for everyone, gives a lot more options to the webmaster, but nothing is free, and the cost is website performance.

                        The cost is trampling on your users font choices.

                        1. 3

                          Of the few users for whom this might be an issue, I’d imagine most are quite capable of overriding the stylesheet.

                          1. 1

                            Yeah, and that means for people who do adjust their font sizes for accessibility reasons, this is untested. Leaving this stuff untested means that text overflows and wraps strangely, and leads to a suboptimal user experience.

                            1. 3

                              I think that’s a bit of a stretch. Unless you’re doing something very strange, text will usually flow just fine. How are you imagining that designers would test every possible font that a system might select anyway? You should always set a fallback like sans-serif, and depending on the system that could be literally anything (not to mention that text rendering can be inconsistent between browsers and platforms even with the same font). Again, unless you’re doing something strange or complex, I don’t think that’s going to lead to any major problems anyway. Whether you’re using web fonts or not, you should design for some level of flexibility.

                              1. 3

                                Most of the problems come from something having a width: 200px or some such, which just fits with the assumed font(s) but won’t fit if you substitute it with a font that’s slightly wider. It’s a problem in some dialogs on Stack Overflow for example (or was, anyway, I think it’s fixed now).

                                This is hardly a web font problem; I use DejaVu Sans instead of Arial for example, which is slightly wider and very occasionally things break on some websites. It’s just the price you pay for frobbing with this kind of stuff.

                                1. 1

                                  There’s a few spots in Slack (yes, I use Slack in Firefox) has an issue where the timestamp on individual messages line wraps. Luckily, nothing is cut off, but it does result in single-line messages that take up two lines because the 10:00 PM takes up two lines of its tiny cell.

                              2. 2

                                Unless you’ve carefully specified aliases for all “web-safe” fonts it’s not like you’re in “full control” anyway. I would wager that a vanishing small amount of people do this.

                                Either way, for most cases it’s not too hard to make sure that $any font works. I use web fonts on my site and my product and you can swap them out with $preferred_font and everything should still work fine.

                                1. 1

                                  Firefox has a toggle to disable web fonts entirely. Chrome has an extension.

                            2. 2

                              I’d be willing to bet that the vast majority of users have never set a font preference in their browser. I certainly never have, and I don’t know anyone who has.

                              1. 1

                                I had to bump up the font size in order to read some pages. My eyes are better than some, but not what they used to be when I was 20.

                                I really wish that browsers picked up the system font settings, though.

                            1. 2

                              Fareed, out of curiosity, you mention The Little Schemer but link to The Little Lisper. I’m curious which one it was since you mention such a positive boost in satisfaction and insight. Thanks!

                              1. 5

                                They are the ‘same’ book. The Little Lisper is the first edition and it has code to follow along in Common Lisp as well. The following editions are under the name little Schemer and are scheme only. Although they are more of a pen and paper book tbh. So get the little schemer. fwiw I enjoyed the reasoned schemer a lot as well.

                                1. 1

                                  You’re right, but The Little Lisper (3rd edition) has some “homework assignments” that are missing from the Little Schemer. This makes the former a slightly better book, IMO, even though the Little Schemer’s paper and print quality are better. For the life of me I can’t understand why they decided to drop the exercises from the newer book.

                                  You mention the Reasoned Schemer which is great in its own way, but a very different book because it’s not about “standard” programming. On the other hand, the Seasoned Schemer basically picks up where the Little Schemer left off, and it goes into continuations if I’m not mistaken. Highly recommended if you enjoyed the first book!

                                2. 2

                                  I’ve only read The Little Schemer, but it was a real joy to read. It hooked me in right from the start and was very easy to follow. You don’t even really need a computer to go through it, I actually wrote all my answers on paper which was fun (for me at least). As a bonus, it got me interested in Scheme!

                                1. 2

                                  In all these cases there was ways to keep a flat UI, while making the interactive elements stand out more.

                                  For example, on the first example they went from what clearly looks like a button to a white box on a white background with a purple border. It should have been possible to make that button flat, but give it a subtle shadow or something to make it stand out more.

                                  I think it’s an interesting study but they aren’t really saying what should be done to improve flat UIs. Going back to skeuomorphism is probably not an option as it would make the UI look dated, but there has to be a middle ground that can work.

                                  1. 3

                                    Going back to skeuomorphism is probably not an option as it would make the UI look dated, but there has to be a middle ground that can work.

                                    I think the concern (and I am not saying this was your argument, the sentence just led me in this direction) with looking dated is where the problem pretty much lies. When you look at industrial or medical products, there is an overwhelming concern with usability, not novelty. Yet there is still variety. I think that UI designers can achieve variety and aesthetic pleasure without abandoning a good idea simply because it looks old. This will probably happen naturally over the coming decades as these technologies become normal, which is why we no longer see insane steering wheels except in concept cars.

                                    1. 3

                                      It should have been possible to make that button flat, but give it a subtle shadow or something to make it stand out more.

                                      I don’t want to “no true Scotchman” things here, but at that point it’s no longer a “flat UI”, right? I think the objection is mostly against UI elements which are truly flat, not those that are “less 3D”. I often use shadows myself, which gives kind of a “pop-out 3D” similar to the 3D UI of yesteryear, except more, well, fashionable I guess?

                                      If you look at the GNOME/Librem screenshots in the another reply here, then the “3D effect” is done by using a gradient background which is slightly different from the surrounding colour, but with a solid border colour which doesn’t pop out (which is what the old UIs used) – stuff like the Bootstrap CSS theme also does it like that by default (or at least, used to do – haven’t used it in a few years).

                                      Even the buttons on Lobsters which have a solid border colour and solid background colour kind of “pop out”. It takes very little to fool out brain in to thinking something “pops out”, and the problem with flat UI is that it makes no effort at all to do that. Anything that does make that effort is – as far as I’m concerned – not really a flat UI.

                                      1. 2

                                        Exactly. There’s a lot you can achieve with subtle suggestions of depth without going in completely the opposite direction and distracting from the content (or just blending in; there’s a reason road signs aren’t detailed and realistic). I’m actually really glad for the cleaner interfaces we have today in general but it still needs to be done thoughtfully. There are definitely too many completely flat designs that just jump on the trend without putting any thought into why or how. That’s lazy. But a UI can certainly take elements of flat design while remaining highly usable.

                                      2. 2

                                        There most certainly exists a middle ground, signifiers are on a spectrum and the example of the underlined vs. just contrasting-colored links reveals you can get away with weakening them (although underlined links is not something I’d personally recommend) and users will still figure them out. This article goes into more details on how to improve flat UIs.

                                      1. 2

                                        My eyes thank you! I think you’ve done a great job with it and I wish that more writers and site owners would take the time to learn about something which I consider pretty fundamental. I also agree that underlining links is generally a good idea, I just wouldn’t leave them at the default as I do think in most browsers it tends to be a bit tight. But underlines for links is pretty universally understood so I think it’s worth it. The way you’ve done it seems fine to me.

                                        Also, are you aware of units like ch? They can be useful for setting the line width and other similar things (grid columns, etc.).

                                        1. 3

                                          Thanks, I appreciate the kind words. 😊

                                          I am aware of ch, but I’ve never really used it. I always use either px or vw for setting the width. I’m an easily confused, simple man haha.

                                        1. 2

                                          NodeJS: let the frontend devs also make the backend

                                          Elixir+Phoenix Live View: let the backend devs also make the frontend

                                          One of these is good, the other is terrible.

                                          1. 4

                                            I’m sorry but I find that a little elitist. There’s always someone to blame, whether it’s designers, web developers, frontend developers, for some reason we can’t stop pointing the finger at certain groups and saying ‘they’ve ruined everything!’. Don’t get me wrong, things are a mess, but this kind of tribalism isn’t the answer.

                                            1. 2

                                              This isn’t a comment on the skills of the developers; it’s a critique on the quality and appropriateness of the tooling.

                                            2. 2

                                              Elixir+Phoenix Live View: let the backend devs also make the frontend

                                              Pretty much how I found myself loving frontend development. Because of Elm, and now Haskell - which is used in frontend and backend.

                                            1. 4

                                              I completely agree with this. What’s the point of making a custom website for an ecommerce of basic company website anymore for most people? There’s so many nice templates on services like Shopify and Squarespace. Better to just recommend those services for simple sites.

                                              1. 4

                                                Absolutely, in most cases. Personally I’m building my own ecommerce site because it’s fun and I can do it exactly my way without restrictions. It’s not economical sure, but I feel like I’m getting a lot out of it. Would I do that for a client? I’d have to be insane (I work with Shopify daily, it’s great).

                                                1. 2

                                                  Exactly my thoughts! I’d code it myself if it were for my own stores, but seems like a waste of effort for (most) customers.

                                              1. 10

                                                I’m with Linus. 80-character limits one the 21st century are an anachronism. The legibility studies being mentioned here are about people reading natural-language prose, in long paragraphs that wrap. The shorter line length helps the eye Jump accurately from the right margin to the start of the next line.

                                                Code isn’t read that way, in any language I know of. It’s short individual lines, with no consistent length, and in most languages they’re indented to varying levels. We scan it in all different ways depending on what we’re doing — I bet someone’s done eye tracking studies, but thinking about myself, a lot of the time I’m skimming the start of each line looking at the statement keywords (if, for, let, return…) and indentation. Or I’m looking at all the instances of one variable name that my editor’s highlighted for me.

                                                Consider also that the real meat of the code, the function bodies, is usually indented, which eats away at the line limit. In C++ I’m usually inside a method in a class declaration in a namespace … that’s 12 spaces subtracted from the line width. I go into an if or `for, another 4 gone. That’s 40% of an 80-char line lost before I start typing anything! (I know some people use 2-space indents to work around this. I find that’s too narrow for me to ‘read’ indentation accurately.)

                                                1. 3

                                                  And I get the feeling also that many languages that use 2 character indent by convention, do so in order to keep lines within these limits. I also find 2 spaces borderline too narrow, despite using it in Ruby code daily (where it is so standard that you’re really better off just going with convention for the sake of the community).

                                                1. 4

                                                  Its funny that this email is formatted to 71 columns.

                                                  1. 2

                                                    What’s funny about it? It’s written text, not code, they are not comparable beyond a superficial level.

                                                  1. 21

                                                    Yeah, 72 is much more reasonable. We need hard limits, or at least ‘annnoying’ conventions to keep the horrors at bay. The human eye scans lines best at around 72 characters wide, and we should put human readability of our code before other concerns. I have worked on projects that had huge long lines and there is no excuse. If a language or tool or whatever can’t deal with human limits, find or make another tool. Linus’ current workstation should not be the standard.

                                                    That being said, I think Racket has made a reasonable compromise:

                                                    A line in a Racket file is at most 102 characters wide.

                                                    If you prefer a narrower width than 102, and if you stick to this width “religiously,” add a note to the top of the file—right below the purpose statement—that nobody should violate your file-local rule.

                                                    This number is a compromise. People used to recommend a line width of 80 or 72 column. The number is a historical artifact. It is also a good number for several different reasons: printing code in text mode, displaying code at reasonable font sizes, comparing several different pieces of code on a monitor, and possibly more. So age doesn’t make it incorrect. We regularly read code on monitors that accommodate close to 250 columns, and on occasion, our monitors are even wider. It is time to allow for somewhat more width in exchange for meaningful identifiers.

                                                    https://docs.racket-lang.org/style/Textual_Matters.html

                                                    1. 25

                                                      The human eye scans lines best at around 72 characters wide

                                                      I would like to have 72 chars wide line but with ignored indentation. It would make nested block readable on its own.

                                                      Example with 40 chars width ignoring indentation white space

                                                      Lorem ipsum dolor sit amet, consectetur
                                                      adipiscing elit. Donec sit amet augue
                                                      felis. Suspendisse a ipsum et sem auctor
                                                      porttitor in ac lacus. 
                                                      
                                                          Curabitur condimentum augue diam, ut
                                                          molestie nibh faucibus nec. Aliquam
                                                          lacinia volutpat tellus, non
                                                          sollicitudin nulla luctus sit amet.
                                                      
                                                              Aenean consequat ipsum sem, ac rutrum
                                                              leo dictum at. Suspendisse purus dolor,
                                                              condimentum in ultrices vel, egestas vel
                                                              ipsum.
                                                      

                                                      Versus 40 chars width including indentation

                                                      Lorem ipsum dolor sit amet, consectetur
                                                      adipiscing elit. Donec sit amet augue
                                                      felis. Suspendisse a ipsum et sem auctor
                                                      porttitor in ac lacus. 
                                                      
                                                          Curabitur condimentum augue diam, ut
                                                          molestie nibh faucibus nec. Aliquam
                                                          lacinia volutpat tellus, non
                                                          sollicitudin nulla luctus sit amet.
                                                      
                                                              Aenean consequat ipsum sem, ac
                                                              rutrum leo dictum at.
                                                              Suspendisse purus dolor,
                                                              condimentum in ultrices vel,
                                                              egestas vel ipsum.
                                                      
                                                      1. 18

                                                        The human eye scans lines best at around 72 characters wide

                                                        With monospace fonts? Or proportional ones? With large text or small?

                                                        With English prose, poetry, or with C code? With hyphenation? Indentation?

                                                        I’ve found that recommendation is pretty good for English text with a middle size proportional font. I do not find it works as well for code.

                                                        1. 5

                                                          100% agreed. As I argued in the comments above, people don’t read code the same way that they read prose, and so I would not try to generalize a heuristic meant for prose to code.

                                                          1. 3

                                                            I agree. Reading written text involves repeatedly shifting your focus to the line below. A consistent and short line length in that case is very important. Code is not the same. It’s far more common reading code to study a single line or small block, and in that case, I find that arbitrarily wrapping a line to stay within 80 characters usually breaks consistency and harms readability. I used to subscribe to the 80 character limit until I realised this difference. We don’t read code like we read written text.

                                                            Terminal/editor windows side by side is a fine point, but in general the vast majority of lines of code are short anyway, often well under 80 characters. If a few lines happen to wrap on your display, I hardly think that’s going to completely kill readability, and it’s certainly a trade-off I’m willing to make. If many lines are wrapping then yes, you probably have a problem with your code formatting (or your code in general). It’s the hard limit that I take issue with. Back when I wrote my own code like this, all too often I would find myself renaming identifiers (usually for the worse) among other arbitrary and unhelpful things, just to fit some code within a line without wrapping. I wouldn’t be surprised if more often than not this is the outcome for many others who attempt this, and it’s almost certainly a net negative for readability. Dropping the idea entirely has been a huge relief. One less thing to think about. Most of my code still consists of short lines, as it always did, and as most code naturally does. But when I need to, and it makes sense to write a longer line, I don’t spend a second agonising over whether it fits within some special number of characters, and instead focus entirely on whether it in itself is clear and understandable.

                                                          2. 10

                                                            I want to reinforce your comment that

                                                            The human eye scans lines best at around 72 characters wide, and we should put human readability of our code before other concerns.

                                                            Recently, I have been trying to optimise my on-screen experience and I found a series of peer-reviewed articles with recommendations that improved my experience.

                                                            In one of those, it is claimed, indeed, that more than 72 and less than 80 characters (precisely, 77) is the optimal number for clear readability.

                                                            The study is for dyslexic people, and I was never diagnosed as so. But it works for me, and I tend to believe it works for most people as well.

                                                            1. 1

                                                              Yeah what I’ve read confirms this, I think the width of the text column can vary based on the character width. For example in CSS I typically set something like:

                                                               p { max-width: 32em; }
                                                              
                                                              1. 4

                                                                You can also use 72ch to set it based on width of the 0 character (which will approximate it)

                                                          1. 2

                                                            Is the “explicit” vs “implicit” distinction a common one for search engines? I’m not familiar with it. I’m guessing that explicit means “literal” match, implicit means some kind of fuzzy/“guess what I mean” matching?

                                                            1. 5

                                                              Most search engines assume you’re looking for something common, and thus:

                                                              • “Fix” queries with typo & misspelling correction based on string edit distance and term frequency
                                                              • “Expand” queries by projecting your terms into some kind of topic space (embedding) and giving you results that score highly on nearby terms

                                                              These are great for natural language queries for popular resources, and really terrible for precise, niche queries.

                                                              1. 1

                                                                Hmm, I haven’t heard those terms being used either in this context. I definitely would find it a little more clear if it used the terminology you mentioned instead.

                                                                1. 4

                                                                  Thanks, I’ve updated the terminology on the frontpage, and it should go live in a few minutes.

                                                              1. 1

                                                                What else people use nowadays? Macaroon?

                                                                1. 4

                                                                  I don’t know much about it, and don’t have any experience with JWT either, but I’ve heard of this:

                                                                  https://github.com/paragonie/paseto

                                                                  1. 3

                                                                    I’ve used paseto in the recent past, and it was extremely easy to use & achieve good results with. Would strongly recommend on that alone. My crypto friends all seem to like it, too, so that seems like a winner.

                                                                1. 11

                                                                  Oh my god, someone put an actual URL with a version number right inside the import statement. I’ve been dreaming about that (albeit on the Python side of dynamics) for years!

                                                                  We kind of grown accustomed to the idea of what belongs to a language, and what’s tooling. But may be it’s time for our assumptions based on decade long habits formed in the times without even the idea of repositories and automated updates to finally become obsolete.

                                                                  1. 3

                                                                    Sounds like a pretty bad idea, to be honest. I can see why no one is doing this (outside of REPLs/scripts).

                                                                    1. 6

                                                                      It’s probably a good idea if you can expand on why it’s a bad idea for the benefit of discussion.

                                                                      1. 6

                                                                        Because doing that would mean spreading this information all across the codebase.

                                                                        And if one goes “oh this is obviously bad, I’ll centralize all the imports into one file and export them to the codebase from there” – congrats, now we have reinvented status quo, just poorly.

                                                                        There is a lot of value in having a dedicated dependency file that records the (orgName, libraryName, version)-triple for a project, such that changes are easy and guaranteed to be consistent.

                                                                        1. 4

                                                                          So I’m looking into this and I have to say I like the way it’s done. Importing by URL is the same way the module system works in the browser. Rather than using URL imports in all your files, you can use an importmap (and I think this idea comes from browsers too) so I don’t think it’s likely people will be reexporting all their dependencies from one file. Deno has integrity checking and lock files, and you can also choose to vendor your dependencies if that works better for you. You can pin versions in the URL in most CDNs and also deno.land, e.g. https://deno.land/std@0.50.0/http/mod.ts. It has a URL rewriting service for third party modules which works the same as the previous standard library example. Obviously you would use an importmap here so you only specify the desired version in one place.

                                                                          1. 2

                                                                            So, we can expect that no one is going to use imports with versions manually because it’s not a good idea in favor of importmaps, as mentioned.

                                                                            1. 1

                                                                              I’m not quite sure what you mean? You use the versioned URL in the import map, and you use the lock file for integrity checking.

                                                                          2. 1

                                                                            I have an impression it’s an opt-in. Like, if your whole code base is one file, then you can do that instead of having a package.json.

                                                                            1. 1

                                                                              Agreed…

                                                                              One feature, I guess, is that you could easily depend on different versions of a package in different files of the same project. I think that Golang already does it to some extent, but I have not needed that, yet.

                                                                            2. 1

                                                                              one serious problem with it is that URLs are evanescent and can change both ownership and content trivially.

                                                                              1. 1

                                                                                That’s where integrity checking comes in. It doesn’t address availability (if that’s important then maybe you should vendor or backup a local copy anyway), but it does at least mean you’re safe from malicious changes at the source.

                                                                        1. 7

                                                                          So, uhh…. what now? Shut down the Internet until this is fixed? Disconnect your wifi router? Never log on to another web site again?

                                                                          1. 30

                                                                            It doesn’t matter at all unless you trust that certificate, or whoever published it. It’s just a self-signed certificate that is valid for any domain. If you don’t trust it, then you don’t trust it, and it will be invalid for any use where you come across it.

                                                                            1. 5

                                                                              Gotcha; I missed the critical detail that it’s self-signed. So to use this in an attack you’d have to trick someone into trusting the cert for some trivial site first.

                                                                              1. 3

                                                                                Exactly. And then they would have to serve some content with that cert that the target would access. There’s essentially no practical way this could be used in an attack except for a man-in-the-middle attack, but you would still need to get the target to trust the certificate first.

                                                                                1. 3

                                                                                  Trusting the cert is easy with technical people. I link you guys to my site, with a self signed cert like this. You accept it because you want to see my tech content.

                                                                                  This is a huge issue.

                                                                                  1. 4

                                                                                    How is this different from using any other self-signed certificate?

                                                                                    1. 4

                                                                                      Here’s what I think @indirection is getting at:

                                                                                      1. Your connection to the net is MITMed.
                                                                                      2. You visit sometechgeek.com, which is serving this wildcard certificate
                                                                                      3. You think “weird, crazy tech bloggers can never take proper care of their servers” and click through the SSL warning
                                                                                      4. Your browser trusts the wildcard cert. Next, you visit yourbank.com
                                                                                      5. Since the wildcard cert is trusted by your browser, the holder of the key for that cert can intercept your communication with yourbank.com

                                                                                      However, I would hope SSL overrides are hostnane-specific to prevent this type of attack…

                                                                                      1. 2

                                                                                        Yep that’s exactly it! Thank you.

                                                                                2. 2

                                                                                  I missed the critical detail that it’s self-signed

                                                                                  You didn’t quite miss it, it’s been misleadingly described by the submitter — they never explicitly mention that this is merely a self-signed certificate, neither in the title here, nor in the GitHub repository. To the contrary, “tested working in Chrome, Firefox” is a false statement, because this self-signed certificate won’t work in either (because, self-signed, duh).

                                                                                  1. 2

                                                                                    I never say that it’s signed by a CA either 😅 I wasn’t trying to mislead folks, but some seem to have interpreted “SSL certificate” as meaning “CA-issued SSL certificate”. It does work in Chrome and Firefox insofar as it is correctly matched against domain names and is valid for all of them.

                                                                              2. 11

                                                                                This isn’t signed by a trusted CA, so this specific cert can’t intercept all your traffic. However, all it takes is one bad CA to issue a cert like this and… yeah, shut down the Internet.

                                                                                1. 4

                                                                                  For any CA that has a death wish sure!

                                                                                  1. 8

                                                                                    Or any CA operating under a hostile government, or any CA that’s been hacked. See DigiNotar for just one example of a CA that has issued malicious wildcard certs.

                                                                                    1. 3

                                                                                      And as you can see it was removed from all browser’s trust stores and soon declared bankrupt (hence, death wish). And that wasn’t even deliberate. I can’t see a CA willfully destroying their own business. Yes, it’s a huge problem if this happens though, and isn’t announced to the public, as the case in the article.

                                                                                2. 2

                                                                                  Normally, certificates are doing three separate things here:

                                                                                  1. Ensuring nobody can read your communications.
                                                                                  2. Ensuring nobody can modify your communications.
                                                                                  3. Ensuring you’re communicating with the entity which validly owns the domain.

                                                                                  Most people who are against HTTPS ignore the second point by banging on about how nobody’s reading your webpages and nobody cares, when ISPs have, historically, been quite happy to inject ads into webpages, which HTTPS prevents. This strikes at the third point… except that it doesn’t. It’s self-signed, which defeats the whole mechanism by which you use a certificate to ensure you’re communicating with the entity you think you are. The weird wildcard stuff doesn’t make it any less secure on that front, since anyone can make their own self-signed certificate without wildcards and it would be just as insecure.

                                                                                  If you could get a CA to sign this, it would be dangerous indeed, but CAs have signed bad certificates before. Again, a certificate can be bad and can get signed by an incompetent or corrupt CA without any wildcards.

                                                                                  So this is a neat trick. I’m not sure it demonstrates any weakness which didn’t exist already.

                                                                                1. 7

                                                                                  Does anyone know of a good book that treats systemd somewhat comprehensively?

                                                                                  At this point, I’ve decided my feelings about systemd don’t matter; it’s clearly here to stay. System administration is not my day job, so I’ve been able to get by with only minimal and superficial knowledge of it. I’ll have to re-learn a lot of tasks that I knew how to do (or could at least figure out how to do) using the previous init systems.

                                                                                  Whenever I do, I find I’m always googling (or reading man pages) in frustration. It’s a very extensive system. Command names are long and non-obvious. That’s okay. After all, if I managed to learn git, I should be able to learn systemd.

                                                                                  But I’d like to do so at a leisurely pace, instead of scraping documentation together from (frequently severely outdated) blog posts and man pages.

                                                                                  So is there a good book that’s up-to-date? Or is Red Hat certification study material my best hope?

                                                                                  1. 15

                                                                                    At this point, I’ve decided my feelings about systemd don’t matter; it’s clearly here to stay.

                                                                                    That’s not totally clear to me. The two biggest bits of the Linux ecosystem are Android and cloud deployments. Android does not use systemd, it uses its own thing. Cloud deployments increasingly use things like Kubernetes to deploy containers. They may use systemd on the oustide (does containerd depend on systemd yet?) but typically the containers don’t use systemd on the inside. Over time, I expect the things on the outside to be simplified and systemd is an obvious contender to go because it doesn’t add much value in this space.

                                                                                    Systemd is really only dominant in individually managed servers and desktop deployments, neither of which are particularly large or growing parts of the overall ecosystem.

                                                                                    1. 5

                                                                                      Plus, nothing is permanent in the world of tech :). At this point, systemd is mature enough, and deployed widely enough, that I’m beginning to see a bunch of anti-patterns. In my experience, this is when people begin thinking of the next step. Anti-patterns aren’t just a symptom of incompetent users/developers, they’re a sign that a technology is reaching its limits.

                                                                                      A pattern that I’m seeing increasingly often in embedded systems is something that I’ll just go ahead and call the “big init service”, for lack of a better word. Basically, there’s a unit file that runs a shell script, which runs all the application-specific programs (random example from the latest gadget I’ve worked on: a zeromq-based daemon, a logging service – I don’t know why that’s a thing, I was paid just to make the previous daemon stop crashing –, a pseudo-watchdog timer, a license manager, some serial-listening daemon, and a Wayland session in kiosk mode). Basically anything that didn’t have a Yocto package in a base layer so that you could just list it as enabled/disabled at build time.

                                                                                      Being the helpful idiot that I am, I asked one my customers why those aren’t separate init services (I generally know better, but we had a history together and figured it wouldn’t hurt). They told me they knew it was possible but it was a lot of hassle, and they frequently had to launch some of these programs with various extra flags or environment variables for debugging or development purposes, or tweak various parameters when testing. Plus they were all designed to work together anyway. It was a lot more convenient to be able to start them all at once, stop them all at once, and tweak how that’s done by just changing a few lines in a shell script, than to mess with half a dozen unit files. I offered a few suggestions about how systemd can manage that. Turns out they’d tried each and every one of them, and never managed to get it to work reliably – and figured they’d rather write a clunky script than risk dealing with all sorts of flops in the field.

                                                                                      (Edit: to be clear, I think systemd can actually handle that case pretty well – but, indeed, I guess it is a lot of hassle, especially if you want to ensure it works the same way every time, which is very much important for unattended devices where interactive access isn’t always easy to get).

                                                                                      1. 1

                                                                                        What advantage does systemd bring in this use case in the first place? It sounds to me that they’re basically bypassing systemd? Or are there other parts that bring value?

                                                                                        1. 1

                                                                                          There aren’t, at least not for them, but at this point it’s so hard to yank it out of (some) systems that they’d rather bypass it.

                                                                                      2. 3

                                                                                        I tend to use containers over VMs specifically because I don’t have to configure a process manager or a log collector or an ssh daemon or host metrics or anything else. More importantly, developers don’t have to know how to do these things, so now they are empowered to own a greater share of the “ops”, and they aren’t bottlenecking on communication and coordination with an ops function.

                                                                                        1. 2

                                                                                          They may use systemd on the oustide (does containerd depend on systemd yet?) but typically the containers don’t use systemd on the inside.

                                                                                          You must work in a much more aesthetically pleasing corner of the software world than I do! At this point I’m rarely surprised to find systemd in an Enterprise Container™.

                                                                                          1. 2

                                                                                            That might be true, but while Gartner might have some say about what kind of work I’m likely to do in the future, it has very little influence on my personal choice of desktop, and the people who maintain open source desktop linux have mostly chosen systemd.

                                                                                            1. 1

                                                                                              That might be true, but while Gartner might have some say about what kind of work I’m likely to do in the future, it has very little influence on my personal choice of desktop, and the people who maintain open source desktop linux have mostly chosen systemd.

                                                                                              I don’t disagree, but I suspect that systemd will make part of the ecosystem increasingly disconnected from everything else. I wouldn’t be at all surprised if Android started to encroach on the desktop Linux market. Android works surprisingly well on devices with keyboards and mice instead of touchscreens and now supports split screen mode, which is what fans of tiling window managers have been telling us all for ages is better than multiple windows. If you use F-Droid, you can install a load of F/OSS apps, including a chroot environment for GNU/Linux things and even an X server so you can run graphical non-Android *NIX apps. There’s even a native OpenOffice port. At the same time, you can run things like MS Office or any of the rest of a huge pile of other supported proprietary software.

                                                                                              If you’re a hobbyist developer, writing an Android app rather than something for GNOME or KDE gives you a lot more potential users, a lot more potential collaborators, and a set of more marketable skills if you want to eventually move into commercial (open source or proprietary) development. How long do you think GNU/systemd/{GTK/Qt}/{X11/Wayland}/Linux is going to be more popular than Android for desktops?

                                                                                              1. 1

                                                                                                That’s an interesting question. Popularity - for me - is not a reason to jump ship to another system, at least not for personal use. You’re a FreeBSD developer; if you’d listened to the cool kids on slashdot back in the day you’d have abandoned that doomed ship a long time ago ;)

                                                                                                But you’ve made me wonder what my criterium for choosing a platform is. I’ve always told myself it’s about antifeatures (as defined by Benjamin Mako Hill), which is why Android doesn’t appeal to me. But arguably, systemd qualifies as an antifeature, and it nevertheless went on to dominate Linux distributions.

                                                                                                1. 1

                                                                                                  You’re a FreeBSD developer; if you’d listened to the cool kids on slashdot back in the day you’d have abandoned that doomed ship a long time ago ;)

                                                                                                  In hindsight, that might have been the right call. At the moment, I’m working more on Linux than on FreeBSD. Some things are nicer on one, some on the other (my ideal *NIX would have clone, futex, ZFS, capsicum and jails, for example). If Linux were not GPL’d, I think I’d be working now on bringing the features that I miss from FreeBSD across and give up on FreeBSD entirely. That said, I see Windows and *NIX as legacy systems at this point and I’m more interested in working on the things that will replace them.

                                                                                            2. 1

                                                                                              Just you wait a few months, and systemd will also do container orchestration! j/k :)

                                                                                              1. 1

                                                                                                Well, there is systemd-nspawn. Which is a sorta alternative to LXC. I haven’t tried either yet, so I can’t tell anything about their respective qualities.

                                                                                            3. 11

                                                                                              I’ve found the systemd man pages to be somewhat complete and comprehensible. Give man systemd a try and then you can gradually discover other tools by jumping from one to the next in the “see also” sections. It’s not the best approach to learning, but it’s not bad either.

                                                                                              1. 6

                                                                                                Exactly. This is what I’ve come to appreciate about systemd. Compared to what Linux had, it’s reasonably consistent, coherent, and thorough. Sure, it makes a few questionable decisions, sure it has had a rocky road at times (I recall my system locking up on shutdown in the early days thanks to something in journald), and sure there are individual parts of it for which I might personally prefer an alternative (runit for process management, cronie, syslog), but taken as a whole I feel so much more comfortable with what we have now. It gives a certain level of polish I’ve always felt lacking in Linux (OpenBSD has this coherent feel too, without systemd, but I can appreciate both approaches). And most importantly, despite the fears, it doesn’t seem to have affected the viability of alternatives. There are still plenty of distributions that avoid systemd entirely.

                                                                                                I’ve recently been looking for a Linux distribution supported (by default) by cloud providers that feels reasonably clean. I’ve been using OpenBSD but sometimes a Linux just makes certain things easier and I needed the option. Long story short I ended up on Fedora and the one part I really didn’t like was NetworkManager. It felt a bit of a mess with documentation scattered all over, disorganised manpages, and legacy formats (ifcfg-rh) just to confuse things. Then I realised that I could just use systemd-networkd which was well documented, far more simple, easy to configure with familiar syntax, and pretty broadly available (in base installs) in case I ever need to switch. l actually ended up using NetworkManager’s keyfile plugin, but discovering that was a chore, and I just want these things to work.

                                                                                              2. 6

                                                                                                One of the early adopters of systemd, Arch Linux, has fine cheat sheet -like documentation that goes through many things: https://wiki.archlinux.org/index.php/Systemd

                                                                                                Systemd isn’t so complex that you should be looking at certifications.

                                                                                                1. 2

                                                                                                  Newest edition of UNIX and Linux System Administration Handbook has a good chapter about systemd, I believe, but I am not sure if it will be as comprehensive as you’d want it.

                                                                                                1. 3

                                                                                                  The inability to mute groups without leaving them problem has been a big source of frustration for me. I looked around on the web and people suggested that an option to remedy that was in the beta version which I installed but never found any such option (so I guess it was probably abandoned). Can’t imagine the reasoning behind this silly behaviour.

                                                                                                  1. 2

                                                                                                    Mute how? Notifications from groups? That definitely is possible. I mute groups all the time.

                                                                                                    1. 4

                                                                                                      Sorry, I mean to archive groups. The moment someone replies, the group is unarchived and shows its little unread messages badge. Right now this group is not relevant to me, but I do not wish to leave it entirely.

                                                                                                      1. 1

                                                                                                        The notification badges. When I am muting something, I want it to stop doing anything. Although its totally possible that they didn’t want to add another line of if/else to the code.

                                                                                                        What I think is that, they can just get rid of both the features and add a block group kindof thing. Where you don’t receive any notifications at all from the group.

                                                                                                    1. 16

                                                                                                      I find all the admonishments to use POSIX shell rather than bash befuddling. Why would you not want to use Bash language features such as arrays and associative arrays? If your goal was portability across a wide swath of UNIX variants, then I understand that you might want to target POSIX shell, but that is a very unusual goal. In most instances why not leverage a shell language with very useful features?

                                                                                                      1. 31

                                                                                                        Because arrays and associative arrays are two of the most broken features in bash. You can use them if you’re an expert, but your coworker who has to maintain the script will likely not become an expert. (I’ve been using shell for 15 years and implementing a shell for 4, and I still avoid them)

                                                                                                        I implemented them more cleanly in Oil and some (not all) of the differences / problems are documented here.

                                                                                                        https://www.oilshell.org/release/latest/doc/known-differences.html

                                                                                                        • ${array} being equivalent to ${array[0]} is very confusing to people. Strings and arrays are confused.
                                                                                                        • associative arrays and arrays are confused. Try doing declare -A array and declare -a assoc_array and seeing what happens.
                                                                                                        • dynamic parsing of array initializers is confusing and buggy
                                                                                                        • in bash 4.3 and below (which is deployed on Ubuntu/Debian from 2016, not that old), empty arrays are confused with unset arrays (set -e). Not being able to use empty arrays makes them fundamentally broken

                                                                                                        I could list more examples, but if you’ve used them even a little bit, you will run into such problems.

                                                                                                        In Oil arrays and assoc arrays are usable. And you even have the repr builtin to tell you what the state of the shell is, which is hard in bash.


                                                                                                        edit: I forgot that one of the first problems I wrote about is related to array subscripts:

                                                                                                        Parsing Bash is Undecidable

                                                                                                        Also the security problem I rediscovered in 2019 is also related to array subscripts:

                                                                                                        https://github.com/oilshell/blog-code/tree/master/crazy-old-bug

                                                                                                        A statement as simple as echo $(( a )) is in theory vulnerable to shell code injection.

                                                                                                        • Shells that do not have arrays do not have the security problem.
                                                                                                        • You can inject code into shells with arrays with a piece of data like a[$(echo 42; rm -rf /)].

                                                                                                        All ksh derived shells including bash have this problem. Note that you don’t even have to use arrays in your program to be vulnerable to this. Simply using arithmetic in a shell that has arrays is enough! Because arithmetic is dynamically parsed an evaluated, and accepts $(command subs) in the array index expression.

                                                                                                        https://unix.stackexchange.com/questions/172103/security-implications-of-using-unsanitized-data-in-shell-arithmetic-evaluation/172109#172109


                                                                                                        bash is poorly implemented along many dimensions, but I would say arrays are one of the worse areas. But I learned that it got a lot of that bad behavior from ksh, i.e. “bash-isms” are really “ksh-isms”. So the blame for such a bad language to some extent goes back to AT&T and not GNU.

                                                                                                        1. 1

                                                                                                          Thanks. Without arrays, how do recommend handling cases where you need to build up a series of arguments to be passed to another command?

                                                                                                          1. 3

                                                                                                            I oscillate between:

                                                                                                            • Using strings and implicit splitting for known/trusted arguments. If you’re building up flags to a compiler like -O3 or -fsanitize=address, this is a reasonable option, and the one I use.
                                                                                                            • Using arrays and taking care not to use empty arrays (which is annoying).

                                                                                                            The fact that it’s awkward is of course a motivation for Oil :)

                                                                                                          2. 1

                                                                                                            in bash 4.3 and below (which is deployed on Ubuntu/Debian from 2016, not that old), empty arrays are confused with unset arrays (set -e). Not being able to use empty arrays makes them fundamentally broken

                                                                                                            Did you mean set -u? And to be fair, I would consider this more of a case of set -u being broken; it seems that their fix for it was to not trigger (even when unset) on * and @ indexes.

                                                                                                            1. 1

                                                                                                              Yes set -u. Either way, you can’t use empty arrays and “strict mode” together, which makes arrays virtually useless in those recent version of bash IMO.

                                                                                                              I have a memory of running into that the first time, and I couldn’t understand what was going on. And I ran into many more times after that, until I just stopped even trying to use arrays.

                                                                                                          3. 15

                                                                                                            Why would you not want to use Bash language features such as arrays and associative arrays?

                                                                                                            If you need them that’s a good sign it’s time to rewrite the script in real programming language.

                                                                                                            1. 12

                                                                                                              It’s not as “unusual” as you might think though. There are many people running OpenBSD on their work machines, where bash is simply not included in the default install. I could see why you’d want to make your script work without issue when bash isn’t an option. At the very core, a shell script should not do much: cleanup your local mailbox, aggregate multiple RSS feeds or perform a quick backup using rsync, … I have run openbsd on a laptop for some time, and I was delighted to see that all my personnal scripts were still working like a charm (besides calls to seq(1), this one killed me…). I’ve also written some simple C software, and wrapper scripts for it using POSIX shells only, and I was glad that nobody from the BSD world bothered me with these scripts.

                                                                                                              Another argument is that POSIX shells is much more simple to use and understand than the myriad of corner-cases and specificities that bash can have. It sure is a strengh to be able to use proper arrays, but bash manpage has 5 times more (48746 words) than sh (10224). Mastering bash as a language is definitely harder than sticking to POSIX, especially when you only write scripts that are < 20 lines.

                                                                                                              1. 4

                                                                                                                Even on Linux bash isn’t always included by default, especially on simple server setups/distros such as Alpine Linux.

                                                                                                                The seq thing is annoying, BSD has jot with different syntax. I wish they could agree on one or the other. Same with stat.

                                                                                                              2. 10

                                                                                                                In my experience, the moment my shell script becomes complex enough for a bash array is the moment it gets rewritten in Python.

                                                                                                                The only features from bash that have convinced me to require it in recent memory are herestrings (<<<) and the regex match operator (=~).

                                                                                                                1. 1

                                                                                                                  In my experience, the moment my shell script becomes complex enough for a bash array is the moment it gets rewritten in Python.

                                                                                                                  Same here. Even for the simple scripts since a script would become more complicated over time.

                                                                                                                2. 4

                                                                                                                  I use systems without bash installed by default. If you’re going to write a complex script that is too awkward to write in POSIX sh, then why not use a better (less quirky) language in the first place? Python for example is probably available in almost as many places as bash now. If you’re writing a shell script, you’re probably aiming for portability anyway, so please write POSIX shell if you can.

                                                                                                                  1. 2

                                                                                                                    I write many bash scripts where portability is not a concern and I think the language can be quite functional and even elegant when you are pulling together different cli tools into a new program.

                                                                                                                    1. 2

                                                                                                                      Certainly if it’s not a concern and you enjoy the language then I don’t see a problem with using bash! My main issue is that I don’t think it’s at all an unusual goal to use POSIX sh for portability.

                                                                                                                      1. 2

                                                                                                                        Ubuntu shaved quite a bit off their (pre-systemd) boot times by just switching /bin/sh from bash to dash (although I can’t find the exact number right now). The increased performance may be a concern (or is at least nice) in some cases (bending over backwards for it would of course be silly).

                                                                                                                      2. 1

                                                                                                                        Because there are plenty of use cases, like installers, where relying on another language isn’t possible.

                                                                                                                        1. 4

                                                                                                                          Why not do the heavy lifting in awk, if your’e in such a restricted environment?

                                                                                                                          1. 1

                                                                                                                            When I talk about writing shell applications, I personally am talking about using bash plus all the standard UNIX tools that come with every UNIX base install.

                                                                                                                            Awk is one of those tools.

                                                                                                                            1. 5

                                                                                                                              And when I say do the heavy lifting in awk, I mean more or less ignore the shell, and use awk. You get proper arrays, reasonable syntax for writing algorithms, relatively little shelling out, relatively sane string manipulation, and sidestep a bunch of issues around shell variable reexpansion.

                                                                                                                              And your code is more portable.

                                                                                                                              1. 2

                                                                                                                                I believe that. I haven’t used awk that way and I’ll admit that’s because I don’t know it well enough.

                                                                                                                                I know I’m asking a lot but might you be able to link to an example of awk being used in this way?

                                                                                                                                I mostly end up using stupid simple awk invocations like awk ‘{print $3}’ :)

                                                                                                                                1. 1

                                                                                                                                  Here’s an example from the FreeBSD tree. Lines 107 through 788 are one big awk script. https://github.com/freebsd/freebsd/blob/master/sys/kern/makesyscalls.sh

                                                                                                                                  It parses the syscall definitions here, and generates C: https://github.com/freebsd/freebsd/blob/master/sys/kern/syscalls.master

                                                                                                                          2. 1

                                                                                                                            Why isn’t it possible and why can’t it be done in POSIX sh?

                                                                                                                            1. 1

                                                                                                                              The OP’s post has two points.

                                                                                                                              1. If you’re writing anything complex at all, use a different language.

                                                                                                                              That’s what I was addressing.

                                                                                                                              To your point, there are customer environments where shipping UNIX systems with languages like Python is prohibited, either because of security constraints, maybe disk usage, etc.

                                                                                                                              1. bash versus POSIX shell? If bash ships as /bin/sh on a given UNIX system, I don’t see the difference as important enough to be worth any gymnastics to get one installed, but you may have a higher dimensional understanding of the differences than I do.
                                                                                                                              1. 1

                                                                                                                                I don’t think it’s really a big issue. Certainly for myself it’s only a minor frustration if I need to install bash for a simple script which gains nothing from using bash. I’ve just seen the #!/bin/bash shebang far too many times when it’s not necessary, and I think that a lot of the time that’s just because people don’t know the difference. It’s certainly not the end of the world and if bash feels like the right tool for whatever your use case is then I’m not going to argue! It would just be nice if there was a little awareness of other systems and that people would default to #!/bin/sh and only consider bash if they find a specific need for it. I imagine that in most cases shell scripts are used there actually is no need.

                                                                                                                                I’m a little obsessed with building my workflow around shell scripts and I have never found a need for bash extensions (YMMV). The other big benefit other than portability as others have suggested is the digestibility of the manpages for, e.g. dash.