1. 16

    If you’re at the point where you need to parse flags, like in this example, you’re no longer writing “a simple script”: it’s now a full fledged program. Do yourself a favor and use an actual programming language. Yes, Bash can technically do a lot, but as someone who works on a project centered around 100k+ lines of Bash, it’s going to slow you down and introduce its own terrible categories of bugs.

    1. 11

      I struggle with this a lot, because there’s definitely some truth to this. For me the test is usually “is the primary role of this script/app to just call other binaries”. If the answer is yes I lean to shell scripts, as I’m unconvinced writing e.g. Python with subprocess calls, or C# with System.Diagnostics.Process, etc … represents an improvement. It’s likely to be quite a bit longer with all the extra process management, with minimal gain if you’re writing Bash in a reasonably disciplined way (e.g. use shellcheck, consider shfmt).

      Part of that discipline for me is the exact “boilerplate” which a template like the linked article provides.

      EDIT: Obviously once we’re talking 100k+ or even 10k+ lines of Bash we’re in an entirely different realm and op has my deepest sympathies.

      1. 7

        Wow really 100K lines of bash? What does it do?

        I keep track of such large programs here:

        https://github.com/oilshell/oil/wiki/The-Biggest-Shell-Programs-in-the-World

        There are collections of scripts that are more than 100K lines for sure, but single programs seeminlyg top out around 20-30K… I’d be interested to learn otherwise!

      1. 28

        I have been in a similar spot and I can understand this kind of small disappointment. I will offer a few thoughts that may or may not help!

        This can be hard, but understand that by the maintainer setting aside your commit, they were surely not making a personal slight. By instead using their own commits for the fix, they are NOT saying your proposal was “wrong” or “not good enough”.

        Remember the only social expectation a good open source maintainer could owe you for an unsolicited proposed contribution is a hearty thanks. You made the valiant choice to give freely of your time and talent. You should feel good about that! What you offered was a gift, and when we offer any gift, we must be mindful that we cannot expect reciprocation. That would make it a transaction, not a gift!

        There are a variety of causes for why the maintainer did not merge your patch as you may have imagined they would.

        You didn’t say what project it was, which is fine, but if it’s a big project or from a large organization, consider they often have strict contribution guidelines that are necessary for legal reasons, such as a “contributor license agreement”.

        If your report was just a small typo or a few words, it would be a little silly for them to ask you to sign a big CLA before merging your patch. It’s possible they might have read your report, ignored the patch, and made their own fix. If that’s what happened, they actually did you a favor, by saving both of you that overhead effort.

        Most large projects have a file in the repo named CONTRIBUTING or similar, that would lay this out.

        It’s also possible they wanted to make the fix a slightly different way, and it was easier for them to do it directly rather than merge your patch and then make another commit on top of it. Maybe they want the commit message writtem a certain way, so the git history is more to their liking. Projects do not get points based on how many commits they merge in! :)

        As a takeaway, remember that what you offered did have value – you saw the pothole, and then your intended impact was achieved, in that it got fixed. The question of whose version of the fix made it to the git history becomes irrelevant. Future strangers will no longer trip over this particular pothole thanks to your report.

        For myself, I am glad the open source community has people doing things like you did!

        1. 4

          I think this comment is spot on but I wanted to add a few points.

          When I first started getting into free/libre/open source, I had a default understanding of what it meant to be “open” and allow for community contributions in that I thought these type of “drive-by” patches/bug fixes/etc. were the norm. Free software projects are varied and have different ideas of what it means to allow for community involvement and this “bazaar” approach of folding in changed proposed by community members with low engineering involvement are one style of contribution. The “cathedral” approach is another, whereby the developers are the ones with sole access to the repository and only allow contributions from their inner circle and extend their inner circle selectively with folks who commit to having a deeper involvement in the project.

          Remember that “The Cathedral and the Bazaar” [1] talks about an Emacs/Stallman type style of contribution (the “cathedral”) vs. Linux/Torvalds style of contribution (the “bazaar”). I had only really heard the title and, in my ignorance, assumed that Raymond was making a “Linux vs. Microsoft” argument.

          From my personal experience, when people submit patches to my own (very small and not very popular) FOSS projects, I have an initial reaction of “not quite like that, like this”. I want to practice a more “bazaar” like methodology and so it’s something I’m trying to ween myself off of but it’s a natural reaction that I have to overcome and one, I imagine, many other people feel. I think accepting contributions of this sort also provide a welcoming approach to people, so they get positive re-enforcement and pave the way for more substantial contributions.

          I think [tedchs] correctly points out that having a CONTRIBUTING file for guidelines (or something similar) is sometimes present and should maybe be the norm (maybe with some type of template people can choose from) but this is a layer of process that needs to be created. For small projects, especially ones that don’t have a concrete idea of how to accept contributions, this is a layer of process infrastructure that adds complexity and might not be appropriate for the scale or scope that the project is currently.

          As a general rule of thumb, when trying to contribute to other projects, I usually create an issue with an offer to help and, only after confirmation, proceed to contribute. I violate this all the time but this is one tactic to differentiate between whether the project is “cathedral-like” or “bazaar-like”

          [1] https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar

          1. 3

            To add a different voice:

            I, as a maintainer ,think this is disrespectful.

            If I receive PR, it’s not even a question for me whether I acknowledge the contributor’s work and retain his/her authorship.

            If the PR needs to be adjusted, I either tell the contributor, or do it myself in a separate commit. If the adjustment is very small, I may do it in the original commit, but still retain him/her as the author.

            1. 2

              If your report was just a small typo or a few words, it would be a little silly for them to ask you to sign a big CLA before merging your patch.

              It sometimes happens even for non-trivial patches and even if you sign CLA. Example: Unix domain sockets (UDS, AF_UNIX) in System.inheritedChannel() and elsewhere in Java. (later implemented internally).

              Nobody is obliged to accept your contribution (which is OK – on the other hand, nobody can force you to merge his code).

              1. 2

                If your report was just a small typo or a few words, it would be a little silly for them to ask you to sign a big CLA before merging your patch.

                Copyright rights in the U.S. can be terminated by the estate after the author dies. This is one of the reasons that most corporate open source projects require a CLA. Here is a post by a lawyer that describes other problems solved by a CLA.

              1. 3

                I would be interested in a good explanation of ECC and its variants like ECDSA. Sadly this article really didn’t give much of an explanation.

                1. 33

                  They’re more popular than ever, as AWS Lambda.

                  1. 27

                    If I was good at writing satire, I would write a satire article that is a CGI scripting tutorial, but never use the word CGI and instead call it a “new Open Source function as a service library”.

                    Similar twist: “Hi folks, I wrote a new library that converts JVM byte code to WebAssembly. For the first time ever, we can write Java that runs in the browser! Never before seen!”

                    1. 11

                      They are both stack machines, so it should be simple enough I guess. Don’t give me ideas.

                      1. 5

                        Do it.

                      2. 3

                        I started trying to write this article a while back, not as satire, but as a direct comparison to the evolution of serverless. But then I realized it’s probably been done better than I could do and aborted.

                        1. 7

                          I have literally never not published something just because I think it might have been done. If I see something that is almost entirely what I wrote, sure, I’ll axe it. (Even retroactively, in one case, where I read someone else with a better take and thought, “oh, never mind then.”) But if I haven’t specifically read an article of what I want to write, then:

                          1. I might have a unique take after all.
                          2. Even if my take isn’t unique, it might be different enough to be helpful to someone else.
                          3. Even if it’s neither unique nor different enough, if I’m not aware of it (and after deliberately cursory search can’t find it, if applicable), it will likely reach a different audience.

                          In the draft post you’ve got, I think you are heading in a good direction, and it might be worth continuing. I’d suggest dropping the FastCGI/SCGI/WSGI/Rack section in favor of diving a lot more into early attempts to speed up CGI requests and how those relate to lambdas (you touch on mod_perl, but I’d also at least touch on PHP in particular, and quite possibly AOLServer, as close peers), highlighting similar issues with startup time and how lambdas are trying to solve them in their own ways/differently.

                          The other way to approach this kind of thing, incidentally, which I like for my equivalent writings on these axes, is to walk through how trying to solve the problems with the old old-and-busted resulted in the new old-and-busted. You can write that kind of article sarcastically, but you definitely don’t have to; my article comparing Win32 to Flux has a bit of humor in it, but I deliberately avoided anything past that. If you went that route, the FastCGI/SCGI section fits better, but also pairs very nicely with talking about things like the Danga Interactive products (Gearman, memcache, Perlbal, etc.), which turn out to be necessarily reinvented whenever a PaaS-like environment is used.

                          Anyway, all this to say: I’d love to see someone actually write a post along these lines. If you really don’t want to finish yours, you’ve given me half a mind to take my own stab.

                        2. 3

                          I have that as a laptop sticker. I don’t know if commercial advertising on lobste.rs is appropriate (even for an enterprise which makes me on the order of $2/month) so I won’t link it directly, but you could probably find it quite easily by searching redbubble.com for “serverless cgi-bin”

                          There is the reasonable objection that using a FaaS platform you have the expectation that it will autoscale to performance levels far in excess of anything that cgi-bin of yore managed, but really, that’s a implementation detail not an attribute of the API

                          1. 2

                            Kelsey Hightower made a similar comparison at Gophercon: https://youtu.be/U7glyWYj4qg

                            1. 2

                              “This is xinetd… the new hotness”

                              Love it.

                        1. 4

                          I love the overall concept, but Scuttlebutt seems to completely ignore all existing protocols for RPC, serialization, LAN discovery, cluster gossip, message encoding, etc. It reinvents a whole lot of wheels all at once. “But it’s simple! It’s just JSON!” they might say. But the difficulty isn’t in the syntax, it’s in the semantics.

                          Unless I missed something, local sync requires visibility of peers’ UDP broadcasts on a shared LAN. Nearly all public hotspots and home guest LANs disable peer visibility, so this seems like a nonstarter. BLE beacons could be an interesting alternative.

                          I’d rather use a version of ActivityPub or Solid (https://solid.mit.edu) that works truly offline, such as with BLE + Wi-Fi Direct.

                          1. 15

                            As further rationale for avoiding master / slave, it’s not even a clear conceptual metaphor. Primary and secondary nodes flip roles all the time, but a human slave never becomes the master. Also there is/was routine and severe distrust, misunderstanding, abuse, and deceipt from a master to a slave, whereas in a database a better metaphor would be pilot/copilot. The pilot is by default in charge but through explicit protocols the copilot will “take the airplane” during flight.

                            1. 7

                              Not a great question pool IMHO Many of these questions are biased toward a person with a very specific personal profile. Asking about a “home lab” is disqualifying of people who don’t spend weekends terminating Cat5 or reimaging old Dell servers from eBay. A lot are outdated, e.g. the difference between a “router” and a “gateway” (I’ve seen that for 15+ years and it didn’t make sense then either). Others are uselessly vague, particularly “what is redirection?” with no context.

                              1. 1

                                The test ended up being an actual in terminal test with specific goals and full access to man pages and the internet. So it wasnt anything like these questions (though they were helpful to distract me the night before). It went pretty well I think!

                              1. 2

                                On this topic does anybody have a good way to address the question “what other branches will have a merge conflict with this one after this merges into master”?

                                1. 1

                                  Where’d we get this trend of libraries self-described as “minimal”? Is there even a definition of what that means? If a library exposes 100 functions, but they’re all used, is that “minimal”? Wouldn’t it be better to have a library described as “Everything you need to solve X without filling in the gaps yourself”? Reminds me of “light” food… after all of human history desperately looking for calories, suddenly having *less” calories is the main selling point – “fat free half and half” being the worst example.

                                  1. 7

                                    Code is different from food. Code is liability.

                                    1. 3

                                      There is also an explanation in the email on why it is useful to have a minimal crypto library in the kernel. In short, you mostly don’t need flexible APIs when in kernel. That might be different in userland.

                                    1. 5

                                      tl;dr: the underlying data is available under /proc and they wrote some Python to traverse those directories and symlinks.

                                      1. 2

                                        Alex is one of the most billiant people in the IT industry and he speaks the truth.

                                        1. 5

                                          The $30 level is totally worth it for The Linux Programming Interface, which is a beautiful reference for anyone who’s doing systems programming or wants to know how all the syscalls and administration commands like “chmod” really work. Very authoritative!

                                          1. 1

                                            Really? I always feel the manual pages are much better documenting these than that particular book. The Linux man pages are much easier to search for particular things than a book, paper or electronic version.

                                          1. 3

                                            you will find that the old way is deprecated, but the new way that they recommend you use is still in beta or alpha

                                            This is broadly applicable to things both inside and outside of Google. It’s pretty normal for them. Fortunately things that are “deprecated” usually continue running for a long time and they are clear about future deadlines.

                                            1. 3

                                              Some facts that remove all the sensationalism:

                                              • This person was an IT contractor,. not an employee
                                              • They weren’t “fired” per se; their contract simply wasn’t renewed due to a silly oversight and bad processes
                                              • The “machine” had nothing to do with the termination, except for implementing some apparently-naive ticket scripting to revoke access

                                              None of this is shocking to me EXCEPT it’s unconscionable his staffing company (and “recruiter” as he refers to them) didn’t call to say plainly “your contract was sadly not renewed, do not go to work as of date X until a new contract may or may not be signed”. The fact his staffing company and host manager either didn’t know or didn’t tell him, and allowed him into the building to do work after the contract was up, are very problematic and could be legally actionable.

                                              1. 1

                                                This reads like a proper sci-fi short story and I suspect that a LOT of it was highly embellished. It makes sense to me that his direct managers would be confused as to what was going on, but it does not make any sense that it took them three weeks to figure it out. “The thing that I’m responsible for is on fire and my contractor can’t access anything” is a managerial complaint that should make its way up to the fucking CEO in a matter of hours if all anyone else wants to do is point fingers. So either important details were omitted to make it a better story or this big business is exceptionally bad at being a business.

                                                1. 3

                                                  I think you have too much faith in the efficiency of business. There are many reasons people might choose not to escalate, for example - even if it doesn’t help the business. Most managers are in the business of protecting their little patch and their own career prospects, nothing else. Unfortunately, I found the story quite plausible (although it could be exaggerated as you say).

                                                  1. 1

                                                    You’re probably right. I’ve had only 10 months experience in a truly bureaucratic business myself (not counting the military which is actually suprisingly efficient, just slow) and that wasn’t even a big company. I do find it plausible, just the details far-fetched.

                                              1. 2

                                                FWIW, for Lets Encrypt I use acme.sh in a daily cronjob that renews the cert every 30 days (i.e. if it’s within 60 days of expiration). Then I have alerts that fire if the time runs down. It’s kind of aggressive but it protects against non-outage risks like subtle API changes or bugs.

                                                1. 2

                                                  Does your acme.sh automatically restart your webserver processes? If so, does it run as root or do you use a sudo or something else to restart them?

                                                  1. 1

                                                    In my case it feeds the new cert into the configuration management tool and it then pushes out the new config like normal. acme.sh itself is decoupled from the web server.

                                                1. 1

                                                  Interesting architecture, but not impressed by the 25M API calls per month.

                                                  1. 2

                                                    It’s always confused me when “queries per month” is used as a headline. That’s an indicator of the user base, not the technical infrastructure. On average it’s about 10 qps, which for this feature requires possibly a large Arduino. :) BTW, no idea why a web site owner would accept a paid dependency just to do a btree lookup “as a service”.

                                                    1. 1

                                                      Because they’re solving hard problems that will change the world through ip geolocation as a service.

                                                  1. 2

                                                    For somebody on the sidelines of Node, what’s the latest recommendation to avoid this? I’ve heard NPM supports namespaced modules like Go does, but is that predominant? Can it reference a package checksum/signature? Is there any way to control 3rd and 4th tier deps?

                                                    1. 6

                                                      It’s pretty easy…don’t use Node or NOM.

                                                      I’m not kidding.

                                                      1. 6

                                                        This is pretty much it. Working with NodeJS/NPM even at distance is like having a romantic dinner in a fucking dumpster fire.

                                                        I’ve had the displeasure of working (backend/ops) on some projects where they insisted on flavour of the week frontend stuff. So there was node, npm. A dozen other interdependent build tools. I ended up helping the guy running the project track down a bug - basically a downstream library had changed something and it broke stuff. That same module was included somewhere like 11 or 12 times, all slightly different versions. The dependency tree for that project’s JS files (keeping in mind this was frontend only) was like one of those “European royalty” family tress: everything is somehow related, and just looking at it makes you want to scrub your eyes with tobasco.

                                                    1. 3

                                                      I’m really curious about the background of Nim users. I imagine its more folks coming from a Python/Ruby/Smalltalk background as compared to a C/C++ background but I have no actual basis for that assumption.

                                                      If you are an active Nim user, I’d love to know your background and why Nim appeals to you.

                                                      1. 2

                                                        I’m a python user; slowly trying to replace python scripts with nim. Nim’s standard library is pretty decent. Unless i’m doing something highly specialized (eg: I’ve not tried to do any AWS automation in Nim), nim is often quick enough to write with the same amount of effort I would have spent writing Python.

                                                        1. 5

                                                          I have 3 primary questions:

                                                          • why are you trying to replace Python with Nim?
                                                          • what it is about Python that has you wanting to replace it?
                                                          • what is it about Nim that lead you to trying it as your replacement?
                                                          1. 4
                                                            • over the last few years, I’ve come to see the value in a compiler and statically type checked language. Especially one like nim, where type checker is your friend and not onerous.
                                                            • I’m tired of tracking dependencies in a virtualenv etc., having a single binary is soo much nicer.
                                                            • There is a huge amount of head room to grow as a programmer (eg: I’ve barely scratched the surface of what I can do with macros, templates and other “higher” language features).
                                                        2. 1

                                                          FWIW, I’m a polyglot programmer and I really like Nim but the only thing I’ve used it for is the Advent of Code puzzle solutions last December. It’s a lot of fun to program with and I like that it compiles to a single binary with no deps like Go.

                                                        1. 8

                                                          It’s a super easy mistake to make! They’re so tiny, and hard to find! One day we’ll have a service where you can just type “how many legs do lobsters have” without personally going to Maine and renting a boat and lobster traps to find out.

                                                          1. 7

                                                            The wikipedia article is fascinating:

                                                            Lobsters live up to an estimated 45 to 50 years in the wild, although determining age is difficult.

                                                            Research suggests that lobsters may not slow down, weaken or lose fertility with age, and that older lobsters may be more fertile than younger lobsters. This longevity may be due to telomerase

                                                            Lobster longevity is limited by their size. Moulting requires metabolic energy and the larger the lobster, the more energy is needed; 10 to 15% of lobsters die of exhaustion during moulting, while in older lobsters, moulting ceases and the exoskeleton degrades or collapses entirely leading to death

                                                            1. 5

                                                              I wonder what would happen if the exoskeleton was artificially reinforced in an old lobster. Would it live a lot longer?

                                                              1. 1

                                                                Isn’t there a similar story with squids, that they grow as long as they have the resources to do so?

                                                            1. 32

                                                              I don’t see why this progress bar should be obnoxiously put at the top of the page. It’s cool if you wanna do a donation drive but don’t push it in the face of everybody who comes here. Honestly at first I thought this was a bar for site expense. Then I realised it’s to ‘adopt’ an emoji.

                                                              1. 7

                                                                Lobsters isn’t a daily visit for most readers, probably even for most users. They can’t see it to join in if there isn’t anything visible for it, and it has an id for adblocking if you prefer not to see it.

                                                                1. 22

                                                                  Personally a check this site quite regularly on my mobile device… which doesn’t have an ad-blocker.

                                                                  1. 13

                                                                    That sounds awful. If you’re an android user, normal uBlock Origin works on Firefox for Android just like it does on desktop. :)

                                                                    1. 3

                                                                      Or use Block This!, which blocks ads in all apps.

                                                                      1. 3

                                                                        Oh, that’s a cool little tool. Using a local VPN to intercept DNS is a neat trick. Unfortunately doesn’t help with in this case because it blocks requests to domains and not elements on a page via CSS selectors.

                                                                        That does make me want to actually figure out my VPN to home for my phone and setup a pi-hole, though.

                                                                      2. 2

                                                                        Ohh! Good to know, thanks.

                                                                      3. 2

                                                                        Firefox 57+ has integrated adblocker nowadays, on both desktop and mobile; plus, there’s also Brave.

                                                                      4. 27

                                                                        That is still annoying that I need to setup my adblocker to fix lobste.rs. So much for all the rant articles about bad UX/UI in here.

                                                                        1. 11

                                                                          maybe one could just add a dismiss button or sometimes like that? I don’t find it that annoying, but I guess it would be a pretty simple solution.

                                                                          1. 1

                                                                            I concur, either a client side cookie or session variable.

                                                                            1. 1

                                                                              Well, yeah… that’s how you could implement it, and I guess that would be the cleanest and simplest way?

                                                                          2. 2

                                                                            It’d be great to see data about that! Personally I visit daily or at least 3 times a week. Lack of clutter and noise is one of the biggest advantages of Lobsters. And specifically, I looked at the link, and I have no idea who this Unicode organization is, or their charitable performance, or even if they need the money. I’d imagine they are mostly funded by the rich tech megacorps?

                                                                            1. 1

                                                                              [citation needed] ;-)

                                                                            2. 3

                                                                              Adopting an emoji isn’t the end goal: the money goes to Unicode, which is a non-profit organization that’s very important to the Internet.

                                                                              1. 5

                                                                                If this bar actually significantly annoys you, I’m surprised you haven’t literally died from browsing the rest of the internet.