Threads for tedchs

  1. 5

    I have mixed feelings about this paper. On the upside, for introductory students who don’t yet have any sense of the speed of light, or the size of the earth, I could see the traceroute being an interesting classroom activity.

    It’s worth being aware of a few confounding factors that will make this estimation technique unreliable:

    • Undersea cables can’t be laid in a theoretically straight line due to obvious seabed obstructions like coral reefs, volcanoes, etc.
    • traceroute and ping responses are often deprioritized or rate limited by routers
    • routers are usually not located exactly at the undersea cable’s landing site, or even particularly near the center of the metro area that may be suggested by their PTR record.

    BTW, what’s up with all the resources that are listed in the acknowledgements section? Why does it require a NASA grant to run a traceroute on a classroom computer that’s already on the internet?

    1. 3

      BTW, what’s up with all the resources that are listed in the acknowledgements section? Why does it require a NASA grant to run a traceroute on a classroom computer that’s already on the internet?

      Probably that this guy had a NASA grant for something else, and was required to acknowledge it.

      1. 2

        My bad. Thanks for posting the correct link.

      1. 3

        That is a surprise. I’ve only used base64 as the final step in encoding data for transport over protocols or formats that don’t support binary data. And there are a lot of them… XML being fairly notable for not being enable to encode NUL.

        1. 1

          XML can encode NUL, different protocols may do it differently though. There is no one “XML encoding”.

          1. 2

            XML can encode NUL, different protocols may do it differently though. There is no one “XML encoding”.

            I can encode NUL as base64-text-rasterized-in-a-png, too.

            There’s a difference between “you can build an encoding scheme in this language which can represent NUL” and “the language has explicit support for this feature, and it works well”.

            1. 1

              Does XML accept a numeric character reference for NUL, like this: � ?

              https://en.m.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references

          1. 2

            I’m afraid my new year’s resolution is really low, 72 DPI. … OK, I’ll see myself out.

            1. 8

              Evil idea: Fingerprint the internal tech stack of a web app by taking advantage of the fact that all JSON parsers have different behavior. Sending carefully-crafted JSON payloads that will throw an error for one “valid, but odd” syntax but not another, like {"a": 0.}.

              1. 4

                This is why Postel’s law is a bad idea something you have to be very careful about.

                1. 2

                  My version of Postel’s Law, with the benefit of hindsight: “If you accept crap, then crap is what you will get.”

              1. 1

                I’ve administered DNS for many years but didn’t know about some of these. I love the commentary!

                1. 6

                  I think the fact that SSH’s wire protocol can only take single string as a command is a huge, unrecognized design flaw. It would be so much better to take an array of command name plus arguments, just like the underlying syscalls. The distinction is similar to Dockerfile giving the choice between a command as string vs. array, where the string gets passed to a shell, and the array doesn’t.

                  1. 8

                    This was a long time ago and I wasn’t doing much system administration then on account of being way too young to hold a job (well, I’m not doing much of that now either but back then I was doing even less of it…) so I might be misremembering it. Please take this with a grain of salt – perhaps someone who did more Unix back then remembers this better? By the time I started doing anything with Unix, rsh was pretty much a relic.

                    I think that this is not specifically SSH’s design, I think it’s deliberately made to be identical to rsh’s. That was actually a very clever design choice – ssh was specifically meant to replace rsh, which was 10+ years at the time, so coming up with a tool that could run 10 years’ worth of scripts was important, otherwise no one would’ve wanted it. It was a very efficient solution to a non-technical problem.

                    I suppose it would be possible to add a second syntax today, similar to Docker’s. I don’t know if that would be a good idea. IMHO it wouldn’t be but I think I’m prooobably not in the target demographics for these things ¯_(ツ)_/¯

                    Edit: oh yeah, if anyone’s wondering what arcane incantation from the ancient lore of the Unix greybeards is required to do that, just do:

                    echo "cd /tmp ; pwd" | ssh user@machine.local
                    

                    There are probably other/better ways, this one’s just my favourite.

                    FWIW, this one comes naturally to me, it’s the Docker way that “feels” weird to me. I don’t think one’s objectively better than the other, it’s just that I’d been doing it this way (with many other tools, not just SSH) for many years by the time Docker showed up. (Then again, it makes a lot of sense in Docker’s context, and Docker didn’t have to be a drop-in replacement for anything other than “works on my machine”, so I think they did the right thing)

                    1. 2

                      I appreciate the point, but one minor warning: unlike ssh user@machine command, your echo command | ssh user@machine does allocate a pty for ssh (unless you use -T), which may affect the behaviour of some commands.

                      1. 3

                        Oh, yeah, this is a useful thing to remember! I think newer versions are (at least on some systems?) smart enough to figure out not to allocate a pty if stdin isn’t an actual terminal but I think that’s a whole other minefield. The reason why I usually don’t add -T explicitly is weird muscle/brain memory trained on a couple of somewhat restricted use cases.

                        I can’t edit my post anymore so here’s hoping nobody reads this thread halfway in the future :-D.

                  1. 3

                    Unpopular opinion: by the time a shell script has reached this level of structure, it’s time to rewrite it in a different programming language that more naturally supports structured code, data types, etc.

                    1. 3

                      Not unpopular.

                    1. 23

                      There’s another reason to fix this: using punctuation properly allows screen readers to pronounce the words without it being a jumble.

                      1. 2

                        I can relate to this problem and I am glad there’s some creative work in this area.

                        The result of this workflow is still a chain of commits in a single branch. That might be ok for a solo project, but in a team environment, the workflow usually calls for separating topics by branch, so they can each be reviewed as a separate pull request (or similar review tool). I guess one could cherry pick changes into different branches after the fact, but could that have a role in this tool?

                        My usual workflow, when making multiple unrelated changes to a repo, is to use git worktree to get isolated directories in which to make each change. Each worktree is a separate branch, which also makes it easy to push each set of commits separately, to be reviewed as PRs.

                        1. 1

                          Hey I’m the author. I think it could be extended to support a more branch-based workflow.

                          When you plan a commit, the working branch is stored with it. In theory, the git plan commit command could switch you onto the plan’s branch first, before creating the commit.

                          Perhaps when creating a planned commit, you could set a commit_in_new_branch flag to control this kind of behaviour.

                          I’m cautious about adding too much functionality to the tool, but if you have any other ideas for branch-based workflows I’d love to hear them.

                        1. 4

                          GitHub has an “account export” feature similar to Google Takeout; see https://github.blog/2018-12-19-download-your-data/ .

                          1. 1

                            Well duh, if Python OO wasn’t pointless it would use -> syntax, not a dot!

                            1. 16

                              If you’re at the point where you need to parse flags, like in this example, you’re no longer writing “a simple script”: it’s now a full fledged program. Do yourself a favor and use an actual programming language. Yes, Bash can technically do a lot, but as someone who works on a project centered around 100k+ lines of Bash, it’s going to slow you down and introduce its own terrible categories of bugs.

                              1. 11

                                I struggle with this a lot, because there’s definitely some truth to this. For me the test is usually “is the primary role of this script/app to just call other binaries”. If the answer is yes I lean to shell scripts, as I’m unconvinced writing e.g. Python with subprocess calls, or C# with System.Diagnostics.Process, etc … represents an improvement. It’s likely to be quite a bit longer with all the extra process management, with minimal gain if you’re writing Bash in a reasonably disciplined way (e.g. use shellcheck, consider shfmt).

                                Part of that discipline for me is the exact “boilerplate” which a template like the linked article provides.

                                EDIT: Obviously once we’re talking 100k+ or even 10k+ lines of Bash we’re in an entirely different realm and op has my deepest sympathies.

                                1. 7

                                  Wow really 100K lines of bash? What does it do?

                                  I keep track of such large programs here:

                                  https://github.com/oilshell/oil/wiki/The-Biggest-Shell-Programs-in-the-World

                                  There are collections of scripts that are more than 100K lines for sure, but single programs seeminlyg top out around 20-30K… I’d be interested to learn otherwise!

                                1. 28

                                  I have been in a similar spot and I can understand this kind of small disappointment. I will offer a few thoughts that may or may not help!

                                  This can be hard, but understand that by the maintainer setting aside your commit, they were surely not making a personal slight. By instead using their own commits for the fix, they are NOT saying your proposal was “wrong” or “not good enough”.

                                  Remember the only social expectation a good open source maintainer could owe you for an unsolicited proposed contribution is a hearty thanks. You made the valiant choice to give freely of your time and talent. You should feel good about that! What you offered was a gift, and when we offer any gift, we must be mindful that we cannot expect reciprocation. That would make it a transaction, not a gift!

                                  There are a variety of causes for why the maintainer did not merge your patch as you may have imagined they would.

                                  You didn’t say what project it was, which is fine, but if it’s a big project or from a large organization, consider they often have strict contribution guidelines that are necessary for legal reasons, such as a “contributor license agreement”.

                                  If your report was just a small typo or a few words, it would be a little silly for them to ask you to sign a big CLA before merging your patch. It’s possible they might have read your report, ignored the patch, and made their own fix. If that’s what happened, they actually did you a favor, by saving both of you that overhead effort.

                                  Most large projects have a file in the repo named CONTRIBUTING or similar, that would lay this out.

                                  It’s also possible they wanted to make the fix a slightly different way, and it was easier for them to do it directly rather than merge your patch and then make another commit on top of it. Maybe they want the commit message writtem a certain way, so the git history is more to their liking. Projects do not get points based on how many commits they merge in! :)

                                  As a takeaway, remember that what you offered did have value – you saw the pothole, and then your intended impact was achieved, in that it got fixed. The question of whose version of the fix made it to the git history becomes irrelevant. Future strangers will no longer trip over this particular pothole thanks to your report.

                                  For myself, I am glad the open source community has people doing things like you did!

                                  1. 4

                                    I think this comment is spot on but I wanted to add a few points.

                                    When I first started getting into free/libre/open source, I had a default understanding of what it meant to be “open” and allow for community contributions in that I thought these type of “drive-by” patches/bug fixes/etc. were the norm. Free software projects are varied and have different ideas of what it means to allow for community involvement and this “bazaar” approach of folding in changed proposed by community members with low engineering involvement are one style of contribution. The “cathedral” approach is another, whereby the developers are the ones with sole access to the repository and only allow contributions from their inner circle and extend their inner circle selectively with folks who commit to having a deeper involvement in the project.

                                    Remember that “The Cathedral and the Bazaar” [1] talks about an Emacs/Stallman type style of contribution (the “cathedral”) vs. Linux/Torvalds style of contribution (the “bazaar”). I had only really heard the title and, in my ignorance, assumed that Raymond was making a “Linux vs. Microsoft” argument.

                                    From my personal experience, when people submit patches to my own (very small and not very popular) FOSS projects, I have an initial reaction of “not quite like that, like this”. I want to practice a more “bazaar” like methodology and so it’s something I’m trying to ween myself off of but it’s a natural reaction that I have to overcome and one, I imagine, many other people feel. I think accepting contributions of this sort also provide a welcoming approach to people, so they get positive re-enforcement and pave the way for more substantial contributions.

                                    I think [tedchs] correctly points out that having a CONTRIBUTING file for guidelines (or something similar) is sometimes present and should maybe be the norm (maybe with some type of template people can choose from) but this is a layer of process that needs to be created. For small projects, especially ones that don’t have a concrete idea of how to accept contributions, this is a layer of process infrastructure that adds complexity and might not be appropriate for the scale or scope that the project is currently.

                                    As a general rule of thumb, when trying to contribute to other projects, I usually create an issue with an offer to help and, only after confirmation, proceed to contribute. I violate this all the time but this is one tactic to differentiate between whether the project is “cathedral-like” or “bazaar-like”

                                    [1] https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar

                                    1. 3

                                      To add a different voice:

                                      I, as a maintainer ,think this is disrespectful.

                                      If I receive PR, it’s not even a question for me whether I acknowledge the contributor’s work and retain his/her authorship.

                                      If the PR needs to be adjusted, I either tell the contributor, or do it myself in a separate commit. If the adjustment is very small, I may do it in the original commit, but still retain him/her as the author.

                                      1. 2

                                        If your report was just a small typo or a few words, it would be a little silly for them to ask you to sign a big CLA before merging your patch.

                                        It sometimes happens even for non-trivial patches and even if you sign CLA. Example: Unix domain sockets (UDS, AF_UNIX) in System.inheritedChannel() and elsewhere in Java. (later implemented internally).

                                        Nobody is obliged to accept your contribution (which is OK – on the other hand, nobody can force you to merge his code).

                                        1. 2

                                          If your report was just a small typo or a few words, it would be a little silly for them to ask you to sign a big CLA before merging your patch.

                                          Copyright rights in the U.S. can be terminated by the estate after the author dies. This is one of the reasons that most corporate open source projects require a CLA. Here is a post by a lawyer that describes other problems solved by a CLA.

                                        1. 3

                                          I would be interested in a good explanation of ECC and its variants like ECDSA. Sadly this article really didn’t give much of an explanation.

                                          1. 33

                                            They’re more popular than ever, as AWS Lambda.

                                            1. 27

                                              If I was good at writing satire, I would write a satire article that is a CGI scripting tutorial, but never use the word CGI and instead call it a “new Open Source function as a service library”.

                                              Similar twist: “Hi folks, I wrote a new library that converts JVM byte code to WebAssembly. For the first time ever, we can write Java that runs in the browser! Never before seen!”

                                              1. 11

                                                They are both stack machines, so it should be simple enough I guess. Don’t give me ideas.

                                                1. 5

                                                  Do it.

                                                2. 3

                                                  I started trying to write this article a while back, not as satire, but as a direct comparison to the evolution of serverless. But then I realized it’s probably been done better than I could do and aborted.

                                                  1. 7

                                                    I have literally never not published something just because I think it might have been done. If I see something that is almost entirely what I wrote, sure, I’ll axe it. (Even retroactively, in one case, where I read someone else with a better take and thought, “oh, never mind then.”) But if I haven’t specifically read an article of what I want to write, then:

                                                    1. I might have a unique take after all.
                                                    2. Even if my take isn’t unique, it might be different enough to be helpful to someone else.
                                                    3. Even if it’s neither unique nor different enough, if I’m not aware of it (and after deliberately cursory search can’t find it, if applicable), it will likely reach a different audience.

                                                    In the draft post you’ve got, I think you are heading in a good direction, and it might be worth continuing. I’d suggest dropping the FastCGI/SCGI/WSGI/Rack section in favor of diving a lot more into early attempts to speed up CGI requests and how those relate to lambdas (you touch on mod_perl, but I’d also at least touch on PHP in particular, and quite possibly AOLServer, as close peers), highlighting similar issues with startup time and how lambdas are trying to solve them in their own ways/differently.

                                                    The other way to approach this kind of thing, incidentally, which I like for my equivalent writings on these axes, is to walk through how trying to solve the problems with the old old-and-busted resulted in the new old-and-busted. You can write that kind of article sarcastically, but you definitely don’t have to; my article comparing Win32 to Flux has a bit of humor in it, but I deliberately avoided anything past that. If you went that route, the FastCGI/SCGI section fits better, but also pairs very nicely with talking about things like the Danga Interactive products (Gearman, memcache, Perlbal, etc.), which turn out to be necessarily reinvented whenever a PaaS-like environment is used.

                                                    Anyway, all this to say: I’d love to see someone actually write a post along these lines. If you really don’t want to finish yours, you’ve given me half a mind to take my own stab.

                                                  2. 3

                                                    I have that as a laptop sticker. I don’t know if commercial advertising on lobste.rs is appropriate (even for an enterprise which makes me on the order of $2/month) so I won’t link it directly, but you could probably find it quite easily by searching redbubble.com for “serverless cgi-bin”

                                                    There is the reasonable objection that using a FaaS platform you have the expectation that it will autoscale to performance levels far in excess of anything that cgi-bin of yore managed, but really, that’s a implementation detail not an attribute of the API

                                                    1. 2

                                                      Kelsey Hightower made a similar comparison at Gophercon: https://youtu.be/U7glyWYj4qg

                                                      1. 2

                                                        “This is xinetd… the new hotness”

                                                        Love it.

                                                  1. 4

                                                    I love the overall concept, but Scuttlebutt seems to completely ignore all existing protocols for RPC, serialization, LAN discovery, cluster gossip, message encoding, etc. It reinvents a whole lot of wheels all at once. “But it’s simple! It’s just JSON!” they might say. But the difficulty isn’t in the syntax, it’s in the semantics.

                                                    Unless I missed something, local sync requires visibility of peers’ UDP broadcasts on a shared LAN. Nearly all public hotspots and home guest LANs disable peer visibility, so this seems like a nonstarter. BLE beacons could be an interesting alternative.

                                                    I’d rather use a version of ActivityPub or Solid (https://solid.mit.edu) that works truly offline, such as with BLE + Wi-Fi Direct.

                                                    1. 15

                                                      As further rationale for avoiding master / slave, it’s not even a clear conceptual metaphor. Primary and secondary nodes flip roles all the time, but a human slave never becomes the master. Also there is/was routine and severe distrust, misunderstanding, abuse, and deceipt from a master to a slave, whereas in a database a better metaphor would be pilot/copilot. The pilot is by default in charge but through explicit protocols the copilot will “take the airplane” during flight.

                                                      1. 8

                                                        Not a great question pool IMHO Many of these questions are biased toward a person with a very specific personal profile. Asking about a “home lab” is disqualifying of people who don’t spend weekends terminating Cat5 or reimaging old Dell servers from eBay. A lot are outdated, e.g. the difference between a “router” and a “gateway” (I’ve seen that for 15+ years and it didn’t make sense then either). Others are uselessly vague, particularly “what is redirection?” with no context.

                                                        1. 1

                                                          The test ended up being an actual in terminal test with specific goals and full access to man pages and the internet. So it wasnt anything like these questions (though they were helpful to distract me the night before). It went pretty well I think!