1. 2

    This is why ExaBGP exists; there’s no need to run something as beastly as BIRD. It also includes powerful Watchdog capabilities where an external script/tool can be used to enable/disable advertising based on the health of other services.

    That said, it’s nice to see more people learning about this functionality. It’s really great.

    edit: ahh, 5880 / BFD is new enough that it’s not in ExaBGP yet. Then again I don’t remember ever needing it; our failovers were almost instantaneous with our watchdog designs and the loss of a single router wasn’t a concern.

    1. 2

      On the other hand, BIRD has fairly extensive documentation, meaning that it’s easy to figure out it’s capabilities and limitations, and how to do things in general, which cannot be said about ExaBGP.

      1. 1

        That was the opposite of what our Ops team found @ Cisco when I was there

    1. 16

      also annonymizes the request IPs using MD5 hashing

      Hashing an IPv4 address, which has a very small search space, especially with something as fast as MD5, is not anonymization. On modern hardware it wouldn’t take very long to generate a rainbow table for the entirety of the IPv4 address space, which you can then look up every IP in.

      1. 2

        Came here to say this. Common anonymization techniques involve replacing the last half of the IP address with nulls.

      1. 3

        Awesome read, thank you calvin and the author!

        I end-up using poll() like average kid, but coroutines and threads with explicit flow controls ala libtask are a good thing to explore.

        It had to have a few asm (maybe go portable with .S files? that worked for libtask and p9p and Go iirc).

        But… Does this work? Only the stack pointer is saved, and not the state of CPU registers:

        void print_4_is_the_thing(void) {
            int a = 4;
            task_relinquish();
            printf("%d\n", a);
        }
        

        Some more asm was required for https://github.com/jamwt/libtask/blob/master/asm.S

        But reading at setjmp()’s man page, it looks like not only it saves the registers state in jmp_buf, but also the stack pointer!

        The setjmp() function saves various information about the calling environment (typically, the stack pointer, the instruction pointer, possibly the values of other registers and the signal mask) in the buffer env for later use by longjmp(). In this case, setjmp() returns 0. – setjmp(3)

        So that means it is possible to get it running without the inline asm compiler extension and using just the POSIX behavior of setjmp()? I want to try it by simply removing the asm and see what it does… But it seems like it has a different role:

        This task has not been started yet. Assign a new stack pointer, run the task, and exit it at the end. – the blog article

        That would make it extremely portable, as every architecture supporting setjmp() and longjmp() would therefore support it.

        1. 2

          Some more asm was required for https://github.com/jamwt/libtask/blob/master/asm.S

          That’s because libtask chooses to save and restore the full context manually rather than using setjmp/longjmp or similar.

          Another one you might want to look at is libmill/libdill, which uses sigsetjmp/siglongjmp by default (presumably to make it singal safe), with a custom assembly version for x86_64 (and i386 for libdill).

          But reading at setjmp()’s man page, it looks like not only it saves the registers state in jmp_buf, but also the stack pointer!

          The setjmp() function saves various information about the calling environment (typically, the stack pointer, the instruction pointer, possibly the values of other registers and the signal mask) in the buffer env for later use by longjmp(). In this case, setjmp() returns 0. – setjmp(3)

          So that means it is possible to get it running without the inline asm compiler extension and using just the POSIX behavior of setjmp()?

          It saves it and then restores it on longjmp(3), yes, but the inline assembly isn’t doing either of those things. It’s setting the new stack pointer to the newly allocated stack, which after that call is the current stack pointer. You can’t have setjmp/longjmp save/restore the pointer to the task’s stack if it was never set to the new stack to begin with.

        1. 27

          It’s worth linking to A&A’s (a British ISP) response to this: https://www.aa.net.uk/etc/news/bgp-and-rpki/

          1. 16

            Our (Cloudflare’s) director of networking responded to that on Twitter: https://twitter.com/Jerome_UZ/status/1251511454403969026

            there’s a lot of nonsense in this post. First, blocking our route statically to avoid receiving inquiries from customers is a terrible approach to the problem. Secondly, using the pandemic as an excuse to do nothing, when precisely the Internet needs to be more secure than ever. And finally, saying it’s too complicated when a much larger network than them like GTT is deploying RPKI on their customers sessions as we speak. I’m baffled.

            (And a long heated debate followed that.)

            A&A’s response on the one hand made sense - they might have fewer staff available - but on the other hand RPKI isn’t new and Cloudflare has been pushing carriers towards it for over a year, and route leaks still happen.

            Personally as an A&A customer I was disappointed by their response, and even more so by their GM and the official Twitter account “liking” some very inflammatory remarks (“cloudflare are knobs” was one, I believe). Very unprofessional.

            1. 15

              Hmm… I do appreciate the point that route signing means a court can order routes to be shut down, in a way that wouldn’t have been as easy to enforce without RPKI.

              I think it’s essentially true that this is CloudFlare pushing its own solution, which may not be the best. I admire the strategy of making a grassroots appeal, but I wonder how many people participating in it realize that it’s coming from a corporation which cannot be called a neutral party?

              I very much believe that some form of security enhancement to BGP is necessary, but I worry a lot about a trend I see towards the Internet becoming fragmented by country, and I’m not sure it’s in the best interests of humanity to build a technology that accelerates that trend. I would like to understand more about RPKI, what it implies for those concerns, and what alternatives might be possible. Something this important should be a matter of public debate; it shouldn’t just be decided by one company aggressively pushing its solution.

              1. 4

                This has been my problem with a few other instances of corporate messaging. Cloudflare and Google are giant players that control vast swathes of the internet, and they should be looked at with some suspicion when they pose as simply supporting consumers.

                1. 2

                  Yes. That is correct, trust needs to be earned. During the years I worked on privacy at Google, I liked to remind my colleagues of this. It’s easy to forget it when you’re inside an organization like that, and surrounded by people who share not only your background knowledge but also your biases.

              2. 9

                While the timing might not have been the best, I would overall be on Cloudflare’s side on this. When would the right time to release this be? If Cloudflare had waited another 6-12 months, I would expect them to release a pretty much identical response then as well. And I seriously doubt that their actual actions and their associated risks would actually be different.

                And as ISPs keep showing over and over, statements like “we do plan to implement RPKI, with caution, but have no ETA yet” all too often mean that nothing will every happen without efforts like what Cloudflare is doing here.


                Additionally,

                If we simply filtered invalid routes that we get from transit it is too late and the route is blocked. This is marginally better than routing to somewhere else (some attacker) but it still means a black hole in the Internet. So we need our transit providers sending only valid routes, and if they are doing that we suddenly need to do very little.

                Is some really suspicious reasoning to me. I would say that black hole routing the bogus networks is in every instance significantly rather than marginally better than just hoping that someone reports it to them so that they can then resolve it manually.

                Their transit providers should certainly be better at this, but that doesn’t remove any responsibility from the ISPs. Mistakes will always happen, which is why we need defense in depth.

                1. 6

                  Their argument is a bit weak in my personal opinion. The reason in isolation makes sense: We want to uphold network reliability during a time when folks need internet access the most. I don’t think anyone can argue with that; we all want that!

                  However they use it to excuse not doing anything, where they are actually in a situation where not implementing RPKI and implementing RPKI can both reduce network reliability.

                  If you DO NOT implement RPKI, you allow route leaks to continue happening and reduce the reliability of other networks and maybe yours.

                  If you DO implement RPKI, sure there is a risk that something goes wrong during the change/rollout of RPKI and network reliability suffers.

                  So, with all things being equal, I would chose to implement RPKI, because at least with that option I would have greater control over whether or not the network will be reliable. Whereas in the situation of NOT implementing, you’re just subject to everyone else’s misconfigured routers.

                  Disclosure: Current Cloudflare employee/engineer, but opinions are my own, not employers; also not a network engineer, hopefully my comment does not have any glaring ignorance.

                  1. 4

                    Agreed. A&A does have a point regarding Cloudflare’s argumentum in terrorem, especially the name and shame “strategy” via their website as well as twitter. Personally, I think is is a dick move. This is the kind of stuff you get as a result:

                    This website shows that @VodafoneUK are still using a very old routing method called Border Gateway Protocol (BGP). Possible many other ISP’s in the UK are doing the same.

                    1. 1

                      I’m sure the team would be happy to take feedback on better wording.

                      The website is open sourced: https://github.com/cloudflare/isbgpsafeyet.com

                      1. 1

                        The website is open sourced: […]

                        There’s no open source license in sight so no, it is not open sourced. You, like many other people confuse and/or conflate anything being made available on GitHub as being open source. This is not the case - without an associated license (and please don’t use a viral one - we’ve got enough of that already!), the code posted there doesn’t automatically become public domain. As it stands, we can see the code, and that’s that!

                        1. 7

                          There’s no open source license in sight so no, it is not open sourced.

                          This is probably a genuine mistake. We never make projects open until they’ve been vetted and appropriately licensed. I’ll raise that internally.

                          You, like many other people confuse and/or conflate anything being made available on GitHub as being open source.

                          You are aggressively assuming malice or stupidity. Please don’t do that. I am quite sure this is just a mistake nevertheless I will ask internally.

                          1. 1

                            There’s no open source license in sight so no, it is not open sourced.

                            This is probably a genuine mistake. We never make projects open until they’ve been vetted and appropriately licensed.

                            I don’t care either way - not everything has to be open source everywhere, i.e. a website. I was merely stating a fact - nothing else.

                            You are aggressively […]

                            Not sure why you would assume that.

                            […] assuming malice or stupidity.

                            Neither - ignorance at most. Again, this is purely statement of a fact - no more, no less. Most people know very little about open source and/or nothing about licenses. Otherwise, GitHub would not have bother creating https://choosealicense.com/ - which itself doesn’t help the situation much.

                          2. 1

                            It’s true that there’s no license so it’s not technically open-source. That being said I think @jamesog’s overall point is still valid: they do seem to be accepting pull requests, so they may well be happy to take feedback on the wording.

                            Edit: actually, it looks like they list the license as MIT in their package.json. Although given that there’s also a CloudFlare copyright embedded in the index.html, I’m not quite sure what to make of it.

                            1. -1

                              If part of your (dis)service is to publically name and shame ISPs, then I very much doubt it.

                    2. 2

                      While I think that this is ultimately a shit response, I’d like to see a more well wrought criticism about the centralized signing authority that they mentioned briefly in this article. I’m trying to find more, but I’m not entirely sure of the best places to look given my relative naïvete of BGP.

                      1. 4

                        So as a short recap, IANA is the top level organization that oversees the assignment of e.g. IP addresses. IANA then delegates large IP blocks to the five Regional Internet Registries, AFRINIC, APNIC, ARIN, LACNIC, and RIPE NCC. These RIRs then further assigns IP blocks to LIRs, which in most cases are the “end users” of those IP blocks.

                        Each of those RIRs maintain an RPKI root certificate. These root certificates are then used to issue certificates to LIRs that specify which IPs and ASNs that LIR is allowed to manage routes for. Those LIR certificates are then used to sign statements that specify which ASNs are allowed to announce routes for the IPs that the LIR manages.

                        So their stated worry is then that the government in the country in which the RIR is based might order the RIR to revoke a LIR’s RPKI certificate.


                        This might be a valid concern, but if it is actually plausible, wouldn’t that same government already be using the same strategy to get the RIR to just revoke the IP block assignment for the LIR, and then compel the relevant ISPs to black hole route it?

                        And if anything this feels even more likely to happen, and be more legally viable, since it could target a specific IP assignment, whereas revoking the RPKI certificate would make the RoAs of all of the LIRs IP blocks invalid.

                        1. 1

                          Thanks for the explanation! That helps a ton to clear things up for me, and I see how it’s not so much a valid concern.

                      2. 1

                        I get a ‘success’ message using AAISP - did something change?

                        1. 1

                          They are explicitly dropping the Cloudflare route that is being checked.

                      1. 1

                        Nice TODOs. ;)

                        1. 7

                          The twitter thread for this was the first time I heard of you, Hillel (I think that you also proposed other programs to be proven, in addition to leftpad? Edit: There it is!). Since then I’ve been reading many of your posts.

                          I recently tried to do a pen-and-paper proof of a bubble sort, and see if and how this could be formalized. The context is “Suppose you’d made a programming language in which proving programs correct is as close as possible to what a mathematician would do on paper”. My experiences:

                          • Mathematics uses a lot of notation that is not easy to write down in ASCII (quantifiers, operators, arrows, …).
                          • You need to ‘encode’ programming language concepts in math. For example, I choose to encode an int array as a function from the natural numbers to the integers.
                          • Some things are hard to formally express. For example, you can specify the requirements for a sorting function as ‘output contains the same elements as the input, but sorted’. The sorted part is relatively easy to express mathematically, but ‘contains the same elements’ is harder.

                          I choose to use functions to model arrays. An int array with n elements becomes a function from { 0, 1, …, n - 1 } to the integers. Now, we can phrase the requirement for a sorting function that the output function/array equals the composition of a permutation of { 0, 1, …, n - 1 } and the input function. Now you need some theory on permutations (most relevant: if you swap two elements in a permutation, it’s still a permutation).

                          On top of that, you also need some axioms for integers. With these techniques and with a lot of loop invariants, it’s possible to prove the correctness of bubble sort. Then you realize you’ll also need to prove termination…

                          Nothing in these steps is prohibitively hard, but it’s a lot of work and quite a rabbit hole!

                          1. 2

                            Some things are hard to formally express. For example, you can specify the requirements for a sorting function as ‘output contains the same elements as the input, but sorted’. The sorted part is relatively easy to express mathematically, but ‘contains the same elements’ is harder.

                            Just to throw it out there since I got nerdsniped, are the following criteria not enough to express “contains the same elements”?

                            • The input and the output contain the same number of elements.
                            • For all elements in the input, the element is present in the output.

                            Maybe one or both of those criteria is itself hard to express.

                            1. 2

                              If the input contains duplicates, the output could contain another element and still fulfil those requirements.

                              You could hane it apply to both in->out and out->in. but but then the same thing could happen if there’s more than one element that has duplicates.

                              1. 1

                                Ah! Yep, I see it now. Thanks!

                              2. 2

                                @kyrias pointed out the issue, but you can make this valid by instead saying “every element of the input has the same count in the output.” In fact when I teach workshops I have students specify sorting both ways, as a permutation spec and as a counter spec.

                            1. 5

                              It’s got to be a keyword, that’s how it can work with is not, not also being a keyword.

                              Also there’s a quite ubiquitous use of is that justifies its inclusion; compare a is None to a == None to not a… only by testing the identity can you know for certain that None was passed.

                              1. 2

                                You put that very clearly, here and in the other comments; I learned something today.

                                One other ubiquitous use of is: type inspection, like if type(a) is MyClass or if type(a) is str.

                                (Some of the time isinstance(a, MyClass) will also do, but if you want to exclude the possibility of subclass instances, only a type identity check suffices.

                                Ooonn the other hand, one could also argue that is tempts people into checking for identity when a subclass would also suffice; and that this may needlessly prevent interoperation with otherwise-valid subclasses. Hm. I really like the keyword, though, especially compared to ===)

                                1. 5

                                  Note that you don’t normally need is for this sort of type inspection, as type(a) == MyClass works fine pretty much always. I think the only time it wouldn’t work correctly is if one of the classes involved had a metaclass that overrode __eq__ and did something perverse with it. I think that the cases where you really need is are uncommon; you need to care about object identity and be in a situation where something is overriding == in a way that affects the comparison you care about. Python definitely should have something that checks object identity in the way that is does, but I don’t know if it needs to be a tempting keyword instead of a built-in function.

                                  (I’m the author of the linked to article.)

                                  1. 2

                                    Every language must have a means of checking both identity and equality … they are fundamentally different things. The choice to make both of them an operator is quite common, the only real distinction Python makes by choosing == and is over, say, == and === is that promotion to also being a keyword.

                                    And just because you find it to be uncommon does not mean that it is, in fact, uncommon. I use is all the time, for instance with enum.Enum members, to properly handle default values that need to be mutable, and to identify sentinel values in iterators (usually None, but not if None is a legitimate value of the iterator) … it’s also extremely necessary to things like the Singleton pattern in Python.

                                    Moreover you’re throwing out the baby with the bathwater … sure, exactly as you said you can use type(a) == MyClass pretty much anywhere you might use type(a) is MyClass, but why would you? Why necessarily invoke the performance cost of equality comparison? The underlying cost of is is near zero, but by using == you force lookup and execution of MyClass.__eq__ and, if that returns False or NotImplemented you further force lookup and execution of type(a).__eq__ … both of which could be implemented in such a way as to be unbearably time consuming.

                                    See the question of identity is, precisely, “are these the same thing” rather than “do these share the same attributes” … they’re fundamentally different questions, and when all you care about is identity, why bother asking for equality?

                                2. 2

                                  Arguing that it has to be a keyword because otherwise you would have to write it differently is a weird argument to me. It just means that it would be written not is(a, b) instead, which might not read as nicely, but that’s a different argument.

                                  1. 7

                                    Perhaps I should have been more specific: is needs to be either a reserved keyword or an operator (in effect it’s both, though it’s not truly an operator because, like and, or, and not it cannot be overloaded) precisely because Guido et all intended it to be used as a syntactic construct… it cannot be a function because it would then have to be used with the call syntax, be able to be assigned to a name, and overloadable in user code. There is no desire for an overloadable identity operation, indeed allowing it to be overloaded would break fundamental language semantics. The same is true of None, which is why it’s also a keyword, though it could just be a globally available variable… in Python the argument for promoting to an operator is “should this bind values around it into an operation” and for promotion to a keyword is “should the meaning of this never even potentially change during runtime” and (optionally) “should this modify other keywords” … in the case of is the answer to all of those is “yes”, so the only option was to make it a keyword.

                                    1. 1

                                      It seems to me like the same arguments would mean that isinstance should also be a keyword though?

                                      1. 3

                                        Sure, it could be extended to any number of things — Python 2 didn’t see True or False as keywords, but that protection was extended to them in Python 3 — but would it add any semantic value to do so, by stripping call semantics from it and allowing it to be used as an operator or to directly interact with other keywords?

                                        Some keywords have been downgraded (most notably print), but that was because there was added value in doing so, and also a colorable argument for overloading existed. The simple fact is that identity testing is a fundamental operation — much more so than introspecting class phylogeny — and the benefits of being a keyword far outweigh the minor confusion it can cause in relative newbies, while there’d be no advantage to removing it except for people who don’t know why it’s fundamental.

                                        In other languages you’ve got equality as == and identity as ===… frankly we’ve got it easy with is.

                                    2. 2

                                      is being built in means you know it can’t be overridden.

                                      It being a function would be ugly

                                  1. 4

                                    It took Y half a year to get the web site up again, and finally all those dead links stopped screaming.

                                    Not only that, most parts of it never came back again at all.

                                    1. 10

                                      What3Words are pretty weird, I’d prefer them not to succeed in the marketplace.

                                      That said, I’m speculating they have a solid DMCA case here. I have not seen the code in question, but I figure W3W does geohashing by translating lat/long into 3 unique word tuples using a lookup table. This table has been chosen to be clear and unambiguous, and that’s the relevant IP (the hashing algo too, I imagine).

                                      If WhatFreeWords had substituted their own lookup table, I doubt this would have been any problem.

                                      I’d rather a competing standard was developed and hosted by OpenStreetMap, instead of trying to reverse-engineer some company that’s legally trigger-happy.

                                      1. 10

                                        This table has been chosen to be clear and unambiguous

                                        You would think so, but it’s not the case. An article about What3Words listed reporter.smoked.received as the address of the Stade de France. But if you search reporter.smoked.received on What3Words, you will see it’s in Laredo, Missouri. Reporter.smoker.received, just one letter’s difference, is the Stade. W3W is not just a badly-licensed land grab, it’s also a terrible product!

                                        1. 4

                                          That’s intentional, the idea being that you know roughly where the identifier is meant to point to, so you would realize that you have the wrong one if it’s on a different continent. (See https://support.what3words.com/en/articles/2212868-why-are-the-words-randomly-assigned-wouldn-t-it-be-better-if-there-was-some-logic-hierarchy-structure-to-the-naming-structure)

                                          1. 2

                                            Thanks for the link. But it doesn’t really detract from my (admittedly wild-ass guess) legal theory.

                                            The fact whether the words are actually unambiguous is not relevant - the point is that W3W can argue that they have made significant effort in choosing the words and that the lookup table is proprietary information, covered by DMCA protections.

                                            (By the way, the article has been corrected to note that the address for Stade de France is the smoker variant)

                                            1. 1

                                              They don’t really argue that, though. As kyrias points out in their link the words are chosen at random. This seems like incredibly thin ice for copyright purposes, almost straying into conceptual art territory.

                                              The W3W promise is three words to define a location. The hedge is three words, plus a general idea of where the location is so you know if there’s an error. The reality is three words plus an unknown probability of an error and no way to correct it. Real addresses feature hierarchy and redundancy for this reason.

                                              1. 3

                                                Since my comment I’ve been reading up a bit. Here’s a purported DMCA takedown notice (not neccessarily the exact one served for WhatFreeWords) that makes the same argument I surmised:

                                                A key feature of W3W’s activities is its creation of unique three-word addresses (“3WAs”) for three-metre by three-metre “squares” of the Earth’s surface. […] 3WAs are generated through the use of software code and approximately 57,000,000,000 word combinations made from a wordlist of over 40,000 words compiled by and proprietary to W3W.

                                          2. 7

                                            I’d rather a competing standard was developed and hosted by OpenStreetMap, instead of trying to reverse-engineer some company that’s legally trigger-happy.

                                            Plus codes come close I think; no need for yet another standard. Granted they’re not pronounceable words, but What3Words also needs homonym disambiguation so I’m not sure much is lost.

                                            See also: Evaluation of Location Encoding Systems.

                                            1. 2

                                              I had plus codes in the back of my mind when I wrote my comment, but believed they weren’t fully open. Thanks for informing me.

                                              Edit - the HN discussion on this submission is quite interesting. Among others, yet another competing geo-phrase service offers this critique of OLC/Plus Codes: https://www.qalocate.com/whats-wrong-with-open-location-code/

                                              1. 1

                                                not pronounceable words

                                                I’m not convinced that English pronounceability is really such a killer feature for something that’s supposed to be usable worldwide. But that’s neither here nor there. Thanks for the links!

                                            1. 18

                                              Applications don’t manage the network

                                              This entire paragraph reeks of “King of the Castle” attitude. 5 years ago I interacted with a sysadmin who had the same arguments for why every network device should go through his local HTTP proxy.

                                              Just imagine the pure mess if you get different DNS results in different applications.

                                              Well, what does the author imagine will happen? I don’t know. I don’t think it matters if two apps resolve to different IPs. What would happen if two devices resolved some hostnames to different IPs? Probably the same thing: Nobody cares.

                                              btw Firefox caches DNS results independently of the OS, and I believe so does Chrome. These out of sync issues already happen and they are not a big deal.

                                              The chaos will be a perfect Trump made Internet.

                                              Oh yeah, that’s how you connect Mozilla to Trump. Nicely done. You just had to taint your best argument (US govt can’t be trusted) with that line.

                                              I expected nothing based on the title and was still disappointed.

                                              1. 9

                                                I don’t think it matters if two apps resolve to different IPs. What would happen if two devices resolved some hostnames to different IPs? Probably the same thing: Nobody cares.

                                                As someone who semi-regularly has to debug connectivity issues people have with various services, it’s rather annoying not to be able to get the same DNS results when manually resolving addresses using the OS settings, especially if one doesn’t know about the fact that Firefox has started doing this.

                                                1. 4

                                                  Yes, it has annoyed me too. DoH is not introducing this problem though.

                                                  1. 7

                                                    It does introduce the problem though. Problems related to application level DNS caching are easily bypassed by just having them restart the browser. But if your browser claims to not be able to resolve a name, or resolves it to something different due to split-horizon DNS, and everything else on the machine is able to resolve the name properly, how would you debug this?

                                                    (So according to https://blog.mozilla.org/futurereleases/2019/09/06/whats-next-in-making-dns-over-https-the-default/ it should fall back to the OS resolver on failures, so at least ones that don’t exist externally should work, though that doesn’t really help when the public address resolve differently.)

                                                    1. 7

                                                      Pertinent example:

                                                      (11:42:26) om:~% dig +short @8.8.8.8 archive.is
                                                      62.192.168.106
                                                      (11:42:35) om:~% dig +short @9.9.9.9 archive.is
                                                      51.15.97.128
                                                      (11:42:43) om:~% dig +short @1.1.1.1 archive.is
                                                      127.0.0.3
                                                      

                                                      uh oh!

                                                      1. 3

                                                        Apparently archive.is is refusing to respond to 1.1.1.1: https://twitter.com/archiveis/status/1018691421182791680

                                                2. 3

                                                  Oh yeah, that’s how you connect Mozilla to Trump. Nicely done. You just had to taint your best argument (US govt can’t be trusted) with that line.

                                                  Exactly! This article felt nothing more than a scare tactic due to the author having un-moving opinions on the matter, resulting in him using points that aren’t even accurate. Author’s need to learn to have more perspective, share their own perspective, and back their sources

                                                1. 5

                                                  What I appreciate most of all is that nobody apparently thought about how to design USB-C plugs so they didn’t slide out.

                                                  1. 3

                                                    Sweet baby Jesus why would you want that? Personally I’m annoyed at how difficult it is to pull out a USB-C compared to the (now) old-fashioned MagSafe. When it’s finally time to replace the wife’s old laptop with whatever’s current at the time, I fear for its life.

                                                    1. 2

                                                      I loved MagSafe connectors and thought them up years before Apple introduced them: magnets get rid of mechanical wear-and-tear while making it easier to plug in! I’m guessing they weren’t used in USB-3 because of the connector size and magnetic interference.

                                                    2. 2

                                                      It seems to me that they did. My regular phone charger slides out way too easily, but my laptop charger (when either plugged into my laptop, or my phone) is quite good at staying in until I try and pull it out.

                                                      1. 3

                                                        Have you checked your phone’s usb-c port and mauybe tried cleaning it with a toothpick? :)

                                                        Not sure this is intentional, but with my Nexus 5X the lint seems to set in such a way that the usb-c cable slides out with the slightest touch once enough has accumulated. The connection is never broken so you’d notice, just the mechanical “lock”.

                                                      2. 2

                                                        I’ve personally always experienced that issue more often with micro-A than I have with any of my devices with C.

                                                        1. 1

                                                          I’ve been so annoyed by this that I’m pondering whether USB-C cables can be used for electronics which don’t get a gentle treatment all of the time (badges, smaller electronic cards, …).

                                                        1. 3

                                                          Their latest reply also feels rather Google:

                                                          If there can be a C++ standard library and runtime implementation as part of the LLVM project, I do not see a reason why there cannot be a libc implementation as part of the LLVM project.

                                                          Of course there can be an LLVM libc, just as there can be an LLVM anything. The question is should, which needs to be justified with sound whys, none of which have been given so far.

                                                          1. 3

                                                            I haven’t thought in detail about what should be public versus what the compiler insists has to be public due to how I’m using these modules. I think for a command-line application it doesn’t matter so much what my public API is because no one will be consuming it, but for a library it’s probably important. I will come back to these considerations later and see if I can understand how to manage separation of concerns in rust in more detail.

                                                            The simplest way to manage it is by just slapping pub(crate) on everything that you aren’t 100% sure you want exposed. It makes the item public to the whole crate, but public to other crates.

                                                            1. 1

                                                              Is there anything particular to Mongo in this situation, or could it have happened to any unsecured DB?

                                                              1. 3

                                                                IIRC mongo doesn’t have authentication by default, and older versions were exposed to the internet by default.

                                                                1. 1

                                                                  Looks like a normal case of poor choices by someone acting as sysadmin.

                                                                  1. 3

                                                                    The phrasing of the article (“important measures which will prevent the attacks are to enable authentication and to not allow the databases to be remotely accessible”) seems to suggest that by default MongoDB instances do not have any authentication and are remotely accessible. I can’t grep it from the docs right now, but this 2016 SO answer verifies.

                                                                    I just tested it, and it does seem to bind to 127.0.0.1 by default, but perhaps that’s distro-specific or the default changed.

                                                                    At any rate, not enabling authentication is a poor default. Not everyone running a MongoDB instance is a MongoDB expert and if the entire security of your installation is dependent on not missing a “warning: please enable authentication” in some documentation then you’re just doing it wrong.

                                                                    People don’t read the docs, or they don’t read the right ones, or they read the right ones, get asked something by a coworker, and continue reading from a different part (skipping the warning), etc. etc.

                                                                    It doesn’t hurt enabling it by default, and it will prevent a world of hurt for everyone involved. We’ve seen this before time and time again going back to (at least) SMTP servers running as open relays by default.

                                                                    Replying to your other comment:

                                                                    Here is the question. How do software people make good defaults when those defaults may make it harder to use and therefore adopt the software?

                                                                    I don’t think enabling authentication makes software that much harder to use. It’s essentially just one command to change a password. Having your MongoDB installation hijacked also doesn’t make it easier to use ;-)

                                                                    1. 2

                                                                      Having your MongoDB installation hijacked also doesn’t make it easier to use ;-)

                                                                      Arguably it makes it easier to use for everyone. ;)

                                                                1. 6

                                                                  We’re currently transitioning to https://notion.so as our single source of truth on these things. It’s not perfect, but it’s discovery and collaboration is must better than Confluence or GDocs, from experience. This is from the perspective of a small engineering team, though.

                                                                  1. 1

                                                                    That looks somewhat neat, but it looks like it has no Linux support and no web interface?

                                                                    1. 3

                                                                      Ah, I can see how you would get that impression from the landing page, but it does in fact have a web interface and I mostly just use it from the browser so that should solve the Linux support issue as well.

                                                                      1. 2

                                                                        It’s fully usable through a web browser, but there’s no Linux client, no.

                                                                        1. 1

                                                                          The native clients are basically web views. The web interface is what I use most of the time.

                                                                          1. 1

                                                                            It’s definitely web based and in browser, the desktop apps are just Electron wrappers. I think there may be an unofficial Linux Electron wrapper too (if you are willing to add to your collection)

                                                                          2. 1

                                                                            Thanks for the link. Definitely going to check it out!

                                                                          1. 14

                                                                            Yay! It’s been great to have you on the team!

                                                                            In my second week I implemented a change to make a certain pattern more ergonomic. It was refreshing to be able to build the initial functionality and then make a project wide change, confident that given it compiled after the change I probably hadn’t broken anything. I don’t think I would have had the confidence to make such a change as early on in the Ruby projects I’ve worked on previously.

                                                                            Yeah, this is what made me consistently frustrated and annoyed at my previous work place (they used Ruby, Elixir, and JS). Static types are a bit of a trade off, and not perfect, but I’m thankful to finally be able to use them in my day job.

                                                                            1. 11

                                                                              I think the same problem exists even in statically typed languages. Java people talk about the open-close principle all the time, but I think it terribly encourages overengineering and bloat (due to append-only programming). What you really want is a language (and a codebase that isn’t trying hard to thwart it) that gives you the confidence to change anything as the requirements change.

                                                                              1. 4

                                                                                Sandi Metz hammers this home. I can’t recommend POODR and every talk she gives on youtube enough. It’s all about the cost of maintaining the code.

                                                                                1. 3

                                                                                  I hadn’t heard about Sandi Metz as a non-Rubyist, so went ahead and listened to some of her talks. Her messages are exactly what I would’ve applauded before I met Haskell and functional programming. When you have a very powerful type system behind your back, you’re not afraid of writing code that mixes abstraction levels. You’re not afraid of hard-coding that configuration value, because when you need it, the time it takes to refactor it out to a configuration file is exactly as much as it is now, and the risk is zero in either case. So, in the long term, you end up with a simpler codebase, because you didn’t introduce needless complexity fearing that you might need it later (that’s a very valid concern in Ruby or Java.).

                                                                                  The principles I recommend to junior Haskellers are much more simple.

                                                                                  1. Stick to DRY as much as possible
                                                                                  2. Make sure your non-local assumptions are reflected in the types (purity is a side-effect of that principle)
                                                                                  3. Don’t work around abstractions, go in and refactor them
                                                                                  1. 2

                                                                                    Yeah, as I said in another comment, I think this is why Prince is such a nice code base, even after 16 years. Although Rust isn’t pure like Mercury or Haskell, it still seems to provide similar benefits as well.

                                                                                  2. 1

                                                                                    Yeah, Sandi Metz’s stuff is really great from what I’ve seen!

                                                                                  3. 2

                                                                                    Aka The Smalltalk Advantage

                                                                                    1. 1

                                                                                      I think it’s fine to have to change code when requirements change, so I’m less strict about following open-close stuff from the Java world. What I’m looking for is tooling that helps me know what to change and where - that’s something that can really help when requirements change underneath you. I definitely find that type systems in Rust, Haskell, OCaml, etc to give that power to refactor in a far more ruthless way than Java’s does - although I find Java a step up from say, Ruby, where there tends to be much fear around changing things, even in with a good test suite in place.

                                                                                    2. 1

                                                                                      Would unit tests help in this case? I’ve regularly had something compile and later segfault , (not sure if that’s possible with rust), whereas often the tests caught that.

                                                                                      1. 3

                                                                                        Barring compiler bugs Rust’s memory model guarantees no segfaults in “safe Rust”. Use of the unsafe keyword allows dereferencing of raw pointers so if you get that wrong you could get a segfault. Our code only uses unsafe for FFI with the existing application so it’s possible to get that wrong but when writing and testing the Rust in isolation we’ve written no unsafe (although the standard library does make use of it).

                                                                                        1. 3

                                                                                          Barring compiler bugs Rust’s memory model guarantees no segfaults in “safe Rust”.

                                                                                          That’s not actually true, stack overflows are for instance not exactly hard to cause. Segfaults just don’t violate memory safety.

                                                                                        2. 2

                                                                                          You can still get panics and stuff in safe Rust, and there are plenty of things that you definitely need tests for. You don’t need to worry about segfaults if you’re in safe code though (barring bugs in Rust and the standard library).

                                                                                          The change in question was more an API change - something that would be tedious and error-prone to do a find and replace for when using Ruby. The type system can do a pretty good job of telling you where you need to change things at the the exact place where you need to make that change.

                                                                                      1. 3

                                                                                        A nice little writeup, though the mind boggles why they want to bring in Redis when the tooling and ecosystem already exist in the Erlang/OTP.

                                                                                        1. 2

                                                                                          Seems like they’re planning on moving away from Redis later:

                                                                                          With Elixir is also possible to get rid of this component, purely relying on the communication between Elixir nodes (we will see this in the next article).

                                                                                          Still seems a tad strange to go with Redis first. The main reason I could see for doing this is if they want to be able to interoperate with non-Erlang systems, but since it uses term_to_binary that won’t actually be the case.

                                                                                        1. 2

                                                                                          Would be interesting to know when these slides were written, to put it in perspective.

                                                                                          Edit: Seems the talk took place on November 6th.

                                                                                          1. 2

                                                                                            Where is the source code? I could not find any. Yes, there is a link to an article that talks about some server software used at ground stations. A Google search of the NASA site does not return anything obvious.

                                                                                            1. 1

                                                                                              It’s not the actual Mars Rover they open sourced, it’s a “scaled down version of the design”, based on the Raspberry PI. You can find the actual design and software here. (And here is the official website of the Open Source Rover.)

                                                                                              1. 1

                                                                                                Or, put differently, tarballs or it didn’t happen.

                                                                                              1. 14

                                                                                                Pretty good article overall, but a few things to fix:

                                                                                                • Please spell UTF-8 and UTF-16 consistently with a hyphen.
                                                                                                • Each plane has 65536 code points, not 65535.
                                                                                                • Rust strings do not support rune indexing; they only support byte indexing. However, the str.chars() method returns an iterator that yields each rune sequentially. Moreover, this iterator can be collected into a Vec<char> which is in UTF-32.
                                                                                                • CJK characters are 3 bytes characters long in UTF-8, not 2 bytes as you stated. This is because the 2-byte limit is U+07FF, and all CJK characters are above U+3000.
                                                                                                • You mention that UTF-8 is a variable-length encoding by design, but fail to acknowledge that UTF-16 is variable-length too.
                                                                                                • Python 2 is a can of worms (which depends on compile-time flags), but Python 3 is not. In Python 3, all strings behave as if they are in UTF-32 (i.e. direct access to every code point). But internally, it can store strings as UTF-32, UTF-16, or UTF-8, depending on the actual content of each string (e.g. a pure ASCII string can be stored as UTF-8 and still allow fast O(1) indexing).
                                                                                                1. 6

                                                                                                  Rust strings do not support rune indexing; they only support byte indexing. However, the str.chars() method returns an iterator that yields each rune sequentially. Moreover, this iterator can be collected into a Vec which is in UTF-32.

                                                                                                  Stringactually doesn’t support byte indexing either, it just has range indexing to get a slice of the string, but you can use the as_bytes method to get a slice of bytes instead.

                                                                                                  1. 3

                                                                                                    This is correct and this is a really important point I think for Rust string indexing, I’ll modify the post to reflect this.

                                                                                                  2. 3

                                                                                                    You mention that UTF-8 is a variable-length encoding by design, but fail to acknowledge that UTF-16 is variable-length too.

                                                                                                    I don’t understand this criticism. The entire article is about the special case where UTF-16 is variable length - the case where you need to use surrogate pairs to represent a single unicode code point.

                                                                                                    1. 2

                                                                                                      Thanks for the feedback! I’m really glad the article initiated some interesting comments here. There are a few points which need clarifying and correcting which I’ll do in an update.

                                                                                                      You’re right about the Rust string indexing requiring .chars() iterator, I think this is a very important point so I’ll edit the post to explain this.

                                                                                                      I tried to point out how UTF-16 isn’t fixed length but I definitely think it could be made more clear.