1. 109

    Thanks for your efforts!

    After four links, a domain can’t have a majority of its stories submitted from a single user.

    As a datapoint, I currently can’t submit stories from my domain as I’ve submitted 14 (9 by others). I’m probably biased, but most stories have been reasonably well received so I’d consider that a loss.

    1. 46

      A simple tweak to this rule: bumping against the ceiling makes new submissions from that domain require mod approval. If posts are consistently well-received, mods can whitelist that (UserName, Domain) pair?

      1. 9

        I like this idea! If this is too much moderation overhead, maybe users with sufficiently high karma could see this queue and add approvals?

        1. 11

          Maybe. I dunno. I just threw it out there, but concerns around mod overreach and cabals of power-users are as old as time.

          Tying site privileges to karma creates all sorts of Goodhart’s-law-shaped problems.

          1. 3

            Yeah, but maybe the same trust system that lobsters already has would work here: namely, a mod can delegate this queue to a user they trust? It’s all highly transparent anyway so abuse could be punished?

            1. 2

              A hidden, secondary confidence score that is calculated based on outcomes that are subjectively chosen is where pushcx may be heading with this in due time. Putting a number to it might be a good idea.

        2. 37

          As a datapoint, you are not alone. I wrote:

          in the meantime bumping up against this limit posts a note to moderators so if it goes wrong we’ll see problems

          This definitely went wrong.

          My apologies to those inconvenienced by it, there’s a lot more false positives than I recognized. We’ve had a couple suggestions on how to reduce the error rate like only looking at the last N months or skipping it if any of the domain’s stories have done especially well (better than average or median?). I especially appreciate the folks writing and tweaking queries to try to build up our understanding, and I expect there’s probably some novel angle to separate noise from signal that we’ll think of in the next few days.

          1. 10

            There’s a “homepage” link in the profile. Perhaps the limit could be increased for your declared domain, (possibly, only if it’s unique across users?)

            1. 4

              This is a good idea, but what if the user is submitting from two blogs? For example, their personal blog and the blog of a project (perhaps a project the user contributes to) that the Lobsters community might be interested in.

              1. 8

                We have an authored by checkmark, that might work?

                1. 2

                  How many people are doing that? I think it may be acceptable collateral damage.

                  1. 1

                    Aren’t hats available for that purpose?

                    1. 2

                      Hats can’t be attached to posts… yet? Also, hats are generally used more for speaking on behalf/with significant involvement for more major projects, less associating sites to users. I suppose it can be changed…

                      1. 1

                        To clarify, are you suggesting that hats be used as a signal for increasing the (proposed) limit as to how many times a user can submit stories from a particular domain?

                        1. 1

                          No, but to have people make it clear that they are posting personal or project related. A separate limit per hat would be an idea yes for the post limit.

                  2. 2

                    Perhaps rate limiting posts rather than an absolute limit (or some combination of trust - whatever that means, account lifespan, etc to generate a score/threshold coupled with rate limits).

                  3. 35

                    Yes, this rule doesn’t really make sense to me. Users who write good stories will most likely be punished in this category.

                    1. 25

                      yes, I came to the comment section to ask specifically how to handle posting entries for our own blog posts. I enjoy blogging and this is one of the few places I share my blog posts. Don’t how to handle this now.

                      1. 5

                        So, it is mostly me posting my own stories as can be seen in https://lobste.rs/domain/andregarzia.com

                        1. 4

                          Yeah. I don’t blog about stuff as much as I should and lobsters is one of the good signal to noise places I’d wanna share with.

                        2. 17

                          Looking at @arp242 submissions, they look relevant and interesting, so I agree it seems to be a problem with the new algorithm. It will reduce the amount of interesting niche content - precisely what Lobste.rs should be about.

                          I’m probably in the same boat as @arp242 as I submit posts from my domain. One of my submissions is a book announcement with 26 upvotes, and the other five are Elm and Postgres posts and projects, which are neither low-effort nor frequent (this is over two years). I agree with @akkartik’s comment that the timeframe needs to be taken into account too.

                          I was going to suggest that the problem could be addressed by checking whether the user submitted other sites or participated in discussions, with an additional check for community approval in the form of a certain number of upvotes across submissions/comments. However, after looking at @UV’s comment history I see that they would have still gamed that, primarily because it’s still easy to get upvoted low-effort comments here.

                          1. 16

                            Same boat. On the other hand, maybe this will motivate me to start digging through your archives to find interesting things, because I can’t rely on you just posting them here for me ;)

                            1. 11

                              Yeah, it’s a hard choice. I like to think that my own stories, at least as of the past couple of years, are a reasonable fit for this community, and at my current rate of about one post per year I don’t feel like I’m spamming down the site. At the same time, we’ve all seen those account which just post blogspam article after blogspam article from the same domain.

                              Maybe these measures are necessary, but I consider it a good thing that people like yourself, and drew devault, and other people who write in-depth about technology topics they’re genuinely interested in, are able to post their stories here.

                              Besides, this restriction would mostly affect real users who have the community’ best interests at heart, right? If I was a marketing shill and wanted eyeballs I can show content advertising to, I could just create a new account every fourth article, right?

                              1. 8

                                If I was a marketing shill and wanted eyeballs I can show content advertising to, I could just create a new account every fourth article, right?

                                I think we’re actually good in that case! You’d have to invite the alt account, making what you’re doing SUPER obvious. And then we’d bad the entire domain, so you’d never get links from lobsters ever again :D

                                1. 3

                                  I sat down at my laptop after work to respond to this because, yes: I was aware of the perverse incentive, but at least it’s pretty darn obvious and it reveals bad intentions. And I was distracted from finishing this comment to investigate and confirm that, yes, this happened.

                                  1. 2

                                    Why was this user banned? The user submitted 3 things, all of which are relevant and on topic? One of the github links is pretty low quality, but again, not off topic.

                                    Or, maybe the things I want to see no longer align with the site…

                                    1. 2

                                      They were a sockpuppet of vermaden, not a person. I left the three on-topic stories that he submitted as cover for a link promoting his blog.

                                      1. 2

                                        Thanks for the explanation!

                                        So, if that’s provably the case that the account was a sock puppet, ban vermaden?

                                        But, how is having multiple accounts any different than asking “joe rando” to post on my behalf, which I did today (it happened to be posted by someone I actually know, but only after I asked)?

                                        I’m going to start following the hashtag on twitter “#promotemeonlobsters” and submit links that appear to be on topic, that don’t appear to be spam to me.

                                        If I get enough people also do this, there will be a wide variety of potential submitters to these stories, making this silly change irrelevant. Additionally, cannot exactly ban submissions found in that stream, since I can plausibly deny I found it there, and not directly from the source by happenstance.

                                        OR, you could stage same domain posters, showing them to a random sampling of users until they reach some upvote threshold (say 3?), at which point everyone can see them. While you’re at it, perhaps this should be the way all posts start out…

                                        1. 1

                                          I thought about banning vermaden for the sockpuppeting, but I don’t read him as a badly intentioned content marketer, I read him as overly enthusiastic about FreeBSD. And if he’s clever enough to find bugs and foolish enough to not realize I’m paying a lot of personal attention to him while he does it, I’d rather let him continue a bit to catch other bugs/misdesigns.

                                          1. 1

                                            I’ve reread your comment multiple times now, am taken aback, and instead of replying how I really feel about it, I’m going to :eyeroll: and leave it be.

                                2. 8

                                  Want to second this.

                                  It feels like a rule that will punish personal blogs. I’ve been posting stories from my personal blog here before, I’m not sure if there are stories from my blog others posted. I think they match the content people expect here (mostly infosec related) and I don’t think that’s abuse, some of them got well received.

                                  If I’d post on medium etc. I wouldn’t have that problem.

                                  1. 5

                                    It could be time bounded, or tested against multiple time ranges?

                                    For instance, user cannot post from a domain if more than half of stories in the last 6 months are from them.

                                    Or combine that with the original: a user cannot post a domain if they are more than half of all time posts AND they posted more than half within the last 6 months. That way if you could be the majority of all time, but not the majority of recent posts, or vice versa, and still be allowed to post for a certain domain.

                                    And “the last 6 months” could be 3 months, could be 1 year, or what-have-you.

                                    1. 3

                                      I agree. The four link thing is kinda messed up. I write essays every couple of weeks or so, all about making better tech. I’ve spent some time making sure there’s no ads, no email lists, no sales at all on these links. I can’t make them any more innocuous, and I use the comments and feedback from here to learn more and (hopefully) write better stuff.

                                      It doesn’t make sense that I can’t keep doing this. Perhaps the four link rule would work better when the domains were already high traffic sites? You don’t wanna kill the mom and pop grocery stores simply because you don’t like the new WalMart moving into town.

                                    1. 4

                                      This is a few years old, but as the text asks why “-fstack-check” is not enabled by default: There was a discussion around this when Qualys published some research on Stack Clash [1] in 2017.

                                      Developers from Redhat back then came to the conclusion that “-fstack-check” has some compatibility issues and they redeveloped a better version of that feature that is behind the “-fstack-clash-protection” flag in latest gcc versions. I think some Linux distros enable that by default and if yours is not you may ask for it.

                                      [1] https://blog.qualys.com/securitylabs/2017/06/19/the-stack-clash

                                      1. 3

                                        So my question now is, how much does this affect SHA-256 and friends? SHA-256 is orders of magnitude stronger than SHA-1, naturally, but is it enough orders of magnitude?

                                        Also, it’s interesting to note that based on MD5 and SHA-1, the lifetime of a hash function in the wild seems to be about 10-15 years between “it becomes popular” and “it’s broken enough you really need to replace it”.

                                        1. 8

                                          […] the lifetime of a hash function in the wild seems to be about 10-15 years […]

                                          That’s assuming that we’re not getting better at creating cryptographic primitives. While there are still any number of cryptanalysis techniques remaining to be discovered, at some point we will likely develop Actually Good hashes etc.

                                          (Note also that even MD5 still doesn’t have a practical preimage attack.)

                                          1. 3

                                            It would stand to reason that we get as good at breaking cryptographic primitives as we get at creating them.

                                            1. 1

                                              Why? Do you believe that all cryptographic primitives are breakable, and that it’s just a matter of figuring out in what way?

                                              1. 1

                                                I have no idea but that sounds like a GREAT theoretical math problem!

                                            2. 2

                                              This seems likely, but we won’t know we’ve done it until 30-50 years after we do it.

                                            3. 5

                                              In the response to the SHA1 attacks (the early, theoretical ones, not the practical ones) NIST started a competition, in part to improve research on hash function security.

                                              There were voices in the competition that it shouldn’t be finished, because during the research people figured out the SHA2 family is maybe better than they thought. Eventually those voices weren’t heard and the competition was finished with the standardization of SHA3, but in practice almost nobody is using SHA3. There’s also not really a reason to think SHA3 is inherently more secure than SHA2, it’s just a different approach. Theoretically it may be that SHA2 stays secure longer than its successors.

                                              There’s nothing even remotely concerning in terms of research attacking SHA2. If you want my personal opinion: I don’t think we’re going to see any practical attack on any modern hashing scheme within our lifetimes.

                                              Also the “10-15 years” timeframe - there is hardly any trend here. How many relevant hash functions did we have overall that got broken? It’s basically 2. (MD5/SHA1). Cryptography just doesn’t exist long enough for there to be a real trend.

                                              1. 5

                                                As any REAL SCIENTIST knows, two data points is all you need to draw a line on a graph and extrapolate! :D

                                                1. 1

                                                  FWIW, weren’t md2 and md4 were both used in real world apps? (I think some of the old filesharing programs used them.) They were totally hosed long before md5.

                                                  1. 1

                                                    I considered those as “not really in widespread use” (also as in: cryptography wasn’t really a big thing back then).

                                                    Surprising fact by the way: MD2 is more secure than MD5. I think there’s still no practical collision attack. (Doesn’t mean you should use it - an attack is probably just a dedicated scientist and some computing power away - but still counterindicating a trend.)

                                                    1. 1

                                                      I have a vague (possibly incorrect) recollection of hearing that RIAA members were using hash collisions to seed broken versions of mp3 files on early file sharing networks that used very insecure hashing which might have been md4 (iirc it was one where you could find collisions by hand on paper). Napster and its successors had pretty substantial user bases that I’d call widespread. :)

                                                2. 2

                                                  The order of magnitude is a derivative of many years of cryptanalysis over the algorithm and the underlying construction. In this case (off the top of my head), this is mostly related to weaknesses to Merke-Damgard, which sha256 ony partially uses.

                                                  1. 1

                                                    How funny!

                                                    What are your relevant estimates for the time periods?

                                                    When was the SHA-256 adoption, again?

                                                    1. 12

                                                      Here’s a good reference for timelines: https://valerieaurora.org/hash.html

                                                      1. 2

                                                        That site is fantastic, thank you.

                                                  1. 15

                                                    Maybe some folk don’t understand what’s going on here, but this is in direction violation of Postel’s law:

                                                    They’re blocking access from old devices for absolutely no technical reason; they’re blocking read-only access from folks that might not have any other devices at their disposal.

                                                    If you have an old iPod lying around, why on earth should you not be able to read Wikipedia on it? Absolutely no valid technical reason to deny access. Zilch. None. Nada.

                                                    There’s no reason it shouldn’t be possible to read Wikipedia over straight HTTP, for that matter.

                                                    1. 9

                                                      I know next to nothing about security so correct me if I’m wrong, but doesn’t leaving old protocols enabled make users vulnerable to downgrade attacks?

                                                      1. 14

                                                        You’re applying bank-level security to something that’s public information and should be accessible to everyone without a licence or access control in the first place. I don’t even know what sort of comparison to make here best, because in my view requiring HTTPS in the first place here was a misguided decision that’s based on politics, corporate interests and fear, not on rational facts. Postel’s law is also a well-known course of action in telecommunication, even Google still follows it — www.google.com still works just fine over straight HTTP, as does Bing, no TLS mandated from those who don’t want it.

                                                        1. 5

                                                          I agree with you, I’d like to be able to access Wikipedia with HTTP, but this is in my opinion a different issue from disabling old encryption protocols.

                                                          Accessing Wikipedia with secure and up to date protocols might not be necessary to you but it might be for people who live under totalitarian regimes. One could argue that said regimes have better ways to snoop on their victims (DNS tracking, replacing all certificates with one they own…) but I still believe that if enforcing the use of recent TLS versions can save even a single life, this is a measure worth taking. It would be interesting to know if Wikipedia has data on how much it is used by people living in dictatorships and how much dropping old TLS versions would help these people.

                                                          1. 4

                                                            totalitarian regimes

                                                            It’s funny you mention it, because this actually would not be a problem under a totalitarian regime with a masquerading proxy and a block return policy for the https port and/or their own certificates and a certificate authority. See https://www.xkcd.com/538/.

                                                            Also, are you suggesting that Wikipedia is basically blocking my access for my own good, even though it’s highly disruptive to me, and goes against my own self-interests? Yet they tell me it is in my own interest that my access is blocked? Isn’t that exactly what a totalitarian regime would do? Do you not find any sort of an irony in this situation?

                                                            1. 3

                                                              “Isn’t that exactly what a totalitarian regime would do?”

                                                              I think you may have overstated your case here.

                                                              1. 2

                                                                this actually would not be a problem under a totalitarian regime with a masquerading proxy and a block return policy for the https port and/or their own certificates and a certificate authority.

                                                                Yes, this is what I meant when I wrote “One could argue that said regimes have better ways to snoop on their victims”.

                                                                Also, are you suggesting that Wikipedia is basically blocking my access for my own good

                                                                No, here’s what I’m suggesting: there are Wikipedia users who live in countries where they could be thrown in jail/executed because of pages they read on Wikipedia. These users are not necessarily technical, do not know what a downgrade attack is and this could cost them their lives. Wikipedia admins feel they have a moral obligation to do everything they can to protect their lives, including preventing them from accessing Wikipedia if necessary. This is a price they are willing to pay even if it means making Wikipedia less convenient/impossible to use for other users.

                                                          2. 1

                                                            If they left http, yeah, sure. But I don’t think any attack that downgrades ssl encryption method exists, both parties always connect using the best they have. If there exists one, please let me know.

                                                            There is no technical reason I’m aware of. Why does wikipedia do this? It’s not like I need strong encryption to begin with, I just want to read something on the internet.

                                                            I still have usable, working smartphone with android Gingerbread, it’s the first smartphone I ever used. It’s still working flawlessly and I’m using it sometimes when I want to quickly find something when my current phone has no battery and I don’t want to turn on my computer.

                                                            This move will for no reason kill my perfectly working smartphone.

                                                            1. 9

                                                              But I don’t think any attack that downgrades ssl encryption method exists,

                                                              Downgrade attacks are possible with older versions of SSL e.g. https://www.ssl.com/article/deprecating-early-tls/

                                                              It’s not like I need strong encryption to begin with, I just want to read something on the internet.

                                                              Which exact page you’re looking at may be of interest, e.g. if you’re reading up on medical stuff.

                                                              1. 1

                                                                Which exact page you’re looking at may be of interest, e.g. if you’re reading up on medical stuff.

                                                                Are you suggesting that we implement access control in public libraries, so that noone can browse or checkout any books without strict supervision, approvals and logging by some central authority? (Kinda like 1984?)

                                                                Actually, are you suggesting that people do medical research and trust information from Wikipedia, literally edited by anonymous people on the internet?! HowDareYou.gif. Arguably, this is the most misguided security initiative in existence if thought of in this way; per my records, my original accounts on Wikipedia were created before they even had support for any TLS at all; which is not to say it’s not needed at all, just that it shouldn’t be a mandatory requirement, especially for read-only access.

                                                                P.S. BTW, Jimmy_Wales just responded to my concerns — https://twitter.com/jimmy_wales/status/1211961181260394496.

                                                                1. 10

                                                                  Are you suggesting that we implement access control in public libraries, so that noone can browse or checkout any books without strict supervision, approvals and logging by some central authority? (Kinda like 1984?)

                                                                  I’m saying that you may not wish other people to infer what medical conditions you may have based on your Wikipedia usage. So TLS as the default is desirable here, but whether it should be mandatory is another question.

                                                                  1. 2

                                                                    Are you suggesting that we implement access control in public libraries, so that noone can browse or checkout any books without strict supervision, approvals and logging by some central authority? (Kinda like 1984?)

                                                                    PSST, public libraries in the western world already do this to some extent. Some countries are more central than others thanks to the US PATRIOT Act.

                                                                    1. 1

                                                                      public libraries in the western world

                                                                      Not my experience at all; some private-university-run libraries do require ID for entry; but most city-, county- and state-run libraries still allow free entry without having to identify yourself in any way. This sometimes even extends to making study-room reservations (can often be made under any name) and anonymous computer use, too.

                                                                2. 8

                                                                  I still have usable, working smartphone with android Gingerbread, it’s the first smartphone I ever used. It’s still working flawlessly and I’m using it sometimes when I want to quickly find something when my current phone has no battery and I don’t want to turn on my computer.

                                                                  This move will for no reason kill my perfectly working smartphone.

                                                                  It’s not working flawlessly, the old crypto protocols and algorithms it uses have been recalled like a Takata airbag, and you’re holding on because it hasn’t blown up in your face yet.

                                                                  1. 2

                                                                    This move will for no reason kill my perfectly working smartphone.

                                                                    (my emphasis)

                                                                    So you just use this phone to access Wikipedia, and use it for nothing else?

                                                                    If so, that’s unfortunate, but your ire should be directed to the smartphone OS vendor for not providing needed updates to encryption protocols.

                                                                    1. 2

                                                                      our ire should be directed to the smartphone OS vendor for not providing needed updates to encryption protocols

                                                                      I think it’s pretty clear that the user does not need encryption in this use-case, so, I don’t see any reason to complain to the OS vendor about encryption when you don’t want to be using any encryption in the first place. Like, seriously, what sort of arguments are these? Maybe it’s time to let go of the politics in tech, and provide technical solutions to technical problems?

                                                                      1. 1

                                                                        As per my comment, I do believe that the authentication provisions of TLS are applicable to Wikipedia.

                                                                        Besides, the absolute outrage if WP had not offered HTTPS would be way bigger than now.

                                                                3. 15

                                                                  I find the connection to Postel’s law only weak here, but in any case: This is the worst argument you could make.

                                                                  It’s pretty much consensus among security professionals these days that Postel’s law is a really bad idea: https://tools.ietf.org/html/draft-iab-protocol-maintenance-04

                                                                  1. 3

                                                                    I don’t think what passes for “postel’s law” is what Postel meant, anyway.

                                                                    AFAICT, Postel wasn’t thinking about violations at all, he was thinking about border conditions etc. He was the RFC editor, he didn’t want anyone to ignore the RFCs, he wanted them to be simple and easy to read. So he wrote “where the maximum line length is 65” and meant 65. He omitted “plus CRLF” or “including CRLF” because too many dotted i’s makes the prose heavy, so you ought to be liberal in what you accept and conservative in what you generate. But when he wrote 65, he didn’t intend the readers to inter “accept lines as long as RAM will allow”.

                                                                    https://rant.gulbrandsen.priv.no/postel-principle is the same argument, perhaps better put.

                                                                    IMO this is another case of someone wise saying something wise, being misunderstood, and the misunderstanding being a great deal less wise.

                                                                    1. 2

                                                                      I can’t really understand advocating laws around protocols except for “the protocol is the law”. Maybe you had to be there at the time.

                                                                    2. 6

                                                                      As I understand it, they’re protecting one set of users from a class of attack by disabling support for some crypto methods. That seems very far from “absolutely no technical reason”.

                                                                      As for HTTP, if that were available, countries like Turkey would be able to block Wikipedia on a per-particle basis, and/or surveil its citizens on a per-article basis. With HTTPS-only, such countries have to open/close Wikipedia in toto, and cannot surveil page-level details. Is that “no reason”?

                                                                      1. 1

                                                                        As for HTTP, if that were available, countries like Turkey would be able to block Wikipedia on a per-particle basis, and/or surveil its citizens on a per-article basis. With HTTPS-only, such countries have to open/close Wikipedia in toto, and cannot surveil page-level details. Is that “no reason”?

                                                                        I don’t understand why people think this is an acceptable argument for blocking HTTP. It reminds me of that jealous spouse scenario where someone promises to inflict harm, either to themselves or to their partner, should the partner decide to leave the relationship. “I’ll do harm if you censor me!”

                                                                        So, Turkey wants to block Wikipedia on a per-article business? That’s their decision, and they’ll go about it one way or another, I’m sure the politicians they don’t particularly care about the tech involved anyways (and again, it’s trivial for any determined entity to block port 443, and do a masquerade proxy on port 80, and if this is done on all internet connections within the country, it’ll work rather flawlessly, and noone would know any better). So, it’s basically hardly a deterrent for Turkey anyways. Why are you waging your regime-change wars on my behalf?

                                                                        1. 1

                                                                          Well, Wikipedia is a political project, in much the same way that Stack Overflow is. The people who write have opinions on whether their writings should be available to people who want to read.

                                                                          You may not care particularly whether all of or just some of the information on either Wikipedia or SO are available to all Turks, but the people who wrote that care more, of course. They wouldn’t spend time writing if they didn’t care, right? To these people, wanting to suppress information about the Turkish genocide of 1915 is an affront.

                                                                          So moving to HTTPS makes sense to them. That way, the Turkish government has to choose between

                                                                          • allowing Turks to read about the genocide
                                                                          • not allowing Turks any use of Wikipedia

                                                                          The Wikipedians are betting that the second option is unpopular with the Turks.

                                                                          It’s inconvenient for old ipad users, but if you ask the people who spend time writing, I’m sure they’ll say that being able to read about your country’s genocide at all is vastly more important than being able to read using old ipads.

                                                                      2. 4

                                                                        I can think of several reasons:

                                                                        • not letting people know what you are reading
                                                                        • not letting people censor some articles
                                                                        • not letting people modify some articles (for example putting an incorrect download link for a popular software without being detected)
                                                                        • making an habit that everything should be HTTPS (for example for people to not be fooled by phishing sites with the lockpad in the URL bar)
                                                                        1. 2

                                                                          So what’s to stop a totalitarian regime from doing the following?

                                                                          • Redirect all DNS queries to their own DNS servers? The root DNS servers use fixed IP addresses, so it would be easy enough to reroute those addresses to return any address they want.
                                                                          • Redirect all DoH to 1.1.1.1 (or other well known DoH addresses) to again, their own server? Is the CloudFlare public key installed on all browsers? How would you know you are hitting CloudFlare, and not TotallyCloudFlare served by TotallyLegitCA?
                                                                          • Given control over DNS, redirect users to TotallyWikipedia? Again, do you know what CA Wikipedia uses? They can then decode (doesn’t matter if it’s SSL/1.0 or TLS/1.3) the request and proxy it or send out security to question the loyalty of the citizen. Or you know, download the entirety of Wikipedia (which anyone can do), and serve up a cleaned up version to their citizens.
                                                                          1. 1

                                                                            The difficulty is to setup/enrole TotallyLegitCA. How do you do that? If TotallyLegitCA is public, the transparency log will quickly reveal what they are doing. The only way to pull that seems to force people to have your CA installed, like Kazakhstan is doing.

                                                                            1. 2

                                                                              We’re talking about a totalitarian regime (or you know, your standard corporation who install their own CA in the browser).

                                                                        2. 3

                                                                          That’s actually incorrect. There are various technical reasons. But also remember that they need to operate on a vast scale as a non-profit. This is hard.

                                                                          Here are some technical reasons. I’m sure others will chime in as there are likely many more.

                                                                          • some attacks on TLSv1.0 can compromise key material which is used for the newer, secure versions of TLS
                                                                          • attacks only get better
                                                                          • removing old code reduces complexity
                                                                          1. 0

                                                                            providing a read-only version without login over HTTP shouldn’t really add any new code except they’d be on a HTTP-2-only webserver if I’m not mistaken.

                                                                          2. 2

                                                                            There are arguments for an inverse-postel’s law given in https://m.youtube.com/watch?v=_mE_JmwFi1Y

                                                                            1. 0

                                                                              But I hear all the time that I must ensure my personal site uses HTTPS and that soon browsers will refuse to connect to “insecure” sites. Isn’t this a good thing Wikipedia is doing? /s

                                                                              Edit also see this discussion: https://lobste.rs/s/xltmol/this_page_is_designed_last#c_keojc6

                                                                              1. 7

                                                                                I have HTTPS on my completely static website mostly so that no one asks why I don’t have HTTPS, but on the other hand, the “completely static” part is only relevant as long as there are only Eves in the middle and no Mallories.

                                                                                If serving everything over HTTPS will make the life of ISPs injecting ads and similar entities harder, it’s a good thing, until there’s a legal rather than technical solution to that.

                                                                                1. 2

                                                                                  I actually think that HTTPS is reasonable for Wikipedia, if for nothing else to hinder 3rd parties for capture your embarrassing edits to “MLP: FIM erotica” and tracing it to back to you. For a static, read-only site it just adds cost and/or a potential point of failure.

                                                                                  1. 1

                                                                                    For a static, read-only site it just adds cost and/or a potential point of failure.

                                                                                    dmbaturin just said what the value add is. HTTPS prevents third parties from modifying the content of your static site.

                                                                            1. 3

                                                                              Shouldn’t nonce contain unique value? I am no Go expert but it seems that it will always be null string.

                                                                              1. 1

                                                                                That is indeed being passed zero bytes. I think this is only dangerous if you reuse the same nonce for different messages with the same key. It’s a bit subtle, but all the usages of that function I can find are given a unique key every time (like the scrypt recipient has a unique salt, and the x25519 recipient uses emphemeral keys).

                                                                                The document at https://age-encryption.org/v1 contains some language about making sure that property is true (e.g. A new salt MUST be generated for every new file key.)

                                                                                1. 10

                                                                                  Indeed, this is intentional and safe, but worth a comment in the code. Coming up, thank you.

                                                                                  1. 4

                                                                                    Is there a reason not to randomize the nonce? This seems like a “we could use a static nonce, but a random nonce costs us nothing and it may save us if other things go wrong” situation.

                                                                                    1. 3

                                                                                      IMO better to not randomize the nonce and leave a comment explicitly stating why. While I appreciate “defensive engineering” I think code that that has no rational justification rots a codebase over the long term. E.g. anything that eventually results in comments like “XXX: not sure why this needed” makes it really hard to modify the code.

                                                                                      1. 2

                                                                                        Not really, it’s just superfluous overhead. It wouldn’t be unreasonable to randomise it, but I’m fairly confident in the randomness of the key here.

                                                                                1. 8

                                                                                  I’d say Debian is one of the most demanding distros for packaging. Judging from your questions the most jarring one: No internet access allowed. All dependencies must come from other Debian packages.

                                                                                  In general, don’t use anything fancy. If the good old configure-make-make-install process works then great. Established older build systems like CMake or Scons should be fine. Bazel is not packaged for Debian yet, so that would be a problem.

                                                                                  1. 15

                                                                                    Judging from your questions the most jarring one: No internet access allowed.

                                                                                    Every sane distro will have that requirement. Fetching dependencies during build time makes you dependend on external services and makes the process reliably non-reproducible.

                                                                                    1. 2

                                                                                      Every sane distro will have that requirement.

                                                                                      I find this sort of phrasing so weird. So what, Archlinux is an insane distro? Arch build scripts for Rust, for example, will just invoke Cargo, which will download any necessary crates for building.

                                                                                      This has its downsides, but it hardly makes it “insane.” The advantage is that packaging Rust applications (and other applications in other languages with similar packaging tools) becomes much simpler.

                                                                                      1. 3

                                                                                        I can auto-generate Rust packages without the need to fetch from the internet. We extract it, read the Cargo.lock file to determine the dependencies, create .cargo/config to point to our own vendored copies, and run cargo with the –frozen flag.

                                                                                        1. 2

                                                                                          Sure, right. I imagine Debian does something similarish. My point was that Arch, as far as I know, doesn’t require builds to not talk to the Internet. Rust was just an example.

                                                                                          1. 1

                                                                                            In my package manager I just to use cargo vendor, and store a hash of the expected vendor directories, I think it’s a good compromise tbh. Its becoming quite difficult to go against what language package managers expect, but features like cargo vendor make it at least reasonable.

                                                                                  1. 11

                                                                                    As a distro maintainer I can tell you the most important thing at all is the less weird your thing is the more likely it is going to be packaged.

                                                                                    Autotools? I don’t like it, but I know how to handle this. CMake? Yeah, that too. A build system I have never heard of? No. Just no. Never do that.

                                                                                    Plain Makefiles are okay if you stick to common practices. I.e. make CC/LD/CFLAGS/LDFLAGS/PREFIX/DESTDIR/etc. should all work.

                                                                                    1. 2

                                                                                      There’s a few odd statements in there.

                                                                                      Examples:

                                                                                      Our implementation of AES 256-GCM, which we use to comprise the cryptographic barrier for Vault’s data at rest, is resistant against most known quantum attacks. But we respect that this may change in the decades to come as quantum computing enters more into the mainstream.

                                                                                      No, it’s resistant against all known quantum attacks. And while things can obviously change, it seems highly unlikely that this is gonna change. (Also there’s from a quantum resistance no difference between Chacha20 and AES.)

                                                                                      Even today’s limited qubit quantum computers generate extremely random sets of entropy in normal operation. This entropy is extremely valuable in cryptography as operations such as key generation for symmetric key crypto like AES or generating ephemeral session keys for SSH/TLS require robust entropy sources for random number generation.

                                                                                      How so? There isn’t really a problem with “entropy” in cryptography, that’s largely a myth.

                                                                                      1. 4

                                                                                        HTML email is a really big mess security wise. The fact that such a page exists highlights a problem in itself: Nobody really knows what HTML mails really are and what features they’re supposed to support.

                                                                                        If there was a reasonable concept behind HTML mail there would be a standard defining exactly which subset of HTML is allowed within mails. There is no such thing. The simple question “How to I process an HTML mail so it’s safe to display in a webmail frontend?” has no clear answer. Unsurprisingly pretty much all webmail frontends suffer from XSS all the time.

                                                                                        I expanded on this a bit back when efail was found: https://blog.hboeck.de/archives/894-Efail-HTML-Mails-have-no-Security-Concept-and-are-to-blame.html

                                                                                        1. 10

                                                                                          As David Roberts of vox.com notes over and over, the most important thing you can do in the US is Vote Democrat. The only thing that will make a significant difference on a global scale is federal policy change and the Republicans have shown they have no interest.

                                                                                          1. 5

                                                                                            I’m certainly not gonna say you should not vote Democrats. But their track record on climate isn’t good. Major democrats like Nancy Pelosi and Dianne Feinstein have acted in an astonishingly arrogant way towards climate campaigners lately.

                                                                                            If you want Democrats to act on climate, of course you have to vote Repulicans out, but you also have to make sure the people within the democratic party that are silent climate deniers (they won’t say so, but they’ll oppose any meaningful action) don’t get the upper hand.

                                                                                            1. 2

                                                                                              Who believes that a politician will do what he says ? Even more when it implies going against the autonomous development of the Capital. Talk is cheap for all who seek power

                                                                                              1. 1

                                                                                                Both parties are corrupt. They’re corrupt in different ways on some issues. Republicans usually vote against anything that helps in this. So, you’re right. Their party also votes against consumer protections, workers’ rights, etc at State and Federal levels. If you vote for them and don’t own a business, you’re voting against yourself. You’re also still voting against yourself if you’re not rich and interact with any other business that might screw you over.

                                                                                                Another key difference is that Democrat policies mostly waste tons of money, often on welfare, where Republicans like to waste tons of money on locking up Americans for victimless crimes and mass-murdering people overseas for often hard to justify reasons. That Republicans are more pro-mass-murder… as a party, not necessarily individuals… made me firmly draw a line on not voting Republican. I’d be voting for six digits worth of innocent people to die in a way that benefits rich people (esp defense contractors), leaves our lower-paid soldiers with PTSD or physical disabilities, and puts us in debt that I’ll owe back. I’d rather the debt or financial bullshit be something like getting people education, health insurance, jobs, or good infrastructure. The stuff Democrats like to overspend on.

                                                                                              1. 0

                                                                                                See if your local power company lets you buy renewable power. My local utility lets you pay 1¢ extra per kWh on any percentage of your electrical usage for renewable investment.

                                                                                                I do this at home, and we do it at my business. Costs less than $10/month for my house to go 100% renewable.

                                                                                                If you’re in the Madison area, check it out: https://www.mge.com/our-environment/green-power/green-power-tomorrow

                                                                                                1. 3

                                                                                                  This heavily depends on country and availability, but if you buy renewable power from the same company that you bought fossil power that’s not really ideal.

                                                                                                  It’s very well possible that it has zero benefit, because the company probably already has some share of renewables and they may just virtually shift more of that to you while increasing the virtual fossil share of their other customers.

                                                                                                  Ideally you buy renewable electricity from a company that a) is only selling renewable electricity and b) commits to invest a certain share into new renewable energy production and not just sell from already existing facilities. If you can’t have a) and b) at least strive for one of them.

                                                                                                  1. 1

                                                                                                    I don’t disagree with anything you’ve said, but my utility (Madison Gas & Electric) seems to have a decent plan for going net-zero-carbon. I’d prefer them to move faster, and I hope that showing them with my wallet will encourage quicker implementation.

                                                                                                  2. 1

                                                                                                    Or move to Tasmania or New Zealand which are both usually powered > 90% by hydroelectricity.

                                                                                                  1. 20

                                                                                                    Sad :-( I still think Mercurial far better meets the needs of most people, and that the chief reasons for git’s popularity are that Linus Torvalds wrote it, GitHub, and that Linus Torvalds wrote it.

                                                                                                    That said, I did end up switching from BitBucket/mercurial to GitHub/git a few years ago, simply because it’s the more pragmatical thing to do and I was tired of paying the “mercurial penalty” in missed patches and the like. I wrote a thing about it a few ago: https://arp242.net/git-hg.html

                                                                                                    1. 6

                                                                                                      Why do you think hg is better for most people? I honestly find it vastly more complex to use.

                                                                                                      1. 15

                                                                                                        The hg cli is light years ahead of git in terms of intuitiveness.

                                                                                                        1. 6

                                                                                                          I’d say it’s years behind ;)

                                                                                                          1. 10

                                                                                                            How long have you been using Mercurial? I find most people who dislike Mercurial’s UI, are mainly coming from years of experience with Git. I disliked Mercurial at first as well, but after a few years of forced usage it clicked. Now I appreciate how simple and well composed it is and get frustrated whenever I need to look up some arcane Git flag on StackOverflow.

                                                                                                            In general, I’d say you need several years experience with both Git and Mercurial before you can draw a fair comparison.

                                                                                                            1. 3

                                                                                                              I used mercurial for about 2 years before using git.

                                                                                                              1. 3

                                                                                                                Sorry if my post came across a bit accusatory (not my intent). In that case I guess to each their own :).

                                                                                                              2. 3

                                                                                                                but after a few years of forced usage it clicked.

                                                                                                                I’m pretty sure that git clicked for me in a much shorter timeframe.

                                                                                                                1. 1

                                                                                                                  Me too, but I know many (otherwise perfectly competent engineers) 5-10 years in who still don’t get it and aren’t likely to.

                                                                                                              3. 9

                                                                                                                I’m going to strongly disagree. I’ve used git intensively and I find Mercurial to be a well-designed delight. I’ve run across features that Mercurial supports flawlessly, with a nice UI, and Git requires a hacky filter-branch that takes hours to run and doesn’t even behave correctly.

                                                                                                                IMO, a lot of the badness in projects is down to Git badness. it doesn’t scale and people feel compelled to break things down into tiny sub-projects.

                                                                                                                The only reason Git is winning anything is GitHub’s support of it.

                                                                                                                1. 3

                                                                                                                  The only reason Git is winning anything is GitHub’s support of it.

                                                                                                                  Why then was github ever used in the first place? Kind of a strange proposition.

                                                                                                                  1. 1

                                                                                                                    Network effect of the social network is pretty important.

                                                                                                                    1. 1

                                                                                                                      Why would there ever be a network effect in the first place if git was so bad that github was the only reason to use it. I get that the argument technically holds but it seems very unlikely.

                                                                                                            2. 8

                                                                                                              You find mercurial more complex to use than git? That’s an… unusual view, to say the least. The usual recitation of benefits goes something like this

                                                                                                              • Orthogonal functionality in hg mostly has orthogonal commands (compare git commit, which does a half-dozen essentially unrelated different things).
                                                                                                              • hg has a somewhat more uniform CLI (compare git branch -a, git remote -v, git stash list).
                                                                                                              • hg either lacks or hides a bunch of purportedly-inessential and potentially confusing git functionality (off the top of my head, partial commits aren’t baked into the flow a la git’s index/staging area; and rebasing and history rewriting are hidden behind an extension).

                                                                                                              I personally prefer git, but not because I think it’s easier or simpler; I’m more familiar with it, and I find many of those purportedly-inessential functions to be merely purportedly, not actually, inessential.

                                                                                                              1. 5

                                                                                                                One more thing I like about mercurial that the default set of commands is enough for >90% of people, and that everything else is “hidden” in extensions. This is a very different approach than git’s “kitchen-sink” approach, which gives people 170 commands (vs. Mercurial’s 50, most of which also have much fewer options/switches than git).

                                                                                                                Git very much feels like “bloatware” compared to Mercurial.

                                                                                                                1. 3

                                                                                                                  I used git for many years, and then mercurial (at FB) ever since we switched over. The cli interface for mercurial is definitely more sensible, crecord is delightful, and overall it was fine. But I was never able to build a mental model of how mercurial actually worked. git has a terrible interface, but it’s actually really simple underneath.

                                                                                                                  1. 1

                                                                                                                    I didn’t think that underneath they were different enough to matter much. What differences do you mean? I guess there’s git’s remote tracking stuff. Generally, it seems like they differ in how to refer to and track commits and topological branches, locally and remotely. (IMHO, neither has great mechanisms for all the things I want to do.) Mercurial is slightly more complex with the manifest, git is more complex with the staging area that feels absolutely critical until you don’t have it (by using hg), at which time you wonder why anyone bothers with it. I’m a heavier hg user than git user, but that’s about all I can come up with.

                                                                                                                  2. 2

                                                                                                                    You find mercurial more complex to use than git?

                                                                                                                    I actually found – in a professional shop – mercurial far more complex to use. Now, the fact is that mercurials core – vanilla hg is IMHO absolutely without doubt vastly superior to git. Git keeps trying to make the porcelain less painful (including a release just a bit ago) – but I still think it is ages behind.

                                                                                                                    The problem is – I never used vanilla mercurial in a professional environment. Not once. It was always mercurial++ (we used $X extension and $Y extension and do it like $Z) which meant even if I knew hg, I felt painfully inexperienced because I didn’t know mq, share, attic, collapse, evolve, and more… not to mention both the bigger shops I worked with using mercurial has completely custom workflow extensions. I suspect part of this was just the ease of writing mercurial extensions, and part of it was wanting to fall into a flow they knew (mq, collapse). But, regardless of how we got there, at each place I effectively felt like I had to relearn how to use the version control system entirely.

                                                                                                                    As opposed to git, wherein I can just drop in and work from day one. It might be less clean, it might be more finicky and enable things like history rewriting by default. But at the end of the day, the day I start, I know how to generally function.

                                                                                                                    I am curious how Mercurial would have faired if instead of shipping default extensions you had to turn on – if they had just baked a little more functionality, to try to cover the 80% of what most shops wanted (not needed, I think most could have gotten by with what vanilla mercurial had) – if the shop to shop transition would have been easier.

                                                                                                                    1. 2

                                                                                                                      mq, I think, is responsible for many of the “mercurial is too complicated” complaints people have. Evolve, if it ever stabilized and ships with core hg would really enable some killer capabilities. Sadly for social and technical reasons it’s perpetually in beta.

                                                                                                                    2. 1

                                                                                                                      whoa, no index? Admittedly I didnt really use index as intended for several years, but now its an important part of my workflow.

                                                                                                                      1. 1

                                                                                                                        In Mercurial, commits are so much easier to make and manipulate (split, fold, move), that you don’t miss the index. The index in git is just a limited special cased “commit”.

                                                                                                                        1. 3

                                                                                                                          The index in git is just a limited special cased “commit”.

                                                                                                                          I disagree.

                                                                                                                          The index is a useful way to say “these lines of code are ready to go”. If you are making a big commit, it can be helpful to add changes in logical blocks to the index as you go. Then the diff is not polluted with stuff you know is already fine to commit.

                                                                                                                          You might say, “why not just make those changes their own commits, instead of trying to do one big commit?” That’s a valid question if you are talking about a 200 line commit or similar, but sometimes the “big” commit is only 50 lines. Instead of making a bunch of one line or few line commits, its helpful to “git add” small chunks, then commit at the end.

                                                                                                                          1. 0

                                                                                                                            You can as well amend to a commit instead of adding to the index.

                                                                                                                            1. 3

                                                                                                                              True, but all thats doing is bastardizing the commit process. If you are committing a one line change, just to rebase minutes or hours later, thats not a commit.

                                                                                                                              Rebase to me is for commits that were intended to be commits, but later I decided it would be better to squash or change the history. The index is for changes that are never meant to be a full commit on their own.

                                                                                                                              1. 1

                                                                                                                                Having a distinction between draft and published phases in mercurial I think makes it easier to rewrite WIP work. There’s also a number of UI affordances for it. I don’t miss the index using mercurial. There’s also academic user interface research that shows the index is a big conceptual barrier for new users.

                                                                                                                                1. 1

                                                                                                                                  There’s also academic user interface research that shows the index is a big conceptual barrier for new users.

                                                                                                                                  this isnt really a valid point in my opinion. some concepts are just difficult. if some goal can be achieved in a simpler way i am on board, but I am not a fan of removing useful features because they are hard to understand.

                                                                                                                                  1. 1

                                                                                                                                    But the point is the index is hard to understand and unnecessary.

                                                                                                                                    There’s no need to have a “commit process”. Just commit whatever you want and rewrite/amend it for as long as you want. As long as your commits are drafts, this is fine.

                                                                                                                                    Is the problem the word “commit”? Does it sound too much like commitment?

                                                                                                                                    There’s no need to have two separate ways to record changes, an index, and a commit, each with different degrees of commitments. This is multiplying entities beyond necessity.

                                                                                                                                    1. 1

                                                                                                                                      That’s your opinion. The index is quite useful to me. I’d rather make a proper commit once it’s ready, not hack together a bunch of one line commits after the fact.

                                                                                                                                      1. 2

                                                                                                                                        The index is a commit. Why have two separate ways of storing the same sort of thing?

                                                                                                                                        Also, it’s not my opinion that it’s hard to understand and unnecessary; it’s the result of usability studies:

                                                                                                                                        https://spderosso.github.io/oopsla16.pdf

                                                                                                                                        You’re also not “hacking together” anything after the fact. There’s no more hacking together after the fact whether you use git amend (hypothetically) or git add. Both of those mean, “record additional changes”.

                                                                                                                                        1. 0

                                                                                                                                          It seems you have a fundamental misunderstanding of the difference between add and commit. Commit requires a commit message.

                                                                                                                                          1. 1

                                                                                                                                            This isn’t a useful distinction. You can also create commits with empty commit messages in both git and Mercurial.

                                                                                                                                            With both git and Mercurial you can also amend commit messages after the fact. The index in git could well be implemented as a commit with an empty commit message that you keep amending and you wouldn’t notice the difference at all.

                                                                                                                                            1. 1

                                                                                                                                              you keep amending and you wouldn’t notice the difference at all.

                                                                                                                                              yeah, you would. again it seems that you either dont know git, or havent used it in some time. when you amend a commit, you are prompted to amend the message as well. another facet that doesnt exist with git add, because add doesnt involve a message.

                                                                                                                                              if you wish to contort git internals to suit your agenda thats fine, but git add has perfectly valid use cases.

                                                                                                                                              1. 0

                                                                                                                                                you are prompted to amend the message as well.

                                                                                                                                                This is UI clutter unrelated to the underlying concepts. You can get around that with wrappers and aliases. I spoke of a hypothetical git amend above that could be an alias that avoids prompting for a commit message.

                                                                                                                                                Don’t git users like to say how the UI is incidental? That once you understand the data structures, everything else is easy? The UI seems to have locked you into believing the index is a fundamentally necessary concept, but it’s not. It’s an artifact of the UI.

                                                                                                                                                1. 1

                                                                                                                                                  The UI seems to have locked you into believing the index is a fundamentally necessary concept, but it’s not.

                                                                                                                                                  Nothing has locked me into believing its a necessary concept. Its not necessary. In fact, for about 7 years I didnt use the index in any meaningful way.

                                                                                                                                                  I think what you are missing is that Im not compelled to use it because its the default workflow, I am compelled to use it because its useful. It helps me accomplish work more smoothly than I did previously, when I would just make a bunch of tiny commits because I didnt understand the point of the index, as you still dont.

                                                                                                                                                  The argument could be made to move the index into an option, like somehow make commit only the default workflow. Im not sure what that would look like with Git, but I dont think its a good idea. It would just encourage people to make a bunch of smaller commits with meaningless commit messages.

                                                                                                                                            2. 1

                                                                                                                                              You have a set of things you want to accomplish. With git, you have N+1 concepts/features/tools to work with. With hg, you have N (because you drop the index). That means you have to expand your usage of the remaining N.

                                                                                                                                              Specifically, since you no longer have this extra index concept, you now expand commits to cover the scenarios you need. Normally, you’d make an initial commit and then amend a piece at a time (probably with the interactive curses hunk selector, which is awesome.) If you’re unsure about some pieces, or you have multiple things going on that you’d like to end up in separate commits, you can always make a series of microcommits and then selectively collapse them later. (In practice, it’s even easier than this, because of the absorb extension. But never mind that.)

                                                                                                                                              Yes, those microcommits need commit messages. They don’t need to be good ones, because they’re temporary until you squash them out of existence. I usually use a one word tag to specify which of the separate final commits they belong to. (If you don’t have separate final commits, you may as well amend, in which case no messages are needed.)

                                                                                                                                              …or on the other hand, maybe mercurial ends up with N+1 concepts too, because phases really help in keeping things separate. As I understand it, one reason git users love the index is because it keeps rapidly changing, work in progress stuff separate from mostly set in stone commits. Phases perform the same purpose, but more flexibly, and the concepts are more orthogonal so they compose better. In my opinion.

                                                                                                                    3. 6

                                                                                                                      I never particularly liked git and find it unintuitive, too.

                                                                                                                      I wouldn’t consider myself a git poweruser. But whenever I had to work with alternatives I got the feeling that they’re just inferior versions of git. Yeah, maybe the usage was a bit more intuitive, but all of them seemed to lack things that I’d consider really basic (bisecting - hg has that, but e.g. svn has not - and shallow copying - not avaible in hg - are examples what I often miss).

                                                                                                                      1. 3

                                                                                                                        Mercurial was actually my first DVCS, and like you I ended up switching to git not out of a sense that it was technically better, just more pragmatic. For me, the change is more of a mixed bag, though. It is definitely the case that Mercurial’s UI is worlds better, and revsets in particular are an amazing feature that I sorely miss, but when I made the switch I found that the way git handles branches was much more intuitive to me than Mercurial’s branch/bookmark system, and that the affordances around selectively editing commit histories were very much worth the risk in terms of being able to manage the narrative of a project’s history in a way that makes it more legible to others. Ultimately, I found that git’s advantages outweighed its downsides for my use case, since learning its UI idiosyncrasies was a one-time cost and since managing branches is a much more common occurrence for me than using revsets. That said, I think this is a really unfortunate development.

                                                                                                                        1. 2

                                                                                                                          I occasionally convert people’s git repos to hg for my use. Stubborn like that.

                                                                                                                        1. 4

                                                                                                                          The opening comments - particularly about print/parse round trips etc. - suggest a link between fuzzing and property-based testing that I’d love to see explored more. I know that a fuzzer based on Haskell QuickCheck exists but haven’t played with it.

                                                                                                                          1. 4

                                                                                                                            Properties are specifications: what your program is supposed to do. Other names include models and contracts. The code itself is how you attempted to do it. Tests generated from them naturally check the how against the what. Finally, you or your tools can convert each property to a runtime check in the code before fuzzing it. Takes you right to point of failure.

                                                                                                                            Design-by-Contract, contract-based test generation, and fuzzing with contracts as runtime checks is a combo that should work across about any language. Add static/dynamic analysis with low false positives if your language has them. Run this stuff overnight to get more CPU time fuzzing without dragging down performance of your system while you use it.

                                                                                                                            1. 2

                                                                                                                              There are a couple papers on Targeted PBT essentially adding argMax semantics to (at least an Erlang) QuickCheck lib. One can say “test this property using this somewhat non trivial generator and also try to maximize code coverage, as this may help the generation of interesting values”. This is exactly what I did in this proof of concept [1]. It indeed finds counter examples faster than the non maximizing code. In this PoC the non maximizing version often doesn’t find anything at all.

                                                                                                                              I have discovered a passion with this technology and (plug!) am building what will essentially be a language agnostic PBT/fuzzing tool and hopefully SaaS at [2]!

                                                                                                                              [1] https://github.com/fenollp/coverage_targeted_property_testing

                                                                                                                              [2] https://github.com/FuzzyMonkeyCo/monkey

                                                                                                                              1. 1

                                                                                                                                The way I use the terms, the link is quite simple: both are instances of automated tests with generated input data, but with property based testing, there is a relatively strong oracle, whereas with fuzzing, the oracle is limited to “did it crash?”

                                                                                                                                This might be slightly different to how the author here uses the terms, though.

                                                                                                                                1. 4

                                                                                                                                  Your point about oracle is the biggest difference; I think I would expand that to; property based testing can give you statistical guarantees, which means that it tries to sample your program input space according to some pre-defined probability distribution. It doesn’t particularly care about things like coverage either (and as far as I understand it, property based testing should not use feedback — but lines are bluring[1]).

                                                                                                                                  Fuzzing, on the other hand does not particularly care about statistical guarantees (not that you cant make it, but typically it is not done). All it cares about is “can I exercise interesting code that is likely to invoke interesting behaviors”. So, while we use coverage for as a feedback for fuzzing, it is OK to leave aside parts of the program that are not interesting enough.

                                                                                                                                  At the end of the day, I would say the similarities are that both are test generation tools (which also include things like Randoop and Evosuite which are neither fuzzers nor property checkers).

                                                                                                                                  [1] ArbitCheck: A Highly Automated Property-Based Testing Tool for Java

                                                                                                                                  1. 3

                                                                                                                                    I used afl fuzzing to find bugs in math libraries, see e.g. [1] (i.e. things like “divide input a through b with two different libraries, see if the result matches, otherwise throw an assert error”). So you can get the “strong oracle” with fuzzing. I guess you can’t really have a strong line between “fuzzing” and “property-based testing”, it’s just different levels of test conditions. I.e. “doesn’t crash” is also a “property” you can test for.

                                                                                                                                    [1] https://www.mozilla.org/en-US/security/advisories/mfsa2016-07/

                                                                                                                                    1. 2

                                                                                                                                      The original twitter thread where he solicited ideas about how to write fuzzable code had a conversation about how PBT and fuzzing relate: https://twitter.com/mgambogi/status/1154913054389178369.

                                                                                                                                      1. 1

                                                                                                                                        Fuzzing does not limit the oracle to “did it crash?” Other oracles (address sanitizers, for example) are quite common.

                                                                                                                                        There’s obviously some overlap between fuzzing and property based testing, but:

                                                                                                                                        Fuzzing tends to work on the whole application, or a substantial part of it, at once. PBT is typically limited to a single function, although both fuzzing and PBT are useful in different scopes.

                                                                                                                                        Fuzzing tends to run for weeks on multiple CPUs, whereas PBT tends to run alongside unit tests, quickly.

                                                                                                                                        Fuzzing (often!) tends to use profile guidance, whereas PBT does not.

                                                                                                                                    1. 4

                                                                                                                                      I’m happy to see FTP die. But aren’t some websites still providing download links over FTP? I think it was just a year ago when I noticed I was downloading an ISO file from an FTP server..

                                                                                                                                      1. 9

                                                                                                                                        There’s nothing wrong with downloading an ISO from an FTP server. You can verify the integrity of a download (as you should) independently of the mechanism (as many package managers do).

                                                                                                                                        1. 4

                                                                                                                                          I agree! The same goes for downloading files from plain HTTP, as long as you verify the download you know the file is okay.

                                                                                                                                          The reason I don’t like FTP has to do with the mode of operation; port 21 as control channel and then a high port for actual data transfer. Also the fact that there is no standard for directory listings (I think DOS-style listings are the most common?).

                                                                                                                                          1. 2

                                                                                                                                            The reason there’s no standard for directory listings is possibly more to do with the lack of convention on filesystem representation as it took off. Not everything uses the same delimiter, and not everything with a filesystem has files behind it (e.g. Z-Series).

                                                                                                                                            I absolutely think that in the modern world we should use modern tools, but FTP’s a lot like ed(1): it’s on everything and works pretty much anywhere as a fallback.

                                                                                                                                            1. 1

                                                                                                                                              If you compare FTP to ed(1), I’d compare HTTP and SSH to vi(1). Those are also available on virtually anywhere.

                                                                                                                                              1. 1

                                                                                                                                                According to a tweet by Steven D. Brewer, it seems that at least modern Ubuntu rescue disks only ship nano, but not ed(1) or vi(1)/vim(1).

                                                                                                                                                1. 1

                                                                                                                                                  Rescue disks are a special case. Space is a premium.

                                                                                                                                                  My VPS running some Ubuntu version does return output from man ed. (I’m not foolish enough to try to run ed itself, I quite like have a usable terminal).

                                                                                                                                            2. 1

                                                                                                                                              Yes, FTP is a vestige of a time where there was no NAT. It was good until the 90s and has been terrible ever since

                                                                                                                                            3. 1

                                                                                                                                              Most people downloading files over FTP using Chrome don’t even know what a hash is, let alone how to verify one.

                                                                                                                                              1. 1

                                                                                                                                                That’s not really an argument for disabling FTP support. That’s more of an argument for implementing some form of file hash verification standard tbh.

                                                                                                                                              2. 1

                                                                                                                                                There is everything wrong with downloading an ISO over FTP.

                                                                                                                                                Yeah, you can verify the integrity independently. But it goes against all security best practice to expect that users will do something extra to get security.

                                                                                                                                                Security should happen automatically whenever possible. Not saying that HTTPS is the perfect way to guarantee secure downloads. But at the very least a) it works without requiring the user to do anything special and b) it protects against trivial man in the middle attacks.

                                                                                                                                                1. 1

                                                                                                                                                  But it goes against all security best practice to expect that users will do something extra to get security.

                                                                                                                                                  Please don’t use the term best practice, it’s a weasel term that makes me feel ill. I can get behind the idea that an expectation that users will independently verify integrity is downright terrible UX. It’s not an unrealistic expectation that the user is aware of an integrity failure. It’s also not unrealistic that it requires the user to act specifically to gain some demonstrable level of security (in this case integrity)

                                                                                                                                                  To go further, examples that expect users to do something extra to get security (for some values of security) include:

                                                                                                                                                  1. PGP
                                                                                                                                                  2. SSH
                                                                                                                                                  3. 2FA

                                                                                                                                                  Security should happen automatically whenever possible.

                                                                                                                                                  And indeed, it does. Even over FTP

                                                                                                                                                  Not saying that HTTPS is the perfect way to guarantee secure downloads

                                                                                                                                                  That’s good because HTTPS doesn’t guarantee secure downloads at all. That’s not what HTTPS is designed for.

                                                                                                                                                  You’ve confused TLS (a transport security mechanism) with an an application protocol built on top of TLS (HTTPS) and what it does with the act of verifying a download (which it doesn’t). The integrity check in TLS exists for the connection, not the file. It’s a subtle but important difference. If the file is compromised when transferred (e.g. through web of trust, through just being a malicious file) then TLS won’t help you. When integrity is important, that integrity check needs to occur on the thing requiring integrity.

                                                                                                                                              3. 7

                                                                                                                                                You got it backwards.

                                                                                                                                                Yeah, some sites still ofter FTP downloads, even for software, aka code that you’re gonna execute. So it’s a good thing to create some pressure so they change to a more secure download method.

                                                                                                                                                1. 9

                                                                                                                                                  Secure against what? Let’s consider the possibilities.

                                                                                                                                                  Compromised server. Transport protocol security is irrelevant in that case. Most (all?) known compromised download incidents are of this type.

                                                                                                                                                  Domain hijacking. In that case nothing prevents attacker from also generating a cert that matches the domain, the user would have to verify the cert visually and know what the correct cert is supposed to be—in practice that attack is undetectable.

                                                                                                                                                  MitM attack that directs you to a wrong server. If it’s possible in your network or you are using a malicious ISP, you are already in trouble.

                                                                                                                                                  I would rather see Chrome stop sending your requests to Google if it thinks it’s not a real hostname. Immense effort required to support FTP drains all their resources and keeps them from making this simple improvemen I guess.

                                                                                                                                                  1. 1

                                                                                                                                                    MitM attack that directs you to a wrong server. If it’s possible in your network or you are using a malicious ISP, you are already in trouble.

                                                                                                                                                    How so? (Assuming you mostly use services that have basic security, aka HTTPS.)

                                                                                                                                                    What you call “malicious ISP” can also be called “open wifi” and it’s a very common way for people to get online.

                                                                                                                                                    1. 1

                                                                                                                                                      The ISP must be sufficiently malicious to know exactly what are you going to download and setup a fake server with modified but plausibly looking versions of the files you want. An attacker with a laptop in an open wifi network doesn’t have resources to do that.

                                                                                                                                                      Package managers already have signature verification built-in, so the attack is limited to manual downloads. Even with resources to setup fake servers for a wide range of projects, one can wait a long time for the attack to succeed.

                                                                                                                                              1. 1

                                                                                                                                                Patch notes say “TLS 1.0-1.2”.

                                                                                                                                                Any particular reason for the omission of TLS-1.3?
                                                                                                                                                Also, I thought TLS-1.0 was considered pretty insecure[1] at this point?

                                                                                                                                                [1]: from: wikipedia TLS_1.0

                                                                                                                                                The PCI Council suggested that organizations migrate from TLS 1.0 to TLS 1.1 or higher before June 30, 2018.[20][21] In October 2018, Apple, Google, Microsoft, and Mozilla jointly announced they would deprecate TLS 1.0 and 1.1 in March 2020.

                                                                                                                                                1. 2

                                                                                                                                                  I don’t think Netflix is focusing on TLS 1.3 because it’s not widely implemented yet. And 1.0 is fallback for older devices. Netflix doesn’t really care so much of someone does a MITM of your movie.

                                                                                                                                                  Edit: I’m sure there are smart TVs with the Netflix app that can’t go newer than TLS 1.0 and Netflix is contractually obligated to keep it functioning for now

                                                                                                                                                  1. 2

                                                                                                                                                    In which way do you think TLS 1.3 is not widely implemented? According to [1] it’s supported by all mainstream browsers in the latest version.

                                                                                                                                                    Things have changed in this regard. For the majority of users these days it’s normal to have a browser that will update itself automatically on a regular basis. I’m pretty sure major sites already see >50% TLS 1.3 traffic.

                                                                                                                                                    Consider this is a performance feature. Which means a) you don’t need 100%, if you support it for 80% you’re already doing pretty fine and b) it seems strange to want the performance of in-kernel TLS and skip the performance benefits of TLS 1.3.

                                                                                                                                                    [1] https://caniuse.com/#feat=tls1-3

                                                                                                                                                    1. 4

                                                                                                                                                      You’re thinking browsers and I’m thinking devices:

                                                                                                                                                      AppleTV/iOS - not yet

                                                                                                                                                      Roku - not yet

                                                                                                                                                      Etc

                                                                                                                                                      And who watches Netflix in their browser? In all the years I’ve been a customer I don’t think I’ve ever watched in my browser :)

                                                                                                                                                      1. 1

                                                                                                                                                        I occasionally watch Netflix in Firefox on Linux. Not happy about the DRM aspect of it all, but…

                                                                                                                                                    2. 1

                                                                                                                                                      Ah right, forgot this is a Netflix thing. That makes sense that they would want to support TLS 1.0 for a while yet.
                                                                                                                                                      Still seems weird to import a possible footgun (TLS-1.0) that will have to be maintained for 5 years (minimum release support guarantee under the new support model?).

                                                                                                                                                      1. 2

                                                                                                                                                        Still seems weird to import a possible footgun (TLS-1.0) that will have to be maintained for 5 years (minimum release support guarantee under the new support model?).

                                                                                                                                                        Like linux, the key negotiation is still done in userland, it’s just the encryption of packets that is being moved to kernel space and closer to the network driver. I wouldn’t exactly call TLS 1.0 a footgun in that regard.

                                                                                                                                                  1. 6

                                                                                                                                                    So… this is privilege escalation on all Windows versions since XP and it is currently unpatched?

                                                                                                                                                    I don’t know about you, but I run binaries from the internet every workday. I’m not talking about FOSS, either. “Web-based” screen-sharing/conferencing applications that require downloading and executing an .exe come to mind.

                                                                                                                                                    Update: To be clear, some conferencing solutions require each user to download a unique .exe each time you join a conference, not just once to install something..

                                                                                                                                                    1. 2

                                                                                                                                                      Seems there is a patch already, see https://twitter.com/taviso/status/1161297483139407873

                                                                                                                                                      1. 2

                                                                                                                                                        I don’t know about you, but I run binaries from the internet every workday. I’m not talking about FOSS, either. “Web-based” screen-sharing/conferencing applications that require downloading and executing an .exe come to mind. Update: To be clear, some conferencing solutions require each user to download a unique .exe each time you join a conference, not just once to install something..

                                                                                                                                                        That sounds like it can’t possibly be secure unless you either trust the people creating this software or you run them in throwaway-VMs. And I wouldn’t trust people creating software that asks you to run random EXEs all the time…

                                                                                                                                                        1. 1

                                                                                                                                                          It’s Cisco.

                                                                                                                                                      1. 8

                                                                                                                                                        I don’t think there’s anything in this that can’t be explained by strong competition and extreme economies of scale.

                                                                                                                                                        Competition: It’s just that so many places need developers these days, yet the earnings you see are probably not the random crappy app creation startup, but the large corps. They pay because they want the best developers.

                                                                                                                                                        Economies of scale: This is I think really unique in software and other nonmaterial/digital goods (which also explains high salaries for popstars, actors etc.). If Amazon develops a new feature it doesn’t really matter a lot in developer costs and time whether they sell it 10 times or 10 million times. But if they sell it 10 million times the cost of the developer become quite insignificant.

                                                                                                                                                        1. 1

                                                                                                                                                          last time I checked none of the svg optimizer tools produced really good results. I ended up using svgcleaner + svgo to get the best outcome.

                                                                                                                                                          1. 1

                                                                                                                                                            I don’t quite understand this. It looks like, for this to be an issue, the attacker has to be able to set the PHP_VALUE env var to whatever they want? Surely you have bigger issues on your hands if attackers can arbitrarily set environment variables?

                                                                                                                                                            1. 3

                                                                                                                                                              Okay, I guess I should’ve explained this better.

                                                                                                                                                              Part of the fastcgi/fpm protocol is to send over the environment of the client. This effectively means this environment variable can be set by the client, i.e. the attacker.

                                                                                                                                                              This should become clearer if you look at the poc script: https://github.com/hannob/fpmvuln/blob/master/fpmrce

                                                                                                                                                            1. 3

                                                                                                                                                              I have no affiliation to the project but I posted this because it seems like a great solution to the on-going problems with the SKS network, particularly surrounding on-going privacy issues and the abuse of key metadata to post illegal content.

                                                                                                                                                              The new keyserver seems to finally allow the deletion of keys—this is not possible with SKS—and also identity verification by email is finally supported. They seem to have clean separation for identity and non-identity information in keys and all in all it looks like a great evolution from SKS.

                                                                                                                                                              1. 3

                                                                                                                                                                Where do we learn more about the concerns around the SKS network? Sounds interesting and it helps build up point you present.

                                                                                                                                                                  1. 4

                                                                                                                                                                    The article has some interesting links, which I’ll post for convenience:

                                                                                                                                                                    The SKS Devel mailing list has actually had quite a few discussions about this too lately—a very small sample:

                                                                                                                                                                      1. 2

                                                                                                                                                                        The maintainer’s attitude in that first linked ticket is alarming. “The user isn’t supposed to trust us, so there’s no reason not to display bogus data.” Are you kidding me?!

                                                                                                                                                                        1. 1

                                                                                                                                                                          Yes, but the bigger problem is that even if they would want to change it SKS is without actual developers. There are people that maintain it by fixing small bugs here and there but the software is completely and utterly bug-ridden (I had the unfortunate “opportunity” to test it).

                                                                                                                                                                          https://keys.openpgp.org is not mind-blowing¹ but it’s basically a sane keyserver. To have something like this in 2019 shows only in what dire situation is PGP now.

                                                                                                                                                                          ¹ actually I think it’s lacking a little bit compared to “modern” solutions such as Keybase

                                                                                                                                                                          1. 2

                                                                                                                                                                            Even the people that work developing GPG would agree that the situation is sort of bad. Real-world adoption of GPG is almost nil. Support of GPG, say by major email clients, is almost nil. The architecture with the trust model is ‘perfect’ but it’s not user-friendly. GPG-encrypted email traffic is almost not measurable. The code base is apparently a bit of a mess. It needs maybe a bit of funding and probably some less perfect, but more pragmatic and usable strategies of improving security.

                                                                                                                                                                            1. 2

                                                                                                                                                                              Agreed with what you said. I spent some time thinking about this and concluded that at the end the problem is mostly in tooling and UX, not inherent to GPG.

                                                                                                                                                                              As an example: XMPP was described by Google as being “non-mobile friendly” and it took just one person to create a really good mobile XMPP client that can be used by regular people. (I’m using it with my family and it’s better than Hangouts!).

                                                                                                                                                                              GPG too can be brought back from the dead but the effort to do that is enormous because there are multiple parties participating. But there are some good things happening, Web Key Directory, easy to use web clients, keys.openpgp.org

                                                                                                                                                                              Why is it important to work on GPG instead of dumping it for Signal et al.? Because GPG is based on a standard, this is not a “product” that can be sunsetted when investors run away or a manager decides that something else is shiny now.

                                                                                                                                                                              1. 2

                                                                                                                                                                                Look at what keybase is doing. That’s what GPG should have been. Some keyserver that actually verifies things, so that when you get a key with an email address, you know that that email belongs to the person who uploaded the key, unlike the current model, where anyone can upload any key with any data.

                                                                                                                                                                                The whole web-of-trust thing doesn’t help me when I want to get an email from some person overseas I have never met.

                                                                                                                                                                                1. 2

                                                                                                                                                                                  That’s what GPG should have been. Some keyserver that actually verifies things, so that when you get a key with an email address, you know that that email belongs to the person who uploaded the key, unlike the current model, where anyone can upload any key with any data.

                                                                                                                                                                                  If I understood the idea correctly the submission is already what you propose (maybe you’re aware of that? Hard to tell through text alone…)