1. 38

    I sent the link to this post, via DM only, to three of the admins with a short note. Not 10, not 100, not a random project: three of the admins of a project in which I am already a participant.

    Within 60 seconds of linking these users to my own webpage, Discord deleted my account.

    No third-party service should be in a position to be deciding for you what your group membership should be allowed to communicate with each other.

    According to the message, people flagged your DMs. That’s not really “censorship”, that’s removing a member considered disruptive by the community. Frankly, I’d do the same if random strangers started DMing me with “this service sucks, you should use something else”.

    Regardless of whatever merit your points against Discord may have, it seems you don’t realize just how disruptive your “advocacy” is perceived by many. Going around telling other people what they “should” do is what people mean with “Open Source entitlement” and quite literally why people get burnt out by being an Open Source maintainer.

    I strongly urge you to reconsider your approach. It will benefit everyone, including yourself since it will be much more effective. It’s a win-win.

    free software-adjacent teams and groups, such as hackerspaces, art camps, and other DIY undertakings should always question falling by default onto the “buy” side of “build vs. buy”. DIY or die! Run your own!

    Are you doing to do the legwork and front the server costs, too? “DIY” isn’t about telling what other people should do, it’s about … doing it yourself.

    If I was unhappy with the communication platform of a project, I’d compile a list of advantages switching would have and offer to help and/or pay. I don’t want to gatekeep “DIY” here, but in my view that’s the “true” DIY way.

    1. 12

      According to the message, people flagged your DMs. That’s not really “censorship”, that’s removing a member considered disruptive by the community. Frankly, I’d do the same if random strangers started DMing me with “this service sucks, you should use something else”.

      Nothing in that email from Discord says people flagged my DMs. I’m also not a random stranger—I am an active participant in that project. I didn’t disrupt anyone or anything.

      If you read the suspension message carefully, it claims that my account violated the ToS—it did not. It was not the result of messages being flagged. They are using the term “the Discord community” as a stand in for Discord’s automated spam detection, which no-questions-asked censors young/new Tor-created accounts that send three similar messages containing the same link in a short period of time.

      Regardless, it’s still censorship when Alice tries to privately message Bob and Mallory decides “Bob isn’t allowed to see this message” and prevents it from reaching its destination, leaving Bob in the dark. That’s pretty much the dictionary definition of censorship. It’s my opinion that Alice and Bob should seriously reconsider their choice of association with Mallory in that instance.

      Regardless of whatever merit your points against Discord may have, it seems you don’t realize just how disruptive your “advocacy” is perceived by many. Going around telling other people what they “should” do is what people mean with “Open Source entitlement” and quite literally why people get burnt out by being an Open Source maintainer.

      I think perhaps the first line of my post was garbled in transmission. I’m not telling anyone to do anything.

      I’m telling people what they should not do: that is, don’t discriminate against people who insist on privacy.

      Choosing to use Discord does that, so people who don’t want to discriminate should not choose to use Discord.

      I’m also offering them alternatives that don’t discriminate against those people, so that they can make better choices if they decide that they don’t want to be the kinds of projects that discriminate against segments of their userbase.

      I feel like it’s a little bit of a stretch to go from “please don’t discriminate against and exclude me and others like me from participating”, which is basically the message in my post, to “open source entitlement”.

      Are you doing to do the legwork and front the server costs, too?

      I mention on the page that the server costs for such things are on the order of $5 per month for most teams.

      I’d compile a list of advantages switching would have and offer to help and/or pay

      There is an explicit offer of expert help at the bottom of the post, including my direct email address and telephone number, and it has been sitting there on the page since before you left your comment. :)

      I have also donated approximately 5-6 years worth of server hosting expenses, anonymously and in cash, to a local nonprofit I am attempting to convince to switch away from Discord, and have offered to personally manage and document 100% of their migration for free—time for which I would bill a theoretical customer in the mid to high five figures.

      1. 37

        Although I’ve been working as a programmer now for many, many years, prior to that I studied, and received a degree in, philosophy.

        The chair of my department was a Kant scholar, and taught many of the courses in ethics and moral philosophy, and there was a saying he was fond of, to the effect that there are two great traps, or errors, in moral philosophy, which are easy to fall into and difficult to climb back out of. The first trap is concluding that there is no correct moral system. The second trap is concluding that there is, and that you have found it.

        You appear to have fallen into the second trap, and this has had a negative impact on your interactions with other people. For example, prior to falling into the trap, you likely would have recognized that sending unsolicited messages to multiple people promoting your blog post is behavior that those people – and probably most neutral observers – would consider spamming. After falling into the trap, you are unable to see this. After all, you are bringing them the truth and the light and the good word! You are like Moses, descending from the mountain bearing the commandments: how could it be incorrect to share such an important message with others? Surely it must be the other people who are at fault if they react negatively.

        My suggestion to you would be to spend some time working on trying to see this situation from the perspectives of other people, rather than only from your own perspective. To help with that, perhaps consider Kant’s categorical imperative, and consider what the world would be like if your approach were to be made universal. Would you enjoy living in such a world, constantly being bombarded by others’ unsolicited manifestos, constantly being ordered by others to stop doing things they consider immoral, and, if you objected, being told that you are the one who is acting wrongly? I do not think you would find such a world to be pleasant, nor would you find it moral. Think on the lesson that example offers.

        1. 10

          I’m reasonably sure that I just did read an unsolicited manifesto on morals, when I read your post. It is all too easy to stand on a soapbox and become morally superior to others. And if Moses did exist, and if he really did receive instructions from Jehovah, then we must keep in mind that immediately upon coming down from the mountain, he had a fight with his brother over morals and ethics. (We must also keep in mind that evidence suggests that Moses is mythical and that the Exodus did not really happen. It is all too easy to draw moral lessons from myths.)

          On Freenode, if I attempt to privately message somebody, and they are not interested in receiving private messages from me, then I am not instantly banned upon my attempt, but instead notified that the recipient has caller ID enabled and will not be receiving my message.

          In a world where it is universally recognized that Discord is actively interfering with and shaping its user base, perhaps people would not use Discord as often. And that’s all that’s really been asked for.

          Finally, on morality, let us not forget Pirsig. Pirsig morality is the fact that atoms obey the laws of chemistry. It is the Kochen-Specker theorem and the Free Will Theorem. Pirsig said that humans are morally free to do what they want/will/desire, but that humans are inherently not as moral as the ideas which they espouse. At the low level, there are few degrees of freedom, but they are clear and easy to see; when we get up to the level of humans and ideas, there are so many degrees of freedom that the possible moral actions of humans become a continuous spectral palette of moral positions. The typical moral action of a human is to think, and in thinking, be acted upon by ideas, in order to create an emotional context for spurring physical actions.

          Why do I mention Pirsig? Because of this Pirsig quote (from memory):

          It is more moral to kill a man than an idea.

          On one hand, Discord is moral in their choice to be heavy-handed on reputation and moderation, and even moral in their choice to deliberately delegate moderation so as to make each Discord “server” a small fiefdom ruled by jealous gamer overlords. On the other hand, the author, myself, and others are moral in our choice to speak out against and criticize Discord’s design and actions. I think that we value the idea of not living in a police state and not having our mail read, and this idea contrasts sharply and precisely with what Discord’s tools and staff appear to be doing here.

          1. 16

            In a world where it is universally recognized that Discord is actively interfering with and shaping its user base, perhaps people would not use Discord as often. And that’s all that’s really been asked for.

            OP has admitted now that what actually happened was connecting via a service designed to hide the origin of traffic, and immediately firing off multiple DMs containing links to different users. I would actively refuse to use any service that didn’t at least treat that as highly suspect – the odds of that behavior indicating a spambot are ludicrously high.

            Unless you and OP truly believe that it is deeply and reprehensibly morally evil – so evil that you yourself suggest homicide as a preferable alternative – to have systems in place which automatically detect and act on patterns of behavior that are overwhelmingly like to be spam, I’m not sure there’s even a case left to make here. All that’s really left of OP’s argument is a set of desired stances for Free software projects, which would inevitably exclude certain segments of the population (but, notably, not the segment OP belongs to, which apparently makes it acceptable).

            1. 1

              desired stances for Free software projects, which would inevitably exclude certain segments of the population

              Which segments do those desired stances exclude? Are you saying that the communication systems that adhere to these desired stances are inherently user-hostile compared to proprietary, more restrictive systems like Discord?

              1. 1

                Some of the proposed alternatives (specifically IRC) are much less user-friendly than Discord.

                To get a feature like chat persistance, the user will have to either

                • set up a bouncer (usually requires access to a server)
                • use WeeChat/Glowing-Bear (ditto)
                • pay for IRCCloud
            2. 7

              I’m reasonably sure that I just did read an unsolicited manifesto on morals, when I read your post.

              Maybe I’m picking nits… but I do believe I’d consider clicking through to a discussion thread about whether a tool is acceptable for those who value freedom and privacy tantamount to soliciting a manifesto on morals.

              “X is not acceptable for free software” is something that makes me expect that some moralizing and probably at least one manifesto lies on the other side of a link, anyway.

            3. 5

              For example, prior to falling into the trap, you likely would have recognized that sending unsolicited messages to multiple people promoting your blog post is behavior that those people – and probably most neutral observers – would consider spamming.

              Well, it turns out I have an existing relationship with these people. I wasn’t spamming anyone.

              The people to whom I sent the messages never had an opportunity to object to them. They didn’t flag them. They didn’t even see them. Discord’s software decided that because I was a new user, and I was connecting via tor, and I sent the same link to three different people within five minutes of signing in, I must be a spammer and be silenced.

              That’s called censorship.

              Regardless, this is a red herring. The main issue is that choosing to use Discord is exclusionary and discriminatory, regardless of whether they censor messages or not.

              1. 33

                Discord’s software decided that because I was a new user, and I was connecting via tor, and I sent the same link to three different people within five minutes of signing in, I must be a spammer and be silenced.

                That seems like a perfectly reasonable conclusion for their software to draw.

                1. 25

                  That’s called censorship.

                  It really, really isn’t.

                  1. 1

                    It absolutely is. Most censorship is not government censorship. It’s also not universally bad: for example, we self-censor to avoid being unkind to others.

                    1. 15

                      If your working definition of censorship is so broad as to encompass anti-spam measures like rate limit violations, then let me suggest that it is not a useful definition in this conversation.

                      1. 2

                        On the contrary, the fact that legitimate anti-spam measures can be used to block the legitimate sending of messages by people seeking to keep their physical and network location private means that the definition of censorship should definitely include anti-spam measures.

                        1. 7

                          Let me make my point in a different way:

                          because I was a new user, and I was connecting via tor, and I sent the same link to three different people within five minutes of signing in, I must be a spammer

                          This isn’t “legitimate sending of messages” — it is actually spamming.

                          1. 3

                            I don’t think these measures “are being used to block legit sending of messages”, rather, these algorithms block you because your behaviour is virtually indistinguishable from someone sending illicit and abusive messages. Lots of legit email is being blocked by spam filters because the sender lacks DKIM and Reverse DNS, but it’s simply because people not having those is a very sure sign that someone is spamming so you block them without wasting additional CPU on it.

                            If your behaviour is identical to abusive behaviour then I don’t see why you get a free pass for your behaviour relating to a “righteous cause” like free software.

                    2. 10

                      Seems like their spam detection algos are pretty good. Spam detection and prevention (actual spam, not false flags) is one of the shortcomings of IRC and other “anonymous” platforms.

                      1. 0

                        If they were pretty good, they would not get such an obvious false positive.

                        The point was that you should use tools that do not give third parties the ability to read your private messages at all.

                        1. 7

                          So, I don’t like when foss communities use discord (or slack for that matter) either. However, what you’re describing - creating a new account via tor and then immediately sending the same message with a link to three people - sounds like exactly what most spammers I’ve seen will do. What makes you say it’s an obvious false positive (from the perspective of spam detection software)?

                  2. 13

                    Are you doing to do the legwork and front the server costs, too?

                    I mention on the page that the server costs for such things are on the order of $5 per month for most teams.

                    I help run a hackerspace. It has about 75 members and they are all volunteers. We do not have the money for real employees, we do not have on-call support, and it is not uncommon for people to get busy with Real Life Stuff and just disappear for a month or three at a time. If the floor gets swept, it’s because someone decided to pick up a broom and help out.

                    At a guess, members are about 30% professional techies of various types (engineers, academics, technicians, mostly in non-computer fields), about 40% interesting but non-tech people (that hippie who makes cool laser cut art, the cosplay guy who builds a full Iron Man suit, etc), and the rest are interested amateurs who just like playing with different stuff. There are a grand total of three people there who I would actually trust to run a server people rely on, I’m one of them, and I go there to get away from that shit. We can and do run several servers, but they’re all things like an internal NAS or shop IoT system that are toys to play around with and not essential services. We have a VPS that runs our website and a couple other mission critical things, but only a few people have access to it and working with it is not much fun so usually we don’t touch it.

                    Chat is an essential communication medium for this place. There was a fire on our block last year and chat was how we notified people and coordinated stuff. For chat we use Slack. Slack is free, it never breaks, and if it does break we don’t have to fix it. It has an interface a child can set up and use, the client never breaks either, and it takes a new member who isn’t a computer guru about 2 minutes to set up an account.

                    I would love to be able to point people at a Matrix server instead, but last I checked it can’t do all the things Slack can and all the clients I tried were buggy, slow, incomplete, or otherwise unpleasant to use – though this was a year or two ago now, maybe it’s better now. That was the time at which we looked at various chat services and chose Slack though. If we ran a server ourselves, we would need to have someone responsible for babysitting it. I don’t see any commercial services we can buy Matrix hosting from, and a custom managed services setup would probably run $hundreds/month. And even then, we’d have to redo a dozen channels, a couple bots, Google Calendar integration, and get 75 people to switch chat programs.

                    Maybe we can do this someday. Maybe even someday soon. But the costs are far greater than the $5/month for hosting.

                    1. 2

                      I believe Slack can be used easily via Tor, and does not demand a phone number to join a group, so users who need privacy of their personal data (IP/location) would not be excluded from participating in your group.

                      The risk of logged DMs remains, but that is a smaller risk. Discord is much more censorship-heavy.

                      Look into Mattermost and a hidden service.

                      1. 6

                        Slack includes your email address in the profile, forcibly.

                        1. 1

                          You can generate new emails that are not linked to anybody’s account. The article wasn’t about don’t use slack after all, just how it’s not as preferred.

                          1. 2

                            And how many users will do that? If privacy preservation is important to you and you want to be a trustable service provider, you can’t have any situation of “the user accidentally omitted that”. Especially as the user must be pre-informed of that behaviour and have the ability to draw this conclusion before using it.

                            Also, I’m replying to a comment on Slack, so I don’t know what the point about the article is.

                    2. 7

                      I have also donated approximately 5-6 years worth of server hosting expenses, anonymously and in cash, to a local nonprofit I am attempting to convince to switch away from Discord, and have offered to personally manage and document 100% of their migration for free—time for which I would bill a theoretical customer in the mid to high five figures.

                      From the view of a well-managed nonprofit, this reads as: if that person goes away or changes their view on things, there’s the risk of mid high five figures costs.

                      1. 3

                        Discord is not yet a profitable company with a sustainable revenue model.

                        GP has those risks presently, PLUS privacy/discrimination/censorship issues for all of GP’s users.

                        GP says it’s a volunteer organization. Then you say that someone volunteering to do the work is a risk.

                        Running communications tools is about 20-40 hours per year. Can management not extract redundant commitments from reliable members to serve as someone’s understudy in the case of disaster?

                        1. 3

                          GP says it’s a volunteer organization. Then you say that someone volunteering to do the work is a risk.

                          Is the mission of this organisation running a chat service? If not, even in a volunteer organisation, the prime goal is that volunteers can work on the mission.

                          Running communications tools is about 20-40 hours per year. Can management not extract redundant commitments from reliable members to serve as someone’s understudy in the case of disaster?

                          20-40 hours for a skilled person, especially if you have security standards. Finding someone to keep this server safe and secure and is on-call if it breaks is hard.

                          There’s a reasons why even collectives that focus on making communication their mission, like system.li shut down their service on major demonstrations to inform people that they cannot be trusted to not be compromised on some level.

                  1. 8

                    Does someone have a proper doc on this I could replace this thin blog post with?

                    1. 5

                      There is the document on homed itself here: https://systemd.io/HOME_DIRECTORY/

                      The merge request is here, merged 10 days ago: https://github.com/systemd/systemd/commit/4119d1e60a111bacca359b61b5bc3dae29932b67

                      1. 18

                        I don’t think we should change the protocols and force every library in every language on every platform to update mountains of code to support a new protocol just so my browser can download Javascript trackers and crappy Javascript frameworks faster.

                        1. 17

                          I’m excited for HTTP/3 because it will allow me to get lower-latency video streaming support for my private stream server.

                          1. 15

                            Well, just like with HTTP/1 and /2, the old protocols are very likely to be supported for a very long while. So you’re not forced to update.

                            1. 12

                              It’s still change just for the sake of allowing people to build even more bloated websites.

                              Making HTTP more efficient isn’t going to mean websites load faster, it means people are going to stuff even more tracking and malware and bloat into the same space. It’s very, very much like building bigger wider roads with more lanes: it doesn’t alleviate congestion, it just encourages more traffic.

                              1. 27

                                I don’t think that’s entirely true, HTTP/3 does address some problems that we have with TCP and HTTP in modern network connections. I encounter those problems every day at work, it’s just background noise but it annoys users and sysadmins.

                                1. 14

                                  As I understand that video, HTTP/3 is not a new protocol, but rather “HTTP/2 over QUIC”, where QUIC is a replacement for TCP. QUIC can be useful for a lot of other applications, too.

                                  People do a lot of stuff to work around limitations, like “bundling” files, image sprites, serving assets from different domains, etc, and browsers work around with parallel requests etc. So it saves work, too.

                                  Whether you like it or not, there are many WebApps like Slack, GitHub, Email clients, etc. etc. that will benefit from this. Chucking all of that in the “tracking and malware”-bin is horribly simplistic at best.

                                  Even a simple site like Lobsters or a basic news site will benefit; most websites contain at least a few resources (CSS, some JS, maybe some images) and just setting up one connection instead of a whole bunch seems like a better solution.

                                  1. 8

                                    Don’t you think that people are going to stuff even more bloat anyway, even if everybody downgrades to HTTP/1?

                                    1. 6

                                      I don’t know that people will drive less if you make the roads smaller. But they won’t drive as much if you don’t make the roads bigger in the first place. They’ll drive less if you provide bike lanes, though.

                                      In an ideal world AMP would be like bike lanes: special lanes for very efficient websites that don’t drag a whole lot of harmful crap around with them. Instead they’re more like special proprietary lanes on special proprietary roads for special proprietary electric scooters all vertically integrated by one company.

                                2. 9

                                  The old protocols over TCP provide terrible experiences on poor networks. Almost unusable for anything dynamic/interactive.

                                  1. 1

                                    TCP is specifically designed and optimised for poor networks. The worst networks today are orders of magnitude better than the networks that were around when TCP was designed.

                                    1. 13

                                      There are certainly types of poor networks that are ubiquitous today that TCP was not designed for.

                                      For instance, Wifi networks drop packets due to environmental factors not linked to congestion. TCP data rate control is built on the assumption that packets are dropped when the network is congested. As a result, considerable available bandwidth goes unused. This can qualify as a terrible experience, especially from a latency point of view.

                                      If your IP address changes often, say in a mobile network, you lose your connection all the time. Seeing that connection == session for many applications, this is terrible.

                                      Also many applications build their own multiplexing on top of TCP, which, constrained by head of line blocking, leads to buffer bloat and a slow, terrible experience.

                                      1. 5

                                        Related to this:

                                        https://eng.uber.com/employing-quic-protocol/

                                        Mobile networks are a prime target for optimizing latency and minimizing round trips.

                                      2. 1

                                        It was designed when latency didn’t matter. Now it does matter. Three-way handshakes and ACKs are killing us.

                                        1. 1

                                          It seems to me that every reasonable website I use is fine with those tiny inefficiencies because they’re generally efficient anyway, while bloated malware-filled tracking javascript-bloated nightmare websites are going to be bad either way.

                                          Who is this actually helping?

                                    2. 6

                                      Without even talking about HTTP/3, it seems that any application that uses a TCP or UDP connection could benefit from using QUIC: web applications yes, but also video games, streaming, P2P, etc…

                                      Daniel Stenberg also mentioned that QUIC would improve client with a bad internet connection because a packet loss on a stream does not affect the others, making the overall connection more resilient.

                                      I do agree it could and will be used to serve even more bloated websites, but it is not the only purpose of these RFC.

                                    1. 0

                                      Unfortunately the bootstrap node that DHT uses must be hardcoded and is a central point of failure.

                                      1. 2

                                        Or just post the IP on a website and let a user copy-paste it. (Yes, I am aware of the difficulties of getting a user to do anything sensible ;-) )

                                        1. 3

                                          that’s a good idea, sort of like how 8.8.8.8 is manually spread around as google’s open DNS.

                                          the comparison to DNS is apt, DHT is a name resolution system similar to DNS.

                                          DNS : names->ips :: DHT : hash->peers

                                          1. 2

                                            Encode the IP address as base32 or a hashid or something and tell them it’s a secret code!

                                          2. 2

                                            it is a central point of failure for the first time connecting to a DHT network, but subsequent connections can be initiated with previously found peers (some clients do this)

                                            1. 1

                                              Not necessarily, there are certainly methods to not need bootstrap nodes as long as you can make some assumptions like “AWS IP space would have a node in it” and then scan likely IP spaces for nodes.

                                              1. 1

                                                I tried a few decentralised chat clients a year back to see what they were like. I recall one program taking hours to find my other device (user). This wasn’t remarked well in the docs, it just mentioned you had to wait :P


                                                Sidenote: I specifically remember the Tox clients. Voicechat was a feature I tested with a friend, but things kept breaking. Voice streams kept becoming one-way after random periods of time and transferring a file was unreliable and would make the voice stream become unreliable.

                                                It would be interesting to have a look at how things are since then.

                                                EDIT: other memories:

                                                • Ring: big browser app only, couldn’t get it to compile on my distro (Void). Website didn’t offer generic pre-built options, instead told me to use my package manager in my distro (of which only a few were listed). Hope it’s still not a giant webapp (electron?) thing.
                                                • Retroshare: feature-filled, didn’t fully test. Looked interesting, not much publicity.
                                                • Qtox and Utox had different/unique bugs.
                                                • Nothing beat or matched running a murmur server + using mumble in terms of convenience or reliability. Alas I’m annoyed about how many deps this software requires: half of Google IIRC, a pain to compile from scratch on distros that don’t package it (alpine?).
                                                1. 1

                                                  that’s still a hardcoded central point of failure, just one that is more robust against failures

                                                  1. 1

                                                    It’s not that hardcoded, as long as some node is reachable on the internet without NAT traversal, you can establish connection to the network.

                                                    1. 1

                                                      you’d be hardcoding a specific IP space to use, which could easily be blocked by a malicious ISP. if you don’t have access to IPv4, only IPv6, then you can’t scan the internet looking for a node either.

                                                      1. 1

                                                        you hardcode some specific IP space to use first and blacklist problematic IPs, otherwise I don’t see how ISPs would go about blocking it. We’re talking /8 Spaces.

                                                2. 1

                                                  You can easily hardcode a fairly large list of bootstrap nodes. It compresses well.

                                                  1. 1

                                                    and a malicious ISP could easily block all the nodes on that list

                                                    1. 1

                                                      …or do DPI and block DHT traffic anyways. However, circumvention and obfuscation is out of the scope of the article.

                                                1. 24

                                                  DoH doesn’t actually prevent ISPs user tracking

                                                  The article argues that DoH is pointless because the ISP can still read HTTP and the SNI part of TLS.

                                                  While that’s true… HTTP is become more rare and SNI is getting an upgrade to be encrypted.

                                                  Where it does actually help is in non-HTTP related requests (ie DNSSEC, SSHFP, TXT, CNAME)

                                                  DoH bypasses enterprise policies

                                                  DoH can be configured via GPO on Windows (for Firefox atleast)

                                                  DoH weakens cyber-security

                                                  Same as above but this time it’s about “how terrible our shitty middle boxes can no longer smear shit all over the connection”. DoH works with local CA’s so your shitty middlebox can still crack open DoH.

                                                  DoH helps criminals

                                                  See all of the above, if you already have a middlebox then you can crack DoH like any other HTTPS traffic, otherwise criminals could have been using this tech for ages without any issue. Malware has also been using Tor over Bridges and other methods to avoid detection, I doubt this is an issue with DoH any more than before.

                                                  DoH shouldn’t be recommended to dissidents

                                                  Is it?

                                                  DoH centralizes DNS traffic at a few DoH resolvers

                                                  Only if nobody ever uses DoH but just today Microsoft wrote that DoH will be supported by windows; known DoH resolvers will automatically upgrade to DoH and prevent cleartext lookups (if DHCP uses 1.1.1.1 as a DNS server for example). They argue that if DoH becomes widely supported, more DNS servers will support it.

                                                  1. 22

                                                    Helps criminals and shouldn’t be recommended to dissidents is a paradox, dissidents are people who have committed the crime of political dissidence.

                                                    1. 5

                                                      You’re doing a strawman here:

                                                      • it “helps criminals” because it’s simply an alternative avenue that some system administrators aren’t aware of yet; e.g., an extra way for malware to avoid detection;

                                                      • it couldn’t be recommended to dissidents because it’s just bad engineering and a partial/incomplete solution, and very easy to block and circumvent.

                                                      The points may seem contradictory when taken out-of-context, but it’s not really controversial at all once you actually do look at the context here.

                                                    2. 13

                                                      Came here to post several of these. “Bypasses enterprise policies”, “weakens cyber-security”, and “helps criminals” all seem basically like unalloyed good things to me. Shitty enterprise middleboxes and the culture of corporate serfdom they support need to die yesterday.

                                                      1. 5

                                                        While that’s true… HTTP is become more rare and SNI is getting an upgrade to be encrypted.

                                                        Does encrypting SNI actually help? If I see you connecting to 2620:0:862:ed1a::1 I know you’re visiting Wikipedia, or 2a03:2880:f10a:83:face:b00c::25de means you’re on Facebook. ESNI only hides requests to large MITM-concentrators.

                                                        1. 3

                                                          eSNI makes it a lot more difficult, especially if you have a CDN, cloud hoster or shared host on the other end. If the other end is an AWS/GCS/Azure IP then you haven’t learned that much.

                                                          1. 6

                                                            CDN: Yes, for a CDN or MITM-proxy, you may be able to hide the name. Although subsequent requests to 3rd party resources may leak information about the site you’re visiting.

                                                            Cloud hoster: Possible, but not necessarily; try visiting this random IP: http://71.19.148.33 - or just check out the reverse.

                                                            It seems that eSNI only provides privacy in very specific situations. I wouldn’t say it’s good protection if it misses most of the cases. This feels like “something has to be done, eSNI is something so it has to be done”

                                                            1. 3

                                                              Various papers have been published on the topic. Correlating ip addresses with websites is extremely effective. More than 90% of websites don’t change ipaddrs often and don’t share the same addrs with other websites.

                                                              1. 1

                                                                90% would still be better than the current 100%.

                                                          2. 5

                                                            These “shitty middleboxes” are keeping our country’s Critical Infrastructure like our power grid secure. Your comment reads as written by someone who has never worked a day in their life in security.

                                                            1. 10

                                                              From experience, 99% of middleboxes are bad and decrease security overall. There are a few exceptions that work well and those will likely not have any trouble with DoH.

                                                              1. 5

                                                                These “shitty middleboxes” are keeping our country’s Critical Infrastructure like our power grid secure.

                                                                Not sure if this is serious, considering the average (desolate) state of software infrastructure done by hardware companies.

                                                                1. 1

                                                                  I’ve actually worked in security for Critical Infrastructure. Some of this data collected by shitty middleware boxes was exported to Homeland Security, for example.

                                                            1. 7

                                                              Still waiting for someone to explain what “security” this provides. They can still see the IPs you connect to. Just look for the next SYN packet after a response comes back from a known DoH endpoint…

                                                              The one thing this standard does is create a backdoor to make it harder for you to filter content on your network (as required by law in some situations) and makes it harder for your security team to detect bots/malware/intrusions by triggering on lookups to known malware C&C servers. TLS 1.3 plus this means it’s extremely difficult especially for critical infrastructure (e.g., power generation companies) to filter egress traffic effectively.

                                                              If you want to stay out of prison for dissenting, you need a VPN*. If you want privacy, use a VPN*. This doesn’t solve either; it only makes it possible to avoid naughty DNS servers that modify your responses. But we already had solutions for that.

                                                              * and make sure the VPN is trustworthy or it’s an endpoint you control.

                                                              1. 7

                                                                No need to put scare-quotes on security. It hides DNS traffic. Along with eSNI it hides the domains you’re visiting. And if the domain uses a popular CDN, this makes the traffic very hard to spy on, which is a measurable improvement in privacy.

                                                                you need a VPN

                                                                Oh no, aren’t VPNs evil, because, as you said yourself, they make “it harder for you to filter content on your network (as required by law in some situations)”?

                                                                The false-sense-of-security traffic inspection middleboxes that were always easy to bypass with a VPN or even a SOCKS proxy, were needlessly weakening TLS for decades. Fortunately, they’re dead now.

                                                                1. 1

                                                                  VPNs are much easier to block. You can do it at the protocol level for most types (you’re whitelisting outbound ports and protocols, right?) then you have lists of the public VPN providers to block as well.

                                                                  If you’re only allowing outbound TCP 443 and a few others someone could do TCP OpenVPN over it, but performance is terrible and it’s unreliable so most people don’t try.

                                                                  Regardless there are DPI devices which can fingerprint the OpenVPN traffic and tell it apart from HTTPS traffic because behaves differently (different send/receive patterns) and then you inject RST packets to break the session.

                                                                2. 4

                                                                  Seeing IP’s that you connect to isn’t always useful, e.g. attacker wouldn’t realistically gain anything if a website you connect to is served through cloudflare, which serves enough different websites that it provides little information for the attacker.

                                                                  1. 4

                                                                    You can easily connect to the IP and grab the list of domains on the SAN certificate that CloudFlare is using on that IP address to figure out where they’re connecting. There’s only like 25 per certificate. It’s not hard to figure out if you are targeting someone.

                                                                    e.g., it would not be difficult to map 104.18.43.206 to the CloudFlare endpoint of sni229201.cloudflaressl.com and once you have that IP to CloudFlare node mapping sorted out you can craft a valid request …

                                                                    Subject Alternative Names: sni229201.cloudflaressl.com, *.carryingcoder.com, *.carscoloringpages101.com, *.caudleandballatopc.com, *.coloringpages101.com, *.cybre.space, *.emilypenley.com, *.indya101.com, *.nelight.co, *.scriptthe.net, *.shipmanbildelar.se, *.teensporn.name, *.thereaping.us, *.totallytemberton.net, *.voewoda.ru, *.whatisorgone.com, carryingcoder.com, carscoloringpages101.com, caudleandballatopc.com, coloringpages101.com, cybre.space, emilypenley.com, indya101.com, nelight.co, scriptthe.net, shipmanbildelar.se, teensporn.name, thereaping.us, totallytemberton.net, voewoda.ru, whatisorgone.com
                                                                    
                                                                    1. 2

                                                                      This list is encrypted in TLS 1.3, so you can’t easily grab it anymore (Firefox and Cloudflare also support eSNI, which plugs another hole).

                                                                      1. 1

                                                                        You misunderstand. I would create a database mapping of all CloudFlare nodes in existence: sniXXXXXX.cloudflaressl.com <—> IP addresses.

                                                                        When I see traffic to one of these IPs, I simply make a new TLS handshake to sniXXXXXX.cloudflaressl.com, grab the certificate, read all of the domain names in the certificate. I don’t need a plaintext SNI request to see where they’re going; I can just infer it by asking the same server myself.

                                                                        1. 2

                                                                          You’ll only learn that all Cloudflare customers share a handful of IP addresses, and there are millions of sites per IP.

                                                                          The certificate bundles aren’t tied to an IP, and AFAIK even the bundles aren’t constant.

                                                                          1. 1

                                                                            The server publishes a public key on a well-known DNS record, which can be fetched by the client before connecting (as it already does for A, AAAA and other records). The client then replaces the SNI extension in the ClientHello with an “encrypted SNI” extension, which is none other than the original SNI extension, but encrypted using a symmetric encryption key derived using the server’s public key, as described below. The server, which owns the private key and can derive the symmetric encryption key as well, can then decrypt the extension and therefore terminate the connection (or forward it to a backend server). Since only the client, and the server it’s connecting to, can derive the encryption key, the encrypted SNI cannot be decrypted and accessed by third parties.

                                                                            That’s fine, then someone will just excise the encrypted SNI part to use it in a crafted packet that’s almost like a replay attack. That will still get you the list of 25ish domains they could have accessed.

                                                                            Hell, this looks like you could eventually build a rainbowtables out of your captured SNI packets once you have sorted through the available metadata to see where they user went. (Assuming CF doesn’t rotate these keys regularly) Just analyze all sites on that cert, see all the 3rd party domains you need to load, and you can figure it out.

                                                                            This is a small hurdle for a state actor

                                                                            edit: I’m pretty sure you can just do a replay of the SYN to CloudFlare and not worry about trying to rip out the SNI part to get the correct certificate (TCP Fast Open)

                                                                            edit2:

                                                                            7.5.1.  Mitigate against replay attacks
                                                                            
                                                                               Since the SNI encryption key is derived from a (EC)DH operation
                                                                               between the client's ephemeral and server's semi-static ESNI key, the
                                                                               ESNI encryption is bound to the Client Hello.  It is not possible for
                                                                               an attacker to "cut and paste" the ESNI value in a different Client
                                                                               Hello, with a different ephemeral key share, as the terminating
                                                                               server will fail to decrypt and verify the ESNI value.
                                                                            

                                                                            Yeah you can’t replay the ESNI value, but if you replay the entire Client Hello I think it should work. The server won’t know the client’s “ephemeral” ESNI key was re-used.

                                                                            https://datatracker.ietf.org/doc/draft-ietf-tls-esni/?include_text=1

                                                                            1. 4

                                                                              The client Hello ideally only contains a client’s public key material, so you can’t decrypt the ESNI even if you replay the client hello. Unless you use a symmetric DH operation (which is rare and not included in TLS1.3) or break ECDH/EdDH/ECDHE.

                                                                              1. 2

                                                                                You are correct. I was going to post this after some coffee this morning. The response is encrypted with the client’s ephemeral ECDHE key.

                                                                                So this breaks this type of inspection.

                                                                                However, if you’re connecting to an endpoint that’s not on a CDN and is unique the observer can still figure out where you’re going. Is the solution we’re going to be promoting over the next few years to increase reliance on these CDN providers? I really don’t like what CloudFlare has become for many reasons, including the well known fact that nothing is free. They might have started with intentions of making a better web but wait until their IPO. Once they go public, all bets are off. All your data will be harvested and monetized. Privacy will ostensibly be gone.

                                                                                In America it’s illegal to make ethical choices if it doesn’t maximize shareholder value. (Ebay v Newmark, 2010)

                                                                    2. 3

                                                                      yes, and that protects exactly.. no one who needs it.

                                                                      if you live somewhere where you need the security to hide your DNS requests, cloudflare will be the first thing to get blocked. the only really secure thing to do is onion routing of the whole traffic. centralizing the internet makes it more brittle.

                                                                      additionally: ease of use is no argument if it means trading-off security. these tradeoffs put people in danger.

                                                                      1. 3

                                                                        As someone who barely knows his TCPs from his UDPs, I had to read up on DoH, and I must say that a technology must be doing something right if it elicits both your reaction and the following from the Wikipedia article:

                                                                        The Internet Watch Foundation and the Internet Service Providers Association (ISPA)—a trade association representing UK ISPs, criticised Google and Mozilla for supporting DoH, as they believe that it will undermine web blocking programs in the country, including ISP default filtering of adult content, and mandatory court-ordered filtering of copyright violations.

                                                                        1. 3

                                                                          i think DoH is the wrong solution for this problem, stuffing name resolution into an unrelated protocol. it may be true that it has the side-effect of removing the ISP-DNS-filters, but those can already be circumvented by using another name server.

                                                                          a better solution would be to have a better UI to change the nameservers, possibly in connection with DNS over TLS, which isn’t perfect, but at least it isn’t a mixture of protocols which DoH is.

                                                                          it could be an argument that the ISP could block port 53, and DoH would fix that. then we have another problem, namely that the internet connection isn’t worth it’s name. the problem with these “solutions” is that they will become the norm, and then the norm will be to have a blocked port 53. it’s a bit like the broken window theory, only with piling complexity and bad solutions.

                                                                          maybe that’s my problem with it: DoH feels like a weird kludge like IP-over-ICMP or IP-over-DNS to use a paid wifi without paying.

                                                                          1. 2

                                                                            maybe that’s my problem with it: DoH feels like a weird kludge like IP-over-ICMP or IP-over-DNS to use a paid wifi without paying.

                                                                            I agree with you that it feels like a kludge, it feels icky to me too.

                                                                            But it’s something that could lead to a better internet - at the moment DNS traffic is both unencrypted, but more importantly, unauthenticated. If a solution can be found that improves this, even if it’s a horrible hack, I think it’s a net win.

                                                                            Internet networking, like politics, is the art of the possible. We can all dream of a perfect world not beholden to vast corporate interests at every level of the protocol stack, but in the meantime the best we can hope for is to leverage some vast corporate interests against others.

                                                                            1. 2

                                                                              But it’s something that could lead to a better internet - at the moment DNS traffic is both unencrypted, but more importantly, unauthenticated. If a solution can be found that improves this, even if it’s a horrible hack, I think it’s a net win.

                                                                              It may be a short term win, but in the end we are stuck forever with another bad protocol because nobody took the time and effort to build a better one, or just had an agenda.

                                                                              Internet networking, like politics, is the art of the possible. We can all dream of a perfect world not beholden to vast corporate interests at every level of the protocol stack, but in the meantime the best we can hope for is to leverage some vast corporate interests against others.

                                                                              DoH is just another way of centralizing the net. sure you can set another resolver in the settings, but for how long? you’d have to do that on every device. or use the syncing functionality which is.. centralized. and even, who does that?

                                                                              i don’t think that “big players” in politics or in tech, do things out of altruistic reasoning, but, in the best case, good old dollar. that paired with most of the things being awful hacks (again in both, politics and tech) paints a bright future.

                                                                              1. 2

                                                                                I mean, the reality we live in now, where a company like Cloudflare has a de-facto veto on Internet content, just grew organically. It’s an inevitable consequence of technical progress, as stuff (like hosting, and DDoS protection) gets commoditized efficiencies of scale make large companies are the only ones who have a hope of making a profit.

                                                                                To their credit, Cloudflare seem aware and uncomfortable about their role in all this, but that’s scant consolation as they’re under the same profitability requirements as the rest of the free world. They can be sold, or move to “evil” to save their profits.

                                                                                1. 3

                                                                                  Yep - even prior to DoH, Cloudflare have BGP announce privileges and can issue certificates which are trusted by browsers, which are two powers which should never have been combined in the same entity (being able to funnel a sites traffic to your servers and also generate valid certs for those requests).

                                                                                  1. 2

                                                                                    I mean, the reality we live in now, where a company like Cloudflare has a de-facto veto on Internet content, just grew organically.

                                                                                    … and with their resolver being the default one they even have control over the rest, amazing!

                                                                                    It’s an inevitable consequence of technical progress, as stuff (like hosting, and DDoS protection) gets commoditized efficiencies of scale make large companies are the only ones who have a hope of making a profit.

                                                                                    the need for something like DDoS protection is more a consequence of full-throttle capitalism ;)

                                                                                    1. 1

                                                                                      with their resolver being the default one

                                                                                      For the fraction of internet users running Firefox, sure. Google will handle the rest. No doubt MSFT will hop on board too.

                                                                                      the need for something like DDoS protection is more a consequence of full-throttle capitalism

                                                                                      Or technical debt inherited from a more trusting vision of the internet…

                                                                                      (edit addressed Cloudflare’s role as default DoH provider for Firefox)

                                                                            2. 2

                                                                              UK ISPs have to block child porn or the CEO will be held accountable and go to prison. They do DNS filtering, because IP filtering is impossible. Now they can’t even do that.

                                                                              1. 5

                                                                                I’m aware of the legal requirements of UK ISPs (although why they feel they need to celebrate this requirement by awarding (then withdrawing) the “Internet Villain of the Year” to Mozilla is beyond me).

                                                                                I guess the “responsibility” for filtering/blocking will move up to Cloudflare.

                                                                                1. 1

                                                                                  we’ve had a lengthy political discussion in germany about this topic (where “filtering” was a long time the preferred political solution) now the policy is to ask the respective hoster to delete these things. i have no good english source for this, so here is the translated german wikipedia article (original)

                                                                                  1. 3

                                                                                    You can push the ISP to DNS block it (though it’s harder and usually leads to years-long court cases as in Vodafone’s case).

                                                                                    Telekom also loves to push their own search engine with advertisements for NXDOMAIN responses.

                                                                          2. 3

                                                                            Still waiting for someone to explain what “security” this provides. They can still see the IPs you connect to. Just look for the next SYN packet after a response comes back from a known DoH endpoint…

                                                                            It does one useful thing: It prevents them from MITMing these packets and changing them.

                                                                            I’d like encrypted DNS, but I’m very strongly against Firefox selecting my DNS resolver for me for reasons that have already been stated in threads here. I also strongly prefer keeping the web stack out of my relatively simple client-side DNS resolver. Diverse ecosystems are important, and the only way to maintain them is to keep software simple enough that it is cheap to implement.

                                                                            1. 1

                                                                              It does one useful thing: It prevents them from MITMing these packets and changing them.

                                                                              Sure, but that’s rare. It would require a targeted attack or a naughty ISP to be altering results.

                                                                              What it most certainly does is prevent me from forcing clients to use my on-premises DNS resolver. Now you have zero controls over the client devices on your network when it comes to DNS and additionally we’re about to lose HTTPS inspection in the near future. This is the wrong approach to solve the problem. Admins need controls and visibility to secure their networks.

                                                                              Mark my words, as soon as this is supported by a few different language libraries you’ll see malware and all sorts of evil things using it to hide exfiltration and C&C because it will be hidden in the noise of normal user traffic.

                                                                              It will be almost impossible now to stop users or bad guys from accessing Dropbox, for example. “Secure the endpoints” is not the answer. You can secure them, deny BYOD, etc, but you have to assume they’re compromised and/or rooted. Only the network is your source of truth about what’s really happening and now we’re losing that.

                                                                              1. 4

                                                                                I guess I don’t have much sympathy for the argument that network administrators will lose insight into the traffic on their networks. That seems like a bonus to me, despite the frustration for blue teams.

                                                                                1. 3

                                                                                  Same. I understand that in some places there are legal auditing requirements, but practically everywhere else it’s just reflexive hostility towards workers that makes us use networks that are pervasively censored and surveilled.

                                                                                2. 4

                                                                                  Sure, but that’s rare. It would require a targeted attack or a naughty ISP to be altering results.

                                                                                  Except that it’s not rare. You will find this in many hotel wifis. This hits you particularly hard if you have a DNSSEC validating resolver, which doesn’t take kindly to these manipulations. Having a trusted recursor is generally important if your want to be sure that you talk to a resolver you can actually trust, which is in turn important if you want to delegate validation.

                                                                                  What it most certainly does is prevent me from forcing clients to use my on-premises DNS resolver.

                                                                                  Just as HTTPS prevents you from forcing your clients to talk to an on-remise cache or whatever. The solution is the same in both cases. You need to intercept TLS, if this is a hard requirement for you. DoH and DoT isn’t making anything more complicated, its just bringing DNS on par with the protection level we have had for other protocols for a while.

                                                                                  1. 3

                                                                                    You hit the nail on the head here. Far from being rare, in the US it’s ubiquitous, whether it’s your hotel, your employer, or your residential ISP.

                                                                                  2. 3

                                                                                    Only the network is your source of truth about what’s really happening and now we’re losing that.

                                                                                    Good. Corporate networks must die. “Secure the endpoints” is THE ONLY answer.

                                                                                    https://beyondcorp.com

                                                                                    If Google can pull it off at Google scale, so can you. Small teams with lots of remote people have always been Just Using The Internet with authentication. It’s the “Enterprise”™ sector that’s been suckered into buying “Security Products”™ (more like “Spying Products”) to keep trying to use this outdated model.

                                                                                    1. 2

                                                                                      What it most certainly does is prevent me from forcing clients to use my on-premises DNS resolver.

                                                                                      Could you please elaborate? Is this about a “non-canonical” local resolver or do you think it also has repercussions for locally hosted zones? For example *.internal.example.org locally versus *.example.org on the official internet. Or did I misunderstand you and you just meant a local forwarding resolver?

                                                                                      I honestly didn’t read up enough on DoH yet, just wondering.

                                                                                      1. 1

                                                                                        Mark my words, as soon as this is supported by a few different language libraries you’ll see malware and all sorts of evil things using it to hide exfiltration and C&C because it will be hidden in the noise of normal user traffic.

                                                                                        Setup your own DoH server and you can once again inspect it. Ideally you use a capable and modern TLS intercepting box to inspect all traffic going in and out (as well as caching it).

                                                                                        1. 1

                                                                                          Mark my words, as soon as this is supported by a few different language libraries you’ll see malware and all sorts of evil things using it to hide exfiltration and C&C because it will be hidden in the noise of normal user traffic.

                                                                                          How? The IP or the URL of the DoH server you are talking to will stand out like a signal flare… I think that dumping the file to a cloud-service is way more efficient, easier and effective.

                                                                                          1. 1

                                                                                            The US Gov often gives early reports to security teams of Critical Infrastructure networks details on all sorts of potential attacks, including early heads up on malware that may or may not be targeted. This includes a list of C&C domains that may be accessed. If the software can hide its DNS requests by making it look like normal HTTPS traffic to CloudFlare, that makes it even harder to identify the malware’s existence on your network.

                                                                                            If you want the Russians or Chinese to hack our grid, this is a great tool for them along with TLS 1.3. The power generation utility that I worked at did HTTPS interception and logging of ALL HTTPS and DNS requests from every device everywhere for analysis (and there was a program coming online to stream it to the government for early detection) and now this is becoming impossible.

                                                                                            1. 1

                                                                                              This pertains only to firefox… So why would an installation of firefox be on one of those networks?

                                                                                              Furthermore: You know the ip of cloudflare’s DoH server. You could just block that and be done with it right? If the malware uses some other server, that will show up as well.

                                                                                              1. 2

                                                                                                Firefox won’t be on that network, but HTTPS certainly will be. Likely not on (hopefully still airgapped) SCADA, but on other sensitive networks that give some level of access into SCADA through various means.

                                                                                                The point is that as DoH thrives and becomes commonplace and someone like CloudFlare runs this service, it’s easy to hide DNS requests mixed in with normal looking HTTPS traffic. The client can be a python script with DoH capability.

                                                                                                As for CloudFlare’s DoH service – it appears to be running on separate IPs at the moment, but there’s no reason why they couldn’t put this on their normal endpoints. DoH is HTTPS, so why not share it with their normal CDN endpoints? This would not be difficult to do in Nginx. In fact this would be far simpler than running HTTPS and SSH on the same port, which is also possible.

                                                                                                Basically any normal-looking HTTPS endpoint could become a DoH provider. Hack some inconspicuous server, reconfigure their webserver to accept DoH too, and now you’ve got the backdoor you need for your malware.

                                                                                                CloudFlare and Firefox are not my concern; DoH as a whole is.

                                                                                                1. 1

                                                                                                  As for CloudFlare’s DoH service – it appears to be running on separate IPs at the moment, but there’s no reason why they couldn’t put this on their normal endpoints. DoH is HTTPS, so why not share it with their normal CDN endpoints? This would not be difficult to do in Nginx. In fact this would be far simpler than running HTTPS and SSH on the same port, which is also possible.

                                                                                                  Fair point…

                                                                                                  But now I’m wondering why you would have access to cloudflare on such a network… Or why there won’t be a root-certificate on all the machines (and firefoxes) in the network so that the organization can MITM’s all outgoing traffic?

                                                                                                  1. 1

                                                                                                    There are going to be some networks running servers that need outbound HTTPS for various reasons, but a lot of that can be locked down. But what about the network that the sysadmins are on? They need full outbound HTTPS, and a collaborating piece of malware on one of their machines gives them access to the internet and to other internal sensitive networks. These types of attacks are always complex and targeted. Think of the incredible work we did with Stuxnet.

                                                                                                    As for MITM the traffic… look at this thread where it’s being discussed further https://lobste.rs/s/pechdy/turn_off_doh_firefox_now#c_inbnse

                                                                                                    1. 1

                                                                                                      There are going to be some networks running servers that need outbound HTTPS for various reasons, but a lot of that can be locked down.

                                                                                                      So why cloudflare? I doubt you’d need any high-volume sites that use cloudflare for those setups.

                                                                                                      But what about the network that the sysadmins are on? They need full outbound HTTPS, and a collaborating piece of malware on one of their machines gives them access to the internet and to other internal sensitive networks. These types of attacks are always complex and targeted. Think of the incredible work we did with Stuxnet.

                                                                                                      If the networks really are that sensitive, just separate them physically, give the sysadmins two machines and never transport data in digital form from the one to the other….

                                                                                                      If you are not willing to take these kinds of steps, your internal networks simply aren’t that critical.

                                                                                                      1. 2

                                                                                                        That is not how the networks at our power utilities work. And it’s not how the employees operate either.

                                                                                                        1. Many power companies refuse to implement new technologies or network topologies unless another utility does it first. Which sadly means that in certain regions like MISO you can expect most of the utilities to be using the same firewalls, etc etc. Very dumb. Can’t wait for Russia to abuse this and take down half the country.

                                                                                                        2. The people that work there aren’t the brightest. “Why are user accounts being managed with a perl script that overwrites /etc/passwd, /etc/shadow, and /etc/groups?” Well because that’s the way they’ve always done it, so if your team needs to install a webserver you also need to tell them to add the www user to their database so the user account doesn’t get removed. “Why are the admins ssh-ing as root everywhere with a DSA key that has no passphrase protection?” because the admins (of 20 years experience) refuse to learn ssh-agent and use basic security practices. I had meetings with developers who needed their application to be accessible across security domains and the developer couldn’t tell me what TCP port their application used. “What’s a port?”. These are people making 6 figures and doing about 30 minutes of work a day. It’s crazy.

                                                                                                        3. These are highly regulated companies with slim margins. You want these kinds of drastic changes to their infrastructure? You better start convincing people to nationalize the grid because they don’t have the money to do it. Remember, it takes about 3 years to get a utility rate change approved. It’s a long process of auditing and paperwork and more auditing and paperwork to prove to the government that they really do need to increase utility rates to be able to afford X Y and Z in the future. They’re slow moving. Very slow.

                                                                                                        4. Do you think customers will want their power bills to go up just so they can hire competent IT staff? Not a chance. (What we really need to do is stop subsidizing bulk power customers and making normal residential customers pay more than their fair share, but that’s a different discussion)

                                                                                                        tl;dr we can all wish hope and pray that companies around the world will do the right thing, but it’s not going to happen anytime soon, especially in Critical Infrastructure environments because they’re so entrenched in their old ways and don’t have the budgets to do it the right way regardless.

                                                                                                        1. 1
                                                                                                          1. In utility companies, the production networks running the power plants should simply not come into contact with the internet. There should always be a human inbetween the network and the internet. If this is not the case, they deserve what’s coming.

                                                                                                          2. Believe it or not. I can actually understand why they dump into /etc/groups, /etc/passwd and /etc/shadow. There is no chance of any machine having an outdated users by accident or by partial configuration this way, and if your network has only a few hundreds of users, which are all more or less trained to deal with complex technological systems on a basic level. Why not? It’s not like they are running a regular common office workplace.

                                                                                                          However, what you are telling me about SSH and TCP is quite shocking. That is just plain incompetence.

                                                                                                          1. I’m not living in the US. In fact; the last time I’ve been there I was at an age from which I can barely remember anything other than that the twin towers still stood. I am often told that it’s a different country now, so I can’t say anything useful about this.

                                                                                                          2. Depends…. If the outages are below about 2 short power outages per year on average, then no I wouldn’t.

                                                                                                          If it starts to escalate to one outage per month and 25% of them can be blamed on incompetent IT-staff? You’ve reached the point where I am going to install my own diesel generators as those will quickly become profitable.

                                                                                              2. 1

                                                                                                I don’t quite understand. Regardless of the TLS version, if you want to inspect https you need to intercept and decrypt outgoing https traffic via a middlebox. This applies to regular https just as it applies to DoH. If you are required to secure your network inspecting encrypted traffic, you will continue to do so just like you’ve always done. In this sense, DoH is even less intrusive than, say, DoT because your standard https intercept proxy can be adapted to deal with it.

                                                                                                1. 1

                                                                                                  Wasn’t the goal of TLS 1.3 to make interception impossible? I am certain that was one of the major goals, but I didn’t follow through the RFC’s development.

                                                                                                  How would interception work? With ESNI in TLS 1.3, the client does a DNS lookup to retrieve the key to encrypt the ESNI request with. The middlebox couldn’t decrypt the ESNI and generate a certificate by the local trusted CA because it doesn’t know the hostname the client wants to access. So now… a middlebox will also have to be a DNS server so it can capture the lookup for the ESNI key, generate a fake key on demand, and have it ready when the TLS connection comes through and is intercepted?

                                                                                                  This is getting quite complex, and there may be additional middlebox defeat features I’m not aware of

                                                                                                  1. 1

                                                                                                    No, the basic handshake can still be intercepted similarly to TLS 1.2, so that’s not a problem with 1.3.

                                                                                                    ESNI might be a slightly different issue. But you could just take a hardline stance and drop TLS handshakes which use ESNI and filter the ESNI-records (with a REFUSED error?) in your resolver. If you need to enforce TLS intercept, you will need to enforce interceptability of that traffic and that might mean refusing TLS handshakes which use ESNI. But I heaven’t read the RFC drafts yet, so there might be easier/better ways to achieve this. In any case, none of this should be a deal breaker. TLS intercept proxies have always been disruptive (e.g. client certificates cannot be forwarded past an intercept proxy) and this will apply to ESNI just as it has done to past aspects of TLS.

                                                                                                    What I feel should be clear is that none if this will suddenly turn existing practices impossible. Restrictive environments will continue to be able to be restrictive, just as they have in the past. The major difference will hopefully be that we will be safer by default even in open networks, such as public wifis, where a large number of users are currently exposed to unnecessary risks.

                                                                                                    1. 1

                                                                                                      ESNI might be a slightly different issue. But you could just take a hardline stance and drop TLS handshakes which use ESNI and filter the ESNI-records (with a REFUSED error?) in your resolver. If you need to enforce TLS intercept, you will need to enforce interceptability of that traffic and that might mean refusing TLS handshakes which use ESNI.

                                                                                                      I don’t think this is possible. TLS 1.3 means ESNI is a given. If half the internet uses TLS 1.3-only, you have no choice but to support it. AIUI they’ve gone to great lengths to prevent downgrade attacks which will stop the interception.

                                                                                                      I have a contact at BlueCoat and am reaching out to see what the current state is because their speciality is exactly this.

                                                                                                      1. 1

                                                                                                        TLS 1.3 means ESNI is a given.

                                                                                                        Right now, ESNI is not mandatory for TLS 1.3. TLS 1.3 is a complete and published RFC standard. ESNI is only a draft and is certainly not mandated by TLS 1.3. You don’t need to run downgrade attacks to “intercept” TLS 1.3. Intercept proxies simply complete the TLS handshake by returning a certificate for a given domain issued by a custom CA that’s (hopefully) in the client’s trust store. This works just the same for 1.3 as it does for any earlier method.

                                                                                                        1. 1

                                                                                                          Do we know the failure mode is if ESNI is rejected? Everyone wants ESNI for their privacy and browsers will certainly implement it, so it will be more common than not I suspect.

                                                                                                          edit: and thanks, I was still operating under the impression that ESNI was part of the final TLS 1.3 draft. I haven’t taken the time to read through it all and there’s a lot of misinformation out there. I’ve been too busy to dig in deeper, and security is not my day job right now.

                                                                                      1. 16

                                                                                        I found the “Enable DNS over HTTPS” setting in Firefox Developer Edition 69.0b16 (64-bit). It’s not (yet) checked automatically, but if I do check it, I have the option select “Cloudflare (default)” or specify a custom provider.

                                                                                        1. 4

                                                                                          It would be nice if you could specify a few to use in round-robin fashion.

                                                                                          1. 3

                                                                                            That’s called loadbalancing and you can usually solve that either via DNS or by any other HTTPS loadbalancing method.

                                                                                            1. 7

                                                                                              via DNS, you say… :-)

                                                                                          2. 2

                                                                                            I implemented my own DNS-over-HTTP/2 implementation for use at home (RFC-8484) isn’t that hard to implement) when it became apparent that Mozilla was shoving this down our throats. The recent version of Firefox wasn’t using it, this possibly explains why.

                                                                                          1. 2

                                                                                            For Gitea, I can recommend setting up OAuth2 for GitHub and GitLab, it’s not that complicated though needs some fiddling in some edge cases, but it’s worth it IMO. GitLab always feels a bit heavy, no matter what I want to deploy it for… there is just too much stuff crammed in (and I’m not against any of the stuff in particular).

                                                                                            I would advice against Sourcehut, since the author seems to have some personal vendetta against making their software accessible to a wide audience of developers/users.

                                                                                            1. 2

                                                                                              I’ve been picking up Perry Rhodan Neo, a german hard scifi series. I was quite the fan of the original series, though as my sister is trying to get into it too, I realize how incredibly dated the first ones are. The Neo series is quite a well done refresh. If you love scifi, especially hard scifi, you’ll enjoy Rhodan, though German knowledge is basically required since I don’t think there are any up-to-date translations available. It’s a lot of politics, sci-fi science nerd-ery and 5D Spacechess (legitimately since they travel FTL in 5D space).

                                                                                              1. 2

                                                                                                Was there a time that Intel / CISC was implemented on top of RISC with a translation layer in microcode or something? I swear I was told this once but maybe it was an urban legend

                                                                                                1. 3

                                                                                                  Modern x86 is translated into microcode that is very similar to what you’d get if RISC and VLIW got frisky with eachother, IIRC ARM does that for some complicated instructions. Microcode lets you optimize the CPU usage more.

                                                                                                  VLIW would be essentially taking out that translation layer, the compiler has to make use of what the hardware offers.

                                                                                                1. 4

                                                                                                  I would sign in with Firefox. I trust Mozilla much more than Google and I trust Mozilla to work out a solution that preserves privacy beyond simply trusting Mozilla instead of Google.

                                                                                                  1. 1

                                                                                                    I’m a bit torn on Mozilla. I trust them more than Google, Apple, Microsoft, Facebook, etc. but not that much more due to some shenanigans in the recent years with the latest example being their DNS-over-HTTPS push with Cloudflare.

                                                                                                    However, I’d still give them money as it would be one avenue where we could hopefully break the hold that the big names are getting over the internet.

                                                                                                    1. 2

                                                                                                      I don’t think their DoH push is that harmful, the tech could help a lot of people with shady ISPs.

                                                                                                      1. 2

                                                                                                        The problem with the DoH push is that their worldview is extremely US centered, while I in Europe would rather have my (not shady) ISP have my DNS data than yet another big US company (with its usual promises).

                                                                                                        (Disclaimer: I’m a PowerDNS employee.)

                                                                                                        1. 1

                                                                                                          I’m in Europe. I don’t trust my ISP that much; their DNS Server redirects me to a search page run by them, which delivery poor results and incredibly annoying adware.

                                                                                                          I would run my own DoH resolver if necessary, CF is Mozilla’s Point of Trust in the US but I don’t see why that prevents me from doing it myself.

                                                                                                          1. 1

                                                                                                            The point is not so much how a technical person like you will handle it, but how they were planning on introducing it to the average user. It was going to be a kind of a “cookie warning” dialog that people automatically click through.

                                                                                                            They’ve made some amendments since then.

                                                                                                            1. 1

                                                                                                              Personally I would hope it stays opt-in or Firefox will simply try the DHCP configured DNS server.

                                                                                                  1. 8

                                                                                                    I’ve got a notification that the last parts of gear arrived that I need to setup a LoRa TTN Gateway, so that means I can start testing around with it. I plan to use that to have remote temperature sensors relay information to my local hass.io instance and do things with it. LoRa would enable much more lifetime than other solutions like a Node MCU with Wifi.

                                                                                                    I’ve been picking up work on my kernel again, I updated a lot of things to later releases, most of which made a lot of things simpler (the new x86_64 crate in rust can do paging now). One problem has been the lack of a good kernel allocator in rust. There are some slab-based ones but they all have a global lock. So I’ve been tinkering on making an atomic+reentrant allocator to fix that. A feature I’ve been wanting is to enable eternal objects, ie, kernel objects that don’t die, ever. It’s something that Rust doesn’t really like.

                                                                                                    Otherwise, there is some general upkeep I need to consider. My hosting box at hetzner has been running out of IOPS and I’ve been considering to add 2TB SSDs to it and migrate to them.

                                                                                                    1. 1

                                                                                                      I’m loving the “log stdout” part, everything else can basically be ignored.

                                                                                                      1. 2

                                                                                                        That’s definitely an improvement over the syslog situation, at least for our deployments. The native Prometheus export is neat as well; saves having to build an adapter to run alongside for metrics.

                                                                                                        1. 2

                                                                                                          I was partly joking. Not being able to log to stderr or stdout has caused so many problems because it’s basically impossible to debug haproxy without syslog being present (and HAProxy has the annoying tendence to stop logging if syslog hangs up such as happens when the network has a hiccup in an rsyslog sitaution)

                                                                                                          1. 1

                                                                                                            That exporter is one of the oldest: https://github.com/prometheus/haproxy_exporter

                                                                                                            1. 2

                                                                                                              Nope, this is a new, exporter-less endpoint, built into HAProxy itself: https://www.haproxy.com/blog/haproxy-exposes-a-prometheus-metrics-endpoint/

                                                                                                        1. 4

                                                                                                          I really like the new frontend, might take a bit to get used to though… Either way, I think a more twitter-like interface will help users adapt more easily to the network.

                                                                                                          1. 2

                                                                                                            subgrid is desperately needed for some thing I work on, so thanks to Mozilla for making it available! I absolutely love CSS grid.

                                                                                                            1. 3

                                                                                                              I’m moving more infrastructure into Docker. I’ve setup a simple Docker Swarm, build from two VM nodes on the same host, the current main holdup is making docker containers for some apps where I made modifications for personal use (so I also setup a registry)

                                                                                                              Also, I’m waiting on parts to finish my DIY Server Rack Closet (hopefully), then I can make my NAS more tidy and reduce the noise floor a bit.

                                                                                                              Other than that, I’m just being lazy outside work, my commute has gotten a bit too long to be productive during the week.

                                                                                                              1. 2

                                                                                                                I’ve been thinking on using some Docker orchestration (Docker Swarm or something like https://k3s.io/) for personal projects, but building all the infrastructure including mandatory orchestrator-required configuration in each project, seems a lot of work and complext to maintain… But in the other hand, having everything completely reproducible so I can migrate or recreate the server whenever I want might be really helpful.

                                                                                                                How’s your configuration and/or processes for working on your personal projects and deploying them to Swarm?

                                                                                                                1. 1

                                                                                                                  The biggest hurdle to Swarm is IMO the storage problem. I solved it by mounting my NFS storage server on each node in the same location, so a simple bind mount works on all nodes (though I do have to remember not to schedule more than 1 of some services).

                                                                                                                  I largely manage the swarm via Portainer, which has a pretty good interface for dealing with Stacks or individual Services. The node itself is entirely managed by ansible and based on a very vanilla Alpine 3.9 install (basically, it’s Alpine 3.9 + build-deps for docker-compose + python2 + docker)

                                                                                                                  I did consider k3s, but it feels to much aimed at IoT rather than a simple distributed system. K8s is very complex compared to swarm, which was largely 3-4 lines in the terminal to setup, initially and for each node.

                                                                                                                  I setup Drone CI on top of it and plan to outsource some parts of ansible into the CI system, likely managing all non-Docker containers and VMs.

                                                                                                                  There is still a lot on my ToDo list until I’ve got everything setup though.

                                                                                                                  1. 1

                                                                                                                    Does it work nicely with a NFS mount? For what kind of storage?

                                                                                                                    I know Portainer, it’s nice, but there were some perks when I tried it, should give a second shot.

                                                                                                                    k3s is aimed to IoT, but it makes sense to run it on a cheap VPS in terms of memory usage, but maybe not in complexity.

                                                                                                                    In my case, I work daily with Kubernetes, so, it’s all shared knowledge… I’m still considering it, once an issue with cert-manager gets properly solved.

                                                                                                                    Once the full setup is done, a post or something like that would be cool :)

                                                                                                                    1. 2

                                                                                                                      It works pretty much as I expected; pretty well. However I would warn against NFSv3, NFSv4 is an absolute must if you like your data.

                                                                                                                      My setup isn’t exactly a small or cheap VPS, it’s a larger dedicated server that I got from the Hetzner auction service for relatively cheap. I feel like Docker Swarm is a good choice for my use cases (I feel k8s is complicated and I don’t like software that takes me more than 1 day to explore all the configuration options I could possibly care about)

                                                                                                                      1. 2

                                                                                                                        Yep, k8s is pretty complicated, but it’s powerful. Anyway, it’s useful when you have a bunch of nodes and different applications running in it, not so much for a single node/two node setup.

                                                                                                                        Thanks!

                                                                                                                    2. 1

                                                                                                                      Can I ask a stupid question about this? What kinds of things are you storing? I’ve managed to work in big infrastructure thus far staying relatively ignorant about containers. In my day job we use regular EC2 instances for just about everything, and from a distance it feels like there’s an awful lot of extra complexity this model brings to the picture.

                                                                                                                      1. 3

                                                                                                                        The largest part of the storage server is my nextcloud instance (about 400GB). Everything after that is largely other data the containers have massed up, ie, sqlite databases, log files, config files, cached data, etc. My mastodon instance is still sitting on it’s own disk and database, which would make it the second largest things once it’s moved over (60GB).

                                                                                                                        Docker isn’t terribly complicated, neither is Swarm. Atleast when compared to K8s and friends. Portainer makes it a lot easier too.

                                                                                                                        I don’t use EC2, I have a large dedicated server (128GB, 6c/12t), so splitting up resources properly is important compared to letting AWS do it (plus it’s a neat learning experience to do it manually)

                                                                                                                        1. 2

                                                                                                                          Oh I totally agree! And being able to actually stand up servers and the like from whole cloth is a dying art these days :)

                                                                                                                          Do you find that your masto instance consumes more than its fair share of resource? When i ran one I found the stack to be incredibly heavyweight.

                                                                                                                          1. 2

                                                                                                                            It’s certainly no lightweight but the performance has been fairly decent (atm it’s handling about 6r/s on 2 CPU cores and 4GB RAM without any issues)

                                                                                                                1. 7

                                                                                                                  I was able to get them back by disabling xpinstall.signatures.required. It might require a nightly version to actually work? Of course, you’d want to turn it back on once it’s fixed…

                                                                                                                  1. 6

                                                                                                                    As you say, it only seems to work on nightly, unfortunately. Most vexing!

                                                                                                                    1. 3

                                                                                                                      i don’t get why on earth these switches aren’t present in the release versions. this feels like the “caution, hot contents!” prints on takeout cups -.-

                                                                                                                      edit: it works on 60.6.1esr (64-bit) though..

                                                                                                                      1. 4

                                                                                                                        because the lambda person is taught to go around this warinings to install bad extension.

                                                                                                                        1. 3

                                                                                                                          then they are entirely responsible for their misery. i never got the trend of making everything.. more proof than fool proof.

                                                                                                                          displaying a warning which can’t be deactivated would be fine.

                                                                                                                          1. 3

                                                                                                                            No, users are not responsible for products having footguns. It’s unfair to expect non-technical users to understand technical issues, especially when these issues may be misrepresented in a hostile context (e.g. “You have a virus! It’ll murder your googles unless you tick this checkbox and click OK”).

                                                                                                                            Most people don’t need to know what a certificate is. They just want to get on with their life, and not have their entire digital life destroyed, because a product had a “harm me” checkbox.

                                                                                                                            1. 5

                                                                                                                              No, users are not responsible for products having footguns. It’s unfair to expect non-technical users to understand technical issues, especially when these issues may be misrepresented in a hostile context (e.g. “You have a virus! It’ll murder your googles unless you tick this checkbox and click OK”).

                                                                                                                              Is it really a footgun when it’s hidden somewhere in about:config where the first thing you see is a warning about not to touch this? Is it a footgun if I staple my toes with a nail gun, or was I too dumb by not wearing the right shoes?

                                                                                                                              Most people don’t need to know what a certificate is. They just want to get on with their life, and not have their entire digital life destroyed, because a product had a “harm me” checkbox.

                                                                                                                              I seem to be rather alone with this, but I expect some basic knowledge and common sense from people using tools. You need a drivers license, you are supposed to read the manual. These things should be taught in school. Alas, they aren’t, but that doesn’t absolve one from knowing about the technology one is using.

                                                                                                                              1. 2

                                                                                                                                Is it really a footgun when it’s hidden somewhere in about:config where the first thing you see is a warning about not to touch this?

                                                                                                                                Yes. “You have a virus! It’ll murder your googles unless you go to about://config, click OK to all the scary warnings (they’re there because of the virus) and then install this.”

                                                                                                                                These things should be taught in school.

                                                                                                                                People are culturally trained to not care about school, so even if it was somehow taught it there it wouldn’t have too much of an effect.

                                                                                                                                Also, where is the manual? How does one even teach people to recognize scams, and then keep them continually up to date on the latest scams for their entire lives? You can’t. The scam that worked in 1999 is nothing like the scams that exist today. The reason that you or I can tell these things is because we do work related to computers and stay up to date on the latest happenings (as evidenced by the fact that we’re on Lobsters). One can’t reasonably expect any “normal” person to do the same.

                                                                                                                                1. 4

                                                                                                                                  Yes. “You have a virus! It’ll murder your googles unless you go to about://config, click OK to all the scary warnings (they’re there because of the virus) and then install this.”

                                                                                                                                  There are already a plethora of ways to do this. The people that can be manipulated to disable random settings in about:config can just as easily be manipulated to run random code in the console, or install random .exe files from haxx0r.ru

                                                                                                                                  1. 1

                                                                                                                                    So why add one more? The browser is a really lucrative thing to pwn. Lots of accounts can be exploited for profit in easier ways than anything else that could be on the system.

                                                                                                                                  2. 3

                                                                                                                                    i just have a problem with dumbed down versions of everything. it teaches people that they are too stupid, and cannot be trusted with anything. things break and you learn from it. the internet isn’t disneyland, every of these measures to “protect” people from themselves can be worked around. just look at the myriad of shady apps for mobile platforms. additionally: maybe people who believe win-xp themed fake popups telling about a virus should just not use the internet, it will never be safe enough.

                                                                                                                                    tinfoil mode: making the internet feel cozy and safe is to protect business interests, as it is easier to sell things in a walled garden where people aren’t expected to think.

                                                                                                                                    1. 4

                                                                                                                                      Civilization advances by extending the number of important operations which we can perform without thinking about them.

                                                                                                                                      It’s not about stupidity, it’s about ignorance. I can use a light switch without knowing how the electricity was generated, the the glass was blown (let alone the LED manufactured), etc. I can make dinner without knowing how the knife was forged, the grain was harvested, the pasta was shipped, etc. The difference between the browser and all the rest of these household objects is that criminals can become millionaires by hijacking them at scale. I earn my paycheck by understanding browsers and how to make things that go in them so, yeah, for myself I want a browser with lots of places I can tinker with them and break things because sometimes I need to break things to make them work. I love that the web is mutable and weird and it’s easy to shift from consumption to production, to peel back the layers and see how everything works. But I also know that very few users come to the web like a developer does, or have any interest in becoming one. They’ve got a lot of other important, private, or sensitive things to do that insecure systems put in jeopardy. If my light flickers, my power is out, my knife breaks, my pasta is spoiled, my dinner is ruined - none of these things happen because I’m immorally ignorant of how they work.

                                                                                                                                      1. 1

                                                                                                                                        I can make dinner without knowing how the knife was forged, the grain was harvested, the pasta was shipped, etc.

                                                                                                                                        still, you’ll have some sensible ideas of how each of these things are done :)

                                                                                                                                        The difference between the browser and all the rest of these household objects is that criminals can become millionaires by hijacking them at scale.

                                                                                                                                        well.. :P

                                                                                                                                        […] But I also know that very few users come to the web like a developer does, or have any interest in becoming one. They’ve got a lot of other important, private, or sensitive things to do that insecure systems put in jeopardy.

                                                                                                                                        yes. i still don’t see a problem with allowing users to flip switches in about:config after displaying them a warning sign. if they ignore the warning and get bitten it’s on their account.

                                                                                                                                      2. 2

                                                                                                                                        i just have a problem with dumbed down versions of everything. it teaches people that they are too stupid, and cannot be trusted with anything.

                                                                                                                                        If the observed fact that warnings get routinely ignored means that users are stupid to you, then I guess users are stupid (or that software, not even necessarily Firefox, pops up far too many warnings that are too complicated for them to understand or even flat-out false alarms).

                                                                                                                                        1. 1

                                                                                                                                          the too many warnings problem is clearly a software problem, but if i “click them away” and something bad happens, i’m still the reason that it happened. nobody else to blame.

                                                                                                                                          imho unsafe settings could be displayed in firefox a bit like private mode (maybe a little bit redder or something), so that one knows this is not safe. i have no clue about the psychological effects, but know from myself that modal popups are pissing me off, and in “pissed off mode” the warning itself is brushed off as bs, because the modal making my life unreasonably worse.

                                                                                                                                  3. 3

                                                                                                                                    I have very mixed feelings about this. On one hand, I agree with you. On the other hand, I feel like a considerable group of more advanced “power users” is consistently left out in the cold because ever user is always treated like my grandmother.

                                                                                                                                    I wonder if we can’t come up with something better than a yes/no dialog for these sort of things. Like allowing users to continue only after typing the text “dangerous”, sort-of similar to GitHub’s repo delete feature. Or perhaps something else entirely which satisfies both demands.

                                                                                                                                    1. 1

                                                                                                                                      We (power users) need to remember that compared to literally billions of web users, we’re a tiny tiny minority.

                                                                                                                                      In case of Firefox, the Nightly build is the solution for power users (plus, it has much cooler icon).

                                                                                                                                      1. 4

                                                                                                                                        A nightly build is also less stable. Power users deserve a stable browser too. And normal users deserve the opportunity to become power users.

                                                                                                                                        Software should treat adults … like adults.

                                                                                                                                        1. 3

                                                                                                                                          I don’t know if it’s such a “tiny tiny minority”. I suspect there are a lot of people who aren’t IT professionals for a living, but are certainly more clued up than your grandmother on these matters. I will admit I don’t have any hard data to back this up, just an observation from the people around me.

                                                                                                                                          And even for non-technical users there can be good reasons to bypass these sort of warnings, as this current Firefox problem demonstrates.

                                                                                                                                    2. 1

                                                                                                                                      Allowing the average user to kneecap themselves is not productive behaviour for widely installed software. That’s how you get people to install 20 toolbars which slow down everything.

                                                                                                                                      If you want to live without the warning, there is the Beta, Developer and Nightly editions of firefox, which come with the switch to disable signature verification.

                                                                                                                                      1. 2

                                                                                                                                        Allowing the average user to kneecap themselves is not productive behaviour for widely installed software. That’s how you get people to install 20 toolbars which slow down everything.

                                                                                                                                        the problem with this approach is that this way people will never learn. about:config says “warranty void”, so i don’t see a problem.

                                                                                                                                        If you want to live without the warning, there is the Beta, Developer and Nightly editions of firefox, which come with the switch to disable signature verification.

                                                                                                                                        which has more telemetry built in, afaik. this is fine for development and debugging, but i don’t want that for my day-to-day use :)

                                                                                                                                        1. 1

                                                                                                                                          You can also disable more of the telemetry via about:config, though you’d void your warranty.

                                                                                                                                  4. 2

                                                                                                                                    Confirmed working also on 60.6.1esr (64-bit), which is the Debian default version in the apt repos

                                                                                                                                  5. 1

                                                                                                                                    While the switch is present in the version Debian has in the repositories, toggling it doesn’t fix the problem. It allows to install AddOns again, but they are dysfunctional, e.g. NoScript and uBlock Origin have buttons in the menu bar that have no icon and do nothing when clicked.

                                                                                                                                    1. 1

                                                                                                                                      I think you’re right it requires nightly. Not working for me on FF 66.0.3 release (installed from the Arch Linux package).

                                                                                                                                      In fact, I was a bit embarrassed to find I already had signature verification disabled (I was doing some WebExtension development ~18 months ago…)

                                                                                                                                      1. 1

                                                                                                                                        After getting a error message and seeing my addons disabled I tried to set it to true on Android phone and there it seemed to have worked. My installed addons are enabled again. Firefox 66.0.2 on Android.

                                                                                                                                      1. 20

                                                                                                                                        Well, this is infuriating. I hate that my browser just became essentially useless to me because someone at Mozilla messed something up. Anyone know if there’s a way to opt out of the extension verification stuff?

                                                                                                                                        1. 11

                                                                                                                                          I’m seriously considering just switching to Chromium (ungoogled-chromium maybe?) as a workaround. I don’t feel like Mozilla is doing too well in general with regards to being pro-user and pro-privacy lately;

                                                                                                                                          • There’s this issue, leaving everyone vulnerable to tracking and disabling protections for tor users.
                                                                                                                                          • The fact that this feature exists at all, and the only supported way to disable signing requires nightly, takes a lot of control out of users’ hands.
                                                                                                                                          • Mozilla have bought companies with closed source products (such as pocket), integrated them into Firefox, promised to open-source those products, and just never open-sourced them, leaving Firefox still with built-in integrations with potentially privacy-breaching inauditable closed-source products.
                                                                                                                                          • They have plans to move away from DNS, where a query first consults my OS (and its hosts file) and then consults my ISP which is a norwegian company following strong privacy laws, to just sending queries directly to a random American company which follows the US’ seemingly non-existant privacy laws.
                                                                                                                                          • It seems likely that they’ll move from IRC to discord or slack, which will be pretty bad if it happens (though this point is invalid if they end up moving to something free and open source). They should at least have come out and clearly stated that they’re not moving to a closed-source solution.
                                                                                                                                          • And, well, Chromium just has better performance on machines I’ve tested it on; having a worse experience for a good cause is worth it, but having a worse experience just to support a company which doesn’t really stand for anything might not be.

                                                                                                                                          I honestly really want to support Mozilla, and to do my small part in avoiding a complete browser monopoly by not using chromium, and I really don’t want to support Google. Mozilla just does so many stupid things which flies in the face of the values they claim to hold.

                                                                                                                                          1. 27

                                                                                                                                            I definitely plan to stay with Firefox. They are sometimes failing, but at least they’re trying to fight. There’s a saying, that “if somebody’s not failing, they’re not trying hard enough”. Whereas Chrome has a fundamental conflict of interest against many user protection mechanisms, because paid by Google Ads.

                                                                                                                                            1. 7

                                                                                                                                              You’re probably right. I’m on chromium right now, but I will probably honestly end up switching back to Firefox when this whole thing is over. It just sucks that Mozilla has to put themselves in the position of being the least bad of two evils, instead of just being plain good.

                                                                                                                                              1. 4

                                                                                                                                                It just sucks that Mozilla has to put themselves in the position of being the least bad of two evils, instead of just being plain good.

                                                                                                                                                You’ve hit the nail on the head. I just want a browser that’s privacy-respecting and good.

                                                                                                                                              2. 5

                                                                                                                                                Mozilla is also paid by Google Ads.

                                                                                                                                              3. 20

                                                                                                                                                Can you not be a drama llama? They goofed up. They will probably fix it soon. So you are without addons for a few days.

                                                                                                                                                As for their decisions, they are clearly straddling a line between purity and a little bit of the dirty stuff to make it more convenient for the non-0.1% of users who are ‘technical’. Meanwhile Google is ACTIVELY TRYING TO FUCK YOUR SHIT UP to maximise their control and profit.

                                                                                                                                                Perfect is the mortal enemy of the good.

                                                                                                                                                1. 2

                                                                                                                                                  I think the problem here is that not only do they enforce the signing, but they also make it impossible for the user to turn it off, unless the user downloads non-stable or non-official versions of software, taking control out of the hands of the user.

                                                                                                                                                  Sure, Google is worse, but what excuse does Mozilla have for the workaround (e.g., disabling the feature) not working on stable versions of Firefox? I see that as the very definition of the lesser of the two evils.

                                                                                                                                                  1. 3

                                                                                                                                                    I think I’ve seen some article long ago, basically saying how users will do everything they’re told by a website if this means they get to watch one more funny cat video - including changing settings in about:config, in OS, etc. Unfortunately I can’t seem to find the article with google nor ddg.

                                                                                                                                                    1. 3

                                                                                                                                                      This rings a bell, I read that too. I think the term you are looking for is “dancing pigs”. The Wikipedia page for dancing pigs cites a few sources for it. The one I think you and I both read is probably one of the Bruce Schneier articles. Wiki suggests the first publicly available written thing using the term was a chapter in a book about the Java security model. Which is kind of funny when one thinks about it because it’s hard to think of a piece of technology that did a worse job of what it was supposed to do than the Java security model.

                                                                                                                                                      1. 1

                                                                                                                                                        You’re saying the users are the only one gullible here?! What about the developers? A couple of folks at Mozilla and Google tell devs to trust LetsEncrypt with all your SSL needs, and pretty much every single developer restricts access to their websites now through LetsEncrypt now. Talking about the folks being gullible!

                                                                                                                                                        1. 1

                                                                                                                                                          Hm, I see now that the way I wrote it may be seen as more ambiguous than I expected! :) Basically, what I meant, and what the article I mention tried to convey AFAIR, was that as a developer, one sometimes needs to protect users from themselves; in this case, I guess the “[Mozilla] mak[ing] it impossible for the user to turn [addon signing verification] off” decision might have been to protect users from themselves. That is, to protect users from being conned into disabling the verification feature “to see this one funny cat video”, and installing some malware addon.

                                                                                                                                                          As to LetsEncrypt, I don’t think I want to engage in a discussion completely (in my opinion) unrelated to the original post/article :)

                                                                                                                                                    2. 1

                                                                                                                                                      this isn’t the only thing they’ve done. it’s part of a longer trend of user-hostility which tells us that the mainstream web will not be compatible with freedom, so long as google controls what standards are implemented.

                                                                                                                                                    3. 5

                                                                                                                                                      Mozilla just does so many stupid things which flies in the face of the values they claim to hold.

                                                                                                                                                      Yeah, remember that “auto install” of the LookingGlass/Mr.Robot thing a while back (end of 2017 I think…)?
                                                                                                                                                      wtf Mozilla. I am going to check out some alternatives.

                                                                                                                                                      Anyone here tried Brave or Vivaldi? If so, any good?

                                                                                                                                                      1. 3

                                                                                                                                                        Been working with Brave and Firefox for quite some time now.

                                                                                                                                                        Brave is less polished and is missing quite a lot of sync-related-features I tend to use quite often on firefox. But the fact that firefox broke at a critical moment on my smartphone, right this morning, was the turning point.

                                                                                                                                                        I haven’t tried Vivaldi as extensively though.

                                                                                                                                                      2. 3

                                                                                                                                                        The fact that this feature exists at all, and the only supported way to disable signing requires nightly

                                                                                                                                                        https://twitter.com/SwiftOnSecurity/status/1124545069078536192

                                                                                                                                                        There’s no solution here that doesn’t involve making normal users more vulnerable to malware. It’s been tried.

                                                                                                                                                        Chrome has had similar problems in the past.

                                                                                                                                                        They have plans to move away from DNS …. to just sending queries directly to a random American company

                                                                                                                                                        Nobody has said that it will be a random American company. Mozilla’s testing this feature out with Cloudflare. I suspect this will be pretty configurable if it becomes an actual thing, and probably more local.

                                                                                                                                                        It seems likely that they’ll move from IRC to discord or slack

                                                                                                                                                        Mozilla’s moving away from IRC, but from the requirements here it doesn’t seem like slack or discord are likely solutions.

                                                                                                                                                        1. 2

                                                                                                                                                          Nobody has said that it will be a random American company. Mozilla’s testing this feature out with Cloudflare.

                                                                                                                                                          Cloudflare is the random American company I’m talking about.

                                                                                                                                                          1. 2

                                                                                                                                                            Right, operative term being “testing this feature out”. There’s no indication that if this feature becomes a thing it will be only cloudflare that it uses. There’s just a lot of FUD around it.

                                                                                                                                                            My comment is not correcting “random American company” to cloudflare, it is correcting your statement about Mozilla plans around this. They have not ever stated that this is the plan. It’s just what they’re testing out, because you have to bootstrap an ecosystem somehow.

                                                                                                                                                        2. 1

                                                                                                                                                          Mozilla isn’t moving away from DNS, you can disable DoH in the network settings and you can set any other DoH endpoint you want in the same dialog (so for example, you could set your Norwegian ISP or no DoH at all).

                                                                                                                                                          The Pocket extension is open source to my knowledge, I do recall a github repo floating around. What isn’t open source (yet) is the backend.

                                                                                                                                                          1. 4

                                                                                                                                                            Sure, it will probably be possible to disable DoH, but how many non-American Firefox users will know to do that, compared to how many will not even know it’s something they have to worry about and send all their queries to a US corporation?

                                                                                                                                                            The pocket extension is open source, but it’s the backend which is interesting, and it’s the backend they promised to open-source a long time ago. (Look at this comment from a Mozilla employee 2 years ago: https://www.reddit.com/r/firefox/comments/5wio45/mozilla_acquires_pocket/deadcf7/ - that didn’t say that the Pocket extension would become open source, but Pocket itself.)

                                                                                                                                                            1. 1

                                                                                                                                                              To my knowledge the current default and to keep it disabled, the DoH provider setting currently defaults to only using standard DNS as well, I don’t know of any plans to change that, Mozilla is still very early in testing the waters on how to deploy it.

                                                                                                                                                        3. 2

                                                                                                                                                          See the description of this post for a workaround.

                                                                                                                                                          1. -9

                                                                                                                                                            Well, this is infuriating. I hate that my browser just became essentially useless to me because someone at Mozilla messed something up. Anyone know if there’s a way to opt out of the extension verification stuff?

                                                                                                                                                            LOL, says a person who’s website is “protected” by a time-bombed HTTPS and is unavailable via HTTP.

                                                                                                                                                            You are aware that your website suffers from the same issues that you appear to condemn in this very comment? That it’s up to external third parties on whether or not the user is allowed to see it, because you decided to cave in to their pressure to “secure” your static website, and yourself made a choice to prohibit folks from accessing it via HTTP through your own policy?

                                                                                                                                                            How are you then act surprised that Mozilla does same?!

                                                                                                                                                            1. 6

                                                                                                                                                              Well firstly, my website is not a tool that people depend on to do work. Firefox is. Secondl, I have automated systems in place to renew the SSL certs & get warned when they’re near to expiry. Thirdly, if you had my site open & the certs somehow expired, you could still see the content; Firefox just disabled a bunch of functionality while it was running without giving me any chance to intervene. Finally, if a website’s certificates are expired, you still have the ability to say “show me anyway”; there doesn’t seem to be any ability to do the same with stable Firefox.

                                                                                                                                                              Glad to see you’re enough of a fan of mine to look into how I configure my site though!

                                                                                                                                                              1. 2

                                                                                                                                                                But how’s a website different from a tool? Firefox is still made by people just like you. The fact that one can click “show me anyway” on your website is merely omission on the part of site’s operator to not install HSTS. With proper HSTS, the user is guaranteed to not have any way to access your site even if you decide to cancel your https policy. There is no way to intervene, either, if HSTS is setup correctly. If you click reload and a new connection has to be established, pretty certain things won’t work no more, either.

                                                                                                                                                                “Automated systems in place to renew SSL certs”? Are they autonomous and self-contained, or do they depend on any third parties? Are the third-parties they depend upon by any chance related to the very same party that caused the incident at stake? Isn’t Mozilla the biggest backer behind LetsEncrypt? This has got to be a joke! The most classic example of #TooBigToFail!

                                                                                                                                                                1. 2

                                                                                                                                                                  Firefox is only a tool you depend on because people serve websites which require a modern browser to be usable. HSTS contributes to this monoculture.

                                                                                                                                                                2. 3

                                                                                                                                                                  HTTPS is a bit different; with a website, you’re inherently relying on someone else paying the bills for the server and domain name continuously anyways, and if they don’t, you can’t use their website even if it”smnot HTTPS. Relying on the owner to renew their certs too doesn’t really change anything. If you want to have access to a website without relying on anyone else, you need to download it and access it locally, whether it’s HTTP or HTTPS.

                                                                                                                                                                  There’s no such expectation for addons I have downloaded to my personal machine which don’t inherently need to rely on any third-party.

                                                                                                                                                                  1. 6

                                                                                                                                                                    This is a personal attack and not something that contributes to the conversation.

                                                                                                                                                                    1. 0

                                                                                                                                                                      How’s something a personal attack if it applies to pretty much every site operator nowadays? The comment purposefully doesn’t even contain any PII, either.

                                                                                                                                                                      1. 2

                                                                                                                                                                        There are better ways of discussing the merits and problems involved with the https certificate system than dismissing what someone said with “LOL, says a person who [..]” and doubting the person’s sincerity with “issues that you appear to condemn”.

                                                                                                                                                                        1. -1

                                                                                                                                                                          the dismissal or questioning of their sincerity is something you’re adding with your interpretation. it doesn’t follow from the parts you quoted.

                                                                                                                                                                          maybe his goal was not to discuss the merits and problems of the https certificate system, but to actually lessen the spread of this scourge.

                                                                                                                                                                      2. -3

                                                                                                                                                                        Pointing out hypocrisy is a good tool when discussing moral issues.

                                                                                                                                                                      3. -4

                                                                                                                                                                        Good post, sad to see it got swarmed by haters.

                                                                                                                                                                    1. 8

                                                                                                                                                                      Does the obfuscated Pocket add-on bundled with Firefox count as an extension?

                                                                                                                                                                      1. 9

                                                                                                                                                                        The Pocket Addon is open source and on github, not obfuscated to my knowledge other than minified.

                                                                                                                                                                        1. 1

                                                                                                                                                                          Pocket’s server-side is not yet open, so you can’t self-host the service. There is Wallabag for those who want a similar self-hosted service.

                                                                                                                                                                        2. 2

                                                                                                                                                                          Maybe this move will accompany a release of Pocket source.

                                                                                                                                                                        1. 9

                                                                                                                                                                          The problem with libre chat applications with federation is that they suck. And that’s an honest statement. Matrix is major suck (source: I tried to run a small chat server for a community of 20 people using matrix).

                                                                                                                                                                          IRC on the other hand sucks because it has barely any features. Anything beyond the most basic and simplest requirements will in turn require you to run atleast several auxiliary services (like NickServ, which exists on almost every IRC network and none of them seem compatible).

                                                                                                                                                                          Any chat service that wants to compete with Discord for the place to communicate about FOSS on, needs to provide the same features as Discord (or atleast a very decent subset of the features) with almost 0 friction. Because that is what Discord offers.

                                                                                                                                                                          The End-Users, who will be the primary consumers of your community chat, do not care about restrictive ToS, lack of control or alternative clients. They want to click a link that opens a webpage and it just works intuitively. Ideally they already opened the app and it works out the details in the background. The admins need to be able to easily and effectively moderate the spaces. The returning users will want a chat log built into the app, something they can easily search. Etc etc.

                                                                                                                                                                          What they do not want is clicking a link and the app immediately trying to get them to sign up to something, patronizing them in the meantime by explaining what a federated service is and that’s why they need to sign up. We have spent decades to teach users not to sign up to random crap services. They don’t want the interface to be non-intuitive (which is an area in which Matrix seems to try to outcompete GNU projects). They do not care if the client is FOSS or not, they want it to immediately work for them.

                                                                                                                                                                          In short, if you want FOSS projects to use FOSS chatops, you need users to pick the FOSS chat app out of practical reasons, not ideological ones. Very few people care about software licensing ideology.

                                                                                                                                                                          1. 3

                                                                                                                                                                            I have been very happy using Discord for the sole instance I frequent (dealing with analog photography). Everything, from the on-boarding to the clients, have been very smooth.

                                                                                                                                                                            I sympathize with the sentiment expressed in the linked argument, but the arguments put forth are only ideological. None of the issues raised in the recently posted article about chat at Mozilla (accessibility in the sense of ease access, moderation of harmful content, ease of use for moderators) are addressed.

                                                                                                                                                                            1. 10

                                                                                                                                                                              I sympathize with the sentiment expressed in the linked argument, but the arguments put forth are only ideological.

                                                                                                                                                                              Perhaps, but… it matters? I mean, yes, it ideologically offends me that I can’t open the developer console in my web browser when I’m using Discord without violating their terms of service, but it is also a practical problem that it doesn’t work well on my laptop, which is not cutting-edge but certainly not out of date by any reasonable measure. It’s not an ideological problem when I ask the Discord devs to fix bugs regarding how their software scans and rescans my video devices hundreds of times a second, wasting CPU time and battery, and am told that I shouldn’t be prying into how the software works. It’s not an ideological problem when I offer them a patch for the issue and am told I’m violating the ToS by modifying the client.

                                                                                                                                                                              Discord is anti-user, plain and simple.

                                                                                                                                                                            2. 4

                                                                                                                                                                              I don’t understand what Matrix is doing.

                                                                                                                                                                              Signal is widespread but doesn’t federate. Okay. Their source is FOSS, though, so prop up a hosted version and put a stake in the ground as a commitment to federate. You’ve just saved all the effort it’s taking to develop Matrix, and can pour 100% of your energy into your value-add on top of what Signal already offers.

                                                                                                                                                                              Heck, I don’t even understand why people are wasting time writing new clients to go along with their bespoke protocols. Suppose that we accept that there’s something magical in the protocol you’re developing. It still doesn’t follow that you need to split your time, attention, and energy between the protocol and creating new client for it. Take the Signal client and wire it up so that it speaks $WHATEVER.

                                                                                                                                                                              1. 8

                                                                                                                                                                                I don’t think SIgnal is a great example. moxie has been pretty hostile in the past towards people who try to fork signal, and he has a history of resisting some pretty, imho, important features (e.g. being able to use signal without google crap services) for silly reasons. He did ultimately accept that feature, but not without a lot of struggle.

                                                                                                                                                                                1. 0

                                                                                                                                                                                  So hostile that he decided to give away the source code?

                                                                                                                                                                                  Besides you’re talking about something completely different. I’‘m not suggesting that people try to work with Signal to get features upstreamed. I’m explicitly saying to do the opposite. Fork it. The code already exists. Why is it a good idea to throw away a team’s scarce development resources just to reimplement a chat UI?

                                                                                                                                                                                  1. 2

                                                                                                                                                                                    Why is it a good idea to throw away a team’s scarce development resources just to reimplement a chat UI?

                                                                                                                                                                                    Well, as someone else pointed out, Signal really isn’t a replacement for IRC/matrix/whatever. They really aren’t even close to being the same thing..

                                                                                                                                                                                    1. 2

                                                                                                                                                                                      Also Signal’s Desktop UI (rather than it’s mobile UI) is actually pretty bad. It takes a long time to start up in a way I haven’t noticed on the mobile android app. I mostly use it on my phone rather than my desktop computer, so I dont mind too much, but I wouldn’t actually suggest forking the signal UI to make other chat clients.

                                                                                                                                                                                      1. 0

                                                                                                                                                                                        I’d say we’re going in circles here, but we’re three messages deep and never actually left where we started.

                                                                                                                                                                                        That person’s quibble is about the use of phone numbers. For the third time now: Why ignore an existing, proven, solid codebase unless you had to?

                                                                                                                                                                                        If you don’t like Signal’s use of phone numbers or something else about the Signal protocol, then don’t use the Signal protocol. Wire it up to the protocol you’re using. There seems to be no good reason for pursuing the totally-from-scratch strategy that folks like Matrix are going after.

                                                                                                                                                                                        1. 2

                                                                                                                                                                                          I never said anything about phone numbers, maybe you replied to the wrong comment? It’s not really ‘proven’ for the purpose of replacing something like IRC, matrix, etc… In fact, it’s not at all the same, as I mentioned previously.

                                                                                                                                                                                          1. 0

                                                                                                                                                                                            This is getting pretty exhausting.

                                                                                                                                                                                            In addition to actually reading my comments before replying to them, maybe you should read your own and the comments from the other people you’re citing for support.

                                                                                                                                                                                            I’m done with this thread.

                                                                                                                                                                                            1. 1

                                                                                                                                                                                              I’m done with this thread.

                                                                                                                                                                                              Good. Because clearly the thread was done with you a long time ago when you stopped comprehending it.

                                                                                                                                                                                              1. 0

                                                                                                                                                                                                What’s the point of a message like that?

                                                                                                                                                                                  2. 6

                                                                                                                                                                                    I don’t see signal as a replacement for IRC at all. Signal uses phone numbers as identifiers - it’s a replacement for text messaging with people you know well enough to give your phone number to. I would actively not want to use a chat system tied to my IRL phone number to talk with people on the internet.