1. 2

    Spoiler: micro-benchmarks comparing rarely used instructions. As ever, if you really care then test on your own workload.

    1. 3

      I still can’t get why there’s no competitive implementation of Org, especially without Emacs dependency.

      1. 1

        There’s an Android App. I think there are some Vim implementations?

        1. 1

          While Turing-completeness means one can do anything in one language you can in another, it doesn’t mean that all languages are equally productive, and it doesn’t mean all environments are equally productive. Emacs and Elisp make a great, productive pair.

          Also, among Org Mode’s features are the ability to execute Elisp with Emacs: a compatible implementation would thus need to include a whole Emacs.

          1. 1

            It’s not that hard to imagine using a subset of org mode that doesn’t include executing elisp. Most org files I’ve seen don’t even have elisp. Perhaps you could write an article on how to use elisp with orgmode for good.

        1. 18

          The entire debate about systemd fascinates me as an expression of FLOSS culture. The way I see it, systemd is a boon for two classes of users - the few who manage huge amounts of “machines” (i.e. “the cloud”), and the many who use Linux as a workstation or on a laptop, where they seldom have any reason to bother about how a service starts, or is scheduled.

          The class of users who don’t like systemd seem to be the Mittelstand - the sysadmin of a few dozen machines, the user of a VPS who manages a lot of their own services to participate in the open web. They’re reliant on the “folk wisdom” of their own experience, and the substrate of blog and forum posts explaining how to set stuff up. Having to learn a new way of dealing with this stuff feels unnecessary - it worked fine before! The issues that systemd is set up to deal with have never impacted them.

          1. 16

            Disagree. I am exactly a user of all 3 groups and

            • had a few problems with systemd in the cloud that wouldn’t have happened with other init systems
            • I don’t care about startup time anywhere, but I also don’t use autoscaling
            • my laptop shuts down slower now
            • if you have bare metal, the 20s saved with systemd don’t compare to the 3min RAM check…

            Overall systemd solved all the problems I never had. Not as a laptop user, not as a small-scale system admin, but ok - I didn’t work with cloud stuff before it happened.

            1. 4

              You shut your laptop down? I haven’t turned a laptop off (until I sell it) in a decade. They crash from time to time, and restart for updates, but I can’t think of a reason to turn one off unless you’re selling it or putting it into storage.

              1. 2

                work laptop: true, I usually don’t shut it down

                my private laptops: I use them once or twice a month, why would I keep all 3 (one is very old and only used every few months) of them running all the time?

              2. 1

                Who runs a RAM check in the modern world? Seems like a complete waste of time to me.

                (Feel free to convince me otherwise - I’m always open to persuasive arguments!)

                1. 10

                  Even without a RAM check, a typical HP server still takes a good minute or two to POST and then prepare all of the firmwares for things like HBAs, NICs, sensors, RAID arrays etc before you even get as far as the operating system bootloader. I imagine the same to be true on most “server-grade” tin.

              3. 13

                I am in “the user of a VPS” category myself and I use Debian. I had some experience of writing init scripts and I am glad I don’t have to do this anymore.

                I also happen to SRE a huge Linux fleet at work, which uses a custom task supervisor written before systemd.

                I am all for more diversity and experimentation in init systems (e.g. one written in Rust would be great to experiment with, given a well-defined format for service files).

                1. 3

                  I also happen to SRE a huge Linux fleet at work, which uses a custom task supervisor written before systemd.

                  I see this once in awhile and always wonder why. Enterprise distros since RHEL 6 (released in 2011!) ship with upstart as an init system, which is a great process manager. I don’t understand the need for installing yet another process manager.

              1. 5

                Friendly warning to anyone manually updating, Firefox will trash any settings you have regarding updating. So if your machine is set for manual updates, and you install Firefox 70, it changes to automatic updates. Mozilla refuses to fix this:

                https://bugzilla.mozilla.org/show_bug.cgi?id=1576400

                1. 22

                  Firefox works fine when managed by, for example, a Linux distribution’s package manager. This necessarily means that the auto updater is disabled. So this use case is handled fine. It’s also open source and Mozilla is non-profit, so I’m not sure why you’ve chosen Firefox as your evil straw man. Your bug report is not productive and your childish antics have only wasted developers’ time, who have plenty of actual work to do.

                  1. 2

                    I think the bug report is legitimate, as are the explanation, resolution and workaround that have been provided. You reply seems unnecessarily harsh to me, though I agree the submitter accepting the Firefox response would be appropriate.

                  2. 18

                    Tbh your behaviour on that bug wasn’t really acceptable. When a maintainer closes an issue please don’t re-open it simply because you disagree. As for the “bug” itself, you were given two possible workarounds. Yet for some reason you still expect developers to spend their time catering to your very obscure edge case.

                    1. 9

                      Thats interesting framing.

                      They broke the feature. Its not me asking for something new. Its me asking them to restore the previously working behavior. Before, if you wanted to permanently disable updates, you just go into about:config and toggle, and done. Now you have to manually create a JSON file, a file which is removed the next time you manually update. Just because you call it “very obscure edge case”, doesnt make it so.

                      I imagine, especially in Lobster community, and especially in the current environment of well deserved distrust with tech companies, that its not as obscure as you think or hope it is.

                      LOL just checked your bio:

                      Engineering productivity at Mozilla

                      https://lobste.rs/u/ahal

                      1. 10

                        There is an XKCD which is precisely applicable to your situation.

                        Your use case is an edge case as far as the Mozilla developers are concerned & since it’s a tiny, tiny minority of users that are relying on it (i.e., you) then the rest of the userbase comes first. Consider yourself fortunate that the mozilla developers were kind enough to tell you how to do what you want to do in future.

                        This is the price of using a piece of software with millions of users in a unique fashion I’m afraid - your use case is never going to trump those millions of users.

                        1. 11

                          You seem to be under the impression that there’s a smoky back room at Mozilla HQ where they twirl their mustaches and cackle about how they’re deliberately taking away your freedom. Admittedly it’s been some years since I worked for Mozilla, and I was A) remote employee and B) didn’t work on the browser, but if there is such a room I certainly never heard about it, let alone got invited to go see it.

                          And what I think from reading that bug report is that you used a feature (the distribution directory) for a purpose it wasn’t intended for, and then were unhappy when it behaved as documented, because it turned out not to support the use case you want. Your use case is apparently very important to you personally, but that doesn’t mean it has to be important to them, or that they have to support it. They have the freedom to decide not to support your use case; you have the freedom not to use their software. And in fact you’re better off than you’d be with some browsers, because you also have the freedom to grab the source, modify it to suit your use case, and use and distribute your fork. But you don’t and never will have a right – moral, legal, or otherwise – to force them to support your use case.

                          1. 4

                            You seem to be under the impression that there’s a smoky back room at Mozilla HQ where they twirl their mustaches and cackle about how they’re deliberately taking away your freedom.

                            That is a strawman.

                            Changes like this always have a reason. Usually, someone runs a study, or reviews a retrospective, and finds that like 20% (I don’t know the number, but I’m sure I could find it if I looked) of Firefox users had auto-updates disabled by some adware installer or whatever. And the only way Mozilla knows of preventing other software running on the same Windows machine from changing a setting is to hardcode it into the executable, where the Windows code integrity system will ensure it doesn’t get changed.

                            That doesn’t change the fact that the solution here is removing power from the end user in ways that are frequently quite harmful. I’m pointing directly at Page Translator here. That kind of “collateral damage” is extremely messed up.

                            Hard-coding auto-updates into the EXE probably isn’t that bad (running an outdated browser with known CVEs on the Internet is just stupid). Neither is the whole practice of shipping a blocklist (obviously, allowing the blocklist to be disabled in about:config, where adware installers can change it, would completely defeat the purpose of the blocklist). The fact that it ended with an add-on that clearly isn’t adware getting blocked, on the other hand, is a scandal.

                            1. 7

                              I read the Bugzilla bug linked from that post, and it appears that there was a policy change from “side-loaded extensions can execute remote-source code” to “they can’t”.

                              I have a hard time seeing that as “a scandal”. Especially given how many times we’ve seen the pattern of an extension/add-on that used to be safe and gets taken over by an entity who abuses the extension’s privileges to do malicious things.

                              So it seems there’s been a decision that nobody gets trusted to execute remote code from an add-on, and while there are certainly going to be examples like the translator add-on that intuitively feel like they should get special exceptions to that policy, special exceptions for the “good” add-on authors don’t scale.

                              Meanwhile, the “freedom” arguments almost always really boil down to demanding that someone else write software in a way that the “freedom” supporter prefers. And I don’t see any principle of software freedom which supports forcing other people to write the things you want.

                            2. 4

                              You seem to be under the impression that there’s a smoky back room at Mozilla HQ where they twirl their mustaches and cackle about how they’re deliberately taking away your freedom.

                              You missed last week’s meeting, BTW.

                              1. 3

                                We mostly talked about you and then assigned all the bugs to you. This is what happens when you miss a meeting.

                              2. -1

                                You should probably read and understand the issue before commenting. It seems you didnt even read the initial report.

                                If you had, youd realize that I am using the distribution folder as intended. Whats not intended is that folder being removed upon manual upgrade of Firefox.

                                Until then, your post just smacks of mischaracterisation. I dont appreciate it.

                                1. 9

                                  I read the bug. One of the earliest replies told you the distribution folder is intended for third-party distributions of the browser, and that a user-initiated install of stock Firefox is intended to overwrite that, on the assumption that the third-party tweaks are no longer wanted.

                                  Meanwhile, you have not stated a moral or legal principle by which you have the right to demand, and they have the obligation to provide, support for your use case. If you want to convince me, you’ll need to start there.

                                  1. 2

                                    If thats the case, then that means that Firefox currently has no official way to permanently disable updates via a setting.

                                    Is that really what youre arguing?

                                    1. 4

                                      You seem to think there is some principle on which they are morally or legally obligated to provide such a setting to you because you demanded it.

                                      You still have not articulated such a principle.

                                      My personal opinion is that they have neither a moral nor a legal obligation to implement what you’re demanding. And that the point of Free/open-source software is to support that position: what you have is the right to fork the source and implement the features you want, not the right to force someone else to do it for you.

                                      1. 4

                                        Since youre refusing to answer, i will assume this is your position:

                                        Firefox currently has no official way to permanently disable updates via a setting

                                        I dont need to make any moral or legal arguments. That sentence says enough. It says that Mozilla isnt interested in giving their users agency, and thats a scary thought given their history of open source advocacy.

                                        1. 10

                                          I don’t work for Mozilla (anymore) and have no earthly idea whether there actually is a built-in, supported setting, toggle or other mechanism for what you want. You seem to be convinced there isn’t, I see no reason to disagree with that, but you also seem to think it’s some sort of vitally important thing to get people to admit it, because you also seem to think it’s some sort of terrible sin Mozilla would be ashamed to fess up to.

                                          But my entire line of questioning to you is to explore by what principle you think Mozilla has an enforceable duty to you which requires them to provide it. As I’ve repeatedly said: you have the freedom to make your own fork of Firefox which behaves the way you want it to. Nobody owes you a configuration option for a feature you want, no matter what feature it is or how important you personally think it is.

                                          The fact that you can’t seem to understand that is worrying. The fact that you also conflate “freedom” with you having the ability to demand someone else do a thing for you is also worrying.

                                          1. 0

                                            You seem to be trying to get me to say something. And I am not going to say it.

                                            My point is, and has always been since this thread started:

                                            Friendly warning to anyone manually updating, Firefox will trash any settings you have regarding updating. So if your machine is set for manual updates, and you install Firefox 70, it changes to automatic updates. Mozilla refuses to fix this.

                                            anything else I dont care about. And BTW, you have failed to refute that point, instead trying to twist it into me demanding for something. Hint, I am not demanding, just putting the word out so people are informed.

                                            1. 7

                                              Hint, I am not demanding

                                              You reopened the bug twice, including after specifically being asked not to. In what world is that not demanding?

                              3. 7

                                As an industry, the risk of users being on outdated versions of software is huge. It is prudent to take measures to auto-update and ensure that users are on the latest versions of software to reduce this risk. An inconvenience to you is a huge boon to me, and the industry as a whole.

                                1. 4

                                  Thats a false choice. If a user understands the consequences, and is warned loud and clear before making this type of change, they should be allowed to do so. Thats why sudo exists and UAC on Windows.

                                  1. 4

                                    TBH users been trained to click “Allow” and “I Agree” until the dialogs go away.

                                    Those decisions such as enabling auto-updating to everyone are based on statistics. They are much more likely to help than to hinder. Just displaying a dialog box is not enough these days.

                                    I too would like if it was simple toggle somewhere in settings to disable updates, but Firefox is a more complex project than people realize and there is a ton of checks, balances, and teams working on different aspects of what is in essence a little virtualized operating system with a poor choice of view model for apps.

                                    I understand why you’re frustrated, I have my pet bugs too. The good thing is that you can change stuff, you can engage in constructive dialog and send a patch. And like everyone, you need to be prepared for the team who develop that app not to want your patch or feature.

                                    The good news is that even if they don’t want that feature, there is nothing stopping you from building your own build at home or forking. Still, I’d think that if you’re forking because you don’t want auto-update because you distrust big tech, then how the hell do you expect a single person to maintain security updates for a browser? I think auto update is really good and brings in a ton of fixes in.

                                    1. 2

                                      That’s a bad analogy. You should probably thinking about how the browsers that come with Windows get updated. Mozilla needs to weigh multiple issues and stakeholders here. It does not serve them well to cater to edge cases which significantly increase risk.

                                      This is also not a user “rights” issue like your language suggests. Your rights are to take the open source code and make your own build if what Mozilla provides doesn’t work for you. Your time might be better spent looking at the overall update space and lobbying for solution that gives you more of what you want while aligning to the high-level goals of Mozilla.

                                    2. 3

                                      I don’t think users end up in about:config without meaning to and accidentally turn off automatic updates. Mozilla’s response to this issue seems silly, since they should never have removed the about:config switch in the first place. At the same time, Mozilla’s decision to remove the distribution directory seems to positively affect a large number of non-technical users who would receive a custom copy of Firefox (maybe alongside another piece of software) and then try to install vanilla Firefox.

                                      Good for non-technical users, bad for corporations trying to unify rollouts of software updates. Users like @cup are probably insignificant to Mozilla in making this decision (as they represent a very small vocal minority of the Firefox userbase).

                                      1. 8

                                        I don’t think users end up in about:config without meaning to and accidentally turn off automatic updates.

                                        Third-party software, installed on the same computer, does that. A lot of the blocked add-ons have block descriptions like “overrides search behavior without user consent or control”. Because it’s adware.

                                        I’ve seen people get infected with that kind of thing; it comes bundled with another application that they installed. If it’s in about:config, then the third-party installer can just change it. If it’s hardcoded in the EXE, then the app can’t change it without re-signing Firefox, which will get their signing key revoked by Microsoft and Apple, and likely get them sued by Mozilla for trademark infringement.

                                        1. 6

                                          That’s sneaky. I guess the fact that I wasn’t aware of that goes to show that Mozilla had more of a point here than I thought they did. Thank you for informing me.

                                          I know that sandboxing on the desktop would solve this particular problem, but I’m afraid of the consequences of that on software development and particularly people learning to code. It’s pretty difficult to be exposed to real-world programming on a locked-down mobile device.

                                          1. 1

                                            I don’t think that’s a problem.

                                            • There are perfectly good ways that an operating system vendor could allow users to turn off the sandbox without allowing arbitrary applications to do that. I particularly like the Chromebook method where you use a literal jumper on the motherboard to switch it off; it’s really just a skewmorph, to make sure the human understands that they’re doing something to their computer at a low level, but it seems effective enough.

                                              Unfortunately, Mozilla’s attempt to ship an actual operating system didn’t go anywhere, so they never got the chance to implement anything like that.

                                            • Breaking out of the sandbox is really only necessary if you want to do systems programming. I love systems programming, but most software is application code, and most of that is written with sandboxed systems like web-based JavaScript and spreadsheet macros in Excel. I can write that kind of stuff on locked-down mobile devices right now.

                                        2. 1

                                          Companies trying to unify rollouts of software is part of the problem. Many major malware incidents get investigated and the findings include well meaning administrators who wished to unify and manage updates - but, failed to update in certain situations.

                                          We shouldn’t maintain the pretense that admins will get this right 100% of the time. They are people and they will fail. Their efforts are best spent elsewhere, including work to encourage devs to test compatibility, giving admins more of a stake in software acquisitions, etc.

                                      2. 5

                                        Thanks for reporting this behavior, and also thank you for pointing out the potential conflict of interest of the person who criticized you for the way you reported it. It’s really unfortunate that the Mozilla devs are optimizing for silently updating the browser and making it difficult for users to disable this behavior. If anyone is aware of a fork of Firefox that doesn’t do this, I’d love to hear about it.

                                        1. 16

                                          The initial report was fine, it’s the way they kept reopening that isn’t

                                  1. 7

                                    IBM disagrees fwiw. (I am in no way competent to judge).

                                      1. 1

                                        Note that Scott pretty much agrees IBM is right: “But I see little reason to doubt that their analysis is basically right. I don’t know why the Google team didn’t consider …, probably they wish they had.” The dispute is about what it means.

                                      2. 1

                                        I’m with them. My two problems when reading the original argument:

                                        1. They rigged the benchmark to be something quantum-specific before saying classical computers couldn’t do it. I get why. I’d just rather it be something they both could do, classical having great algorithms, and quantum one outperforms it many-fold. An old example was factoring. I’m sure there’s other examples out there. Maybe try to simplify them for low-qubit circuits.

                                        2. The biggest gripe: basing the argument on what classical computers can’t do. Can’t is an assumption that’s been dis-proven about so many things so many times. Who knows what new algorithms people will come up with down the line doing what we previously couldn’t do. I want an argument that’s instead based on what can and has been done vs the new thing being done. I similarly preferred constructivist logic to classical since the latter was assuming things didn’t exist. Universe surprises me too often for that stance.

                                        Another thing about these is the comparison to supercomputers. Me personally, I don’t care if the quantum computer outperforms supercomputers. I’m looking at practical QC like anything else: does it do the job more cost-effectively than alternative solutions? That cost includes hardware, any leasing/licensing, maintenance, specialists, etc. If it does, then they’ve achieved something for me. If it doesn’t, then QC was an implementation detail for a system that didn’t make the cut.

                                        FPGA’s are already an example of how this might play out. They’re friggin amazing with all kinds of potential. I’d love to have a desktop loaded with them with libraries using them to accelerate my apps. Them being invented and improved wasn’t enough for that, though. The duopoly, esp supported by patents, has kept prices of high-performance FPGA’s way too high.

                                        It would be better if they were priced more like flash memory with the synthesis software being more like mass-market A/V software. We’d see an explosion of use. Instead, the legal system allowed the inventors to keep competition and innovation at low levels to maximize profit. I expect the same with practical QC for at least 20 years. I’ll still be using a mix of NUMA, GPU’s and FPGA’s while early adopters mess around with quantum computers.

                                        1. 3

                                          Scott Aaronson has been pushing the Wright Brothers analogy, and I think it’s a good framing: The first airplane wasn’t very practical, but it was a pretty convincing proof of concept. It didn’t prove that you could one day have airplanes that outperformed trains on speed and volume of transport, but it was a really strong indicator that it Could Be Done. (Bear in mind that this was done on only 53 qubits, and IBM’s hypothetical classical computation would involve a massive supercomputer.)

                                          I share your frustration that they used a task whose results can only be verified by… another quantum computer. -.- Something that can only be computed quickly on quantum but also verified quickly on classical would be the best demonstration.

                                          1. 2

                                            I’m all for starting small. That’s why I mentioned something like factoring. What you said at the end also sounds good. I just want it to be something well-studied that we’re not just missing a classical algorithm and/or some ASIC’s for.

                                            1. 3

                                              For what it’s worth, they also did the same computation with a smaller number of qubits, and spent a bunch of classical computer time to verify it directly. I don’t know enough about random quantum circuits or whatever the hell problem they used to opine on whether there is room for credible scientific doubt, but I do find that at least partially reassuring.

                                              Scott Aaronson’s posts: Main FAQ after the leak, and then his official post without having to be coy.

                                              1. 1

                                                I don’t know enough about random quantum circuits or whatever the hell problem they used to opine on whether there is room for credible scientific doubt, but I do find that at least partially reassuring.

                                                You’re touching on another thing my intuition was worried about. The proof is better if it’s widely understandable instead of more in QC folks hands. We see AI research doing their thing with all kinds of papers being BS or useless with 3rd parties unable to verify that.

                                                I’m not saying that’s happening here. I just like blanket statements to concern problems many sub-fields are on top of since there’s wider peer-review. The 3rd parties might respect results they haven’t seen before on hard problems they understand even if they don’t understand how it was accomplished. Meanwhile, you and I are in the same boat about evaluating their problem. We don’t know what the hell is going on but X, Y, and Z have things to tell us about it. (shrugs) (who knows)

                                          2. 2

                                            2 is extremely unfair. Google’s theoretical foundation is very good. To use an analogy, your argument is like this: someone proves the problem X is NP-complete. You argue since we don’t know the answer to P vs NP, “who knows what new algorithms people will come up with”. That’s ridiculous.

                                        1. 1

                                          What legitimate reasons would there be to worry about that a specific piece of paper is produced at a particular printer, and that that could be traced?

                                          I can think of issues where someone wants to blow the whistle anonymously, and must deliver evidence on paper. Other than that, I can’t think of a case where this is a problem for “normal people”.

                                          1. 3

                                            That’s pretty much the reason. It’s a mechanism of control.

                                            1. 1

                                              False attribution is always a problem. Like with botnets: they’re often hacked computers leased by the bot herder to third parties to commit crimes in name of hacked person. Those crimes range from annoying to serious. The Feds are probably used to accounting for that. Would they consider false attribution if evidence, esp a “confession,” came from my printer?

                                              1. 1

                                                This works both ways - if I were to be framed by someone, the printer fingerprint can help to exonerate me.

                                                1. 1

                                                  Maybe. It might be treated differently in that scenario. It’s worth remembering that we have an adversarial system. The Feds and prosecutor are looking to convict, not exonerate, you. They might be selective in what they “find” and present. I prefer they have fewer types of fungible evidence just in case. Making them stick to stronger types of evidence keeps targets of overzealous prosecution safer.

                                                  1. 2

                                                    This privacy issue is piddling compared to the fact that LEO can usually track your every move using cellphone networks if they get the warrant. And if they suspect a certain printer, most modern ones have a record of every document printed and by whom in its internal harddrive.

                                                    Heck, every time I print something here at work I have to use my access card to get to my docs in the spool.

                                                  2. 1

                                                    If printer fingerprints can be used as evidence, that weakens the defense in cases of false attribution. Best to leave burden of proof where it is. (IANAL, obviously)

                                              1. 9

                                                Quit doing ML. Training ML models uses ridiculous amounts of compute.

                                                1. 11

                                                  In the same vein: do not support cryptocurrency pyramids, one of which already turns more energy into pointless heat than some small countries.

                                                1. 1

                                                  Oh no, not again.

                                                  It’s clearly time exim was sent to the great mailer-daemon in the sky. How many RCE CVE’s in the last two years? Too many.

                                                  Sadly there don’t seem to be any open source SMTP servers written in memory safe langauges around. Unless I’ve missed one?

                                                  1. 1

                                                    Fortunately there are SMTP servers with a proper design that greatly reduce the severity of the effects of memory corruption.

                                                    1. 2

                                                      I disagree that those 2 examples are good solutions: Postfix’s configuration is even less readable than Exim’s, and OpenSMTPd is really under-documented and looks much more trouble to run on Linux than it’s worth. Exim is still the least bad of the bunch.

                                                      1. 1

                                                        All of them should be run in a container (or jail/chroot) if not a VM (QubesOS). Furthermore, we need to get rid of root.

                                                        **edit, oh jebus. It looks like ASN.1 parsing strikes again. The most profitable back door in the history of computers.

                                                    1. 19

                                                      The hardest part about e-mail setup is to not get flagged/rejected as spam. As for this:

                                                      I would add a few things to that list but this highlights that some people are already “golden”, just by setting up proper rDNS, SPF and DKIM.

                                                      I dispute that this is factually accurate. I have run my mail server since years from the same IP, and had all of these set up properly for years. outlook.com still rejected my mail outright, and gmail delivered my mail to the spam folder.

                                                      One thing not mentioned here is signing up for so-called “feeback loops” at the big e-mail providers. But creating an account somewhere at some U.S. company I don’t have direct business with is something I simply don’t want to do.

                                                      On a sidenote, I ever only had problems delivering to U.S.-based e-mail providers. Everybody else seems to not have had any problem with my mail.

                                                      1. 6

                                                        I also can confirm this point. I frequently have to resend critical emails, using my gmail account, twice if I dont hear back after a day or two. I’ve been slowly unrolling my own email server and going back to gmail full time solely because of this.

                                                        I also get a lot more spam than my much more publicly listed gmail address. spamassassin and greylisting are much easier to beat than Google’s dedicated team of paid professionals.

                                                        1. 6

                                                          That’s precisely what they want you do to!

                                                          I’m going to maintain that it’s an intentional, anti-competitive practice on Google’s side that demands legal scrutiny.

                                                          1. 4

                                                            I’m not entirely sure it’s done with malice. Federated just means open to spam and abuse in many cases.

                                                            1. 8

                                                              I agree with the “cock-up before conspiracy” principle, but a sufficiently advances conspiracy is indistinguishable from a cock-up. ;)

                                                              1. Google is one of the largest email providers
                                                              2. Small independent servers/providers are disproprortionately affected
                                                              3. No other provider rejects so much legitimate email
                                                              4. Google often rejects them silently, without sending DMARC reports even if DMARC address is configured
                                                              5. It’s profitable for Google if people give up and switch to GMail
                                                              6. Google is known to engage in anti-competitive practices, such as making their services Chrome-only and lowering the search rating of websites that aren’t using AMP

                                                              In this situation, possibility that it’s intentional shouldn’t be rejected.

                                                              1. 3

                                                                In this situation, possibility that it’s intentional shouldn’t be rejected.

                                                                Especially if the incentives and past practices tied to incentives line up to make something likely to happen. You covered that in 5 and 6. I’m adding this for emphasis. You could add Google services on Android to that list, too, given how that plays out on stock vs phones focused on privacy or user freedom.

                                                          2. 3

                                                            You can also use something like FastMail that’s cheap, respects your privacy, and I’m told is faster due to less surveillance tech in their apps.

                                                            1. 2

                                                              This is what I did after I was sick of disappearing mail from my nice setup with opensmtpd. I didn’t mind administering it, but email is useless if it’s unreliable.

                                                            2. 2

                                                              I also get a lot more spam than my much more publicly listed gmail address. spamassassin and greylisting are much easier to beat than Google’s dedicated team of paid professionals.

                                                              I suppose their approach is to better reject a legitimate mail than to allow a spam e-mail.

                                                              I’ve been slowly unrolling my own email server and going back to gmail full time solely because of this.

                                                              I found a counter-measure, as I mentioned in other threads on the topic. You can use an SMTP relay provider, i.e. relay all your outgoing e-mail through a third party’s SMTP server. You’d then have to pay for the SMTP relay provider, though.

                                                              1. 1

                                                                I also get a lot more spam than my much more publicly listed gmail address. spamassassin and greylisting are much easier to beat than Google’s dedicated team of paid professionals.

                                                                rspamd is very, very effective at killing spam in my experience. Give it a try if you haven’t already.

                                                              2. 1

                                                                I spent a number of years working for an Email Service Provider that shifted enough emails to saturate a gigabit connection; we always had issues with Gmail and Hotmail (sometimes but not often Yahoo, everyone else seemed to play along nicely) even with a direct hotline to the respective engineers at their ends.

                                                                Their systems would often purposfully drop incoming email with various error responses that often were seemingly random in nature and nothing to do with anything our end, sometimes they would accept email as deivered while silently dropping it.

                                                                Nowadays I use tutanota for simplicity, someone else can manage the maintenance :)

                                                              1. 1

                                                                The windows 10 terminal has been steadily improving as MS fixes the corner cases in order to get a decent WSL experience. It seems quite capable of (for instance) allowing me to ssh into a Linux server & run emacs inside tmux there. So for me the most tolerable version of Windows is probably the latest Windows 10 release (1904?) with WSL if required.

                                                                1. 2

                                                                  One more bug on ethereum, at which point they will understand that ethereum is definitely a bad idea ?

                                                                  1. 1

                                                                    What in Ethereum is a bad idea? I personally think the implementation is not without flaws but the project seems pretty cool.

                                                                    1. 7

                                                                      Because it assume the ethereum contracts are programs set in the stone and managing money without control. And to do so you must have free of bug programs. Such programs only exists in a very limited extent, as for example limited programs done for bitcoin.

                                                                      Each contract which has a vulnerability show that the assumption of bug free program don’t stand.

                                                                      Without bug,ethereum would be a splendid idea, but this condition don’t hold and as such, anybody putting money on any smart contract is very likely ot lose it. And as basically the whole ethereum promise is that the smart contract work as their creators says so. it is basically a scam.

                                                                      1. 1

                                                                        “not without flaws” is an interesting way of spelling “a garbage fire of bad ideas in which it is almost impossible, even for experts, to write correct code”.

                                                                    1. 7

                                                                      You know you’re getting the good stuff when you see a paragraph like this:

                                                                      This post represents a lot of research and brute-force attempts at trying to do this. I have had to assemble things together using old resources, reading kernel source code, intense debugging of code that was last released when I was in elementary school, tracking down a Heroku buildpack with a pre-built binary for a tool I need and other hackery that made people in IRC call me magic. I hope that this post will function as reliable documentation for doing this with a modern kernel and operating system.

                                                                      1. 5

                                                                        Other gems from that article:

                                                                        This is about where things get really screwy. Networking for a user mode Linux system is where the “user mode” facade starts to fall apart. Networking at the system level is usually limited to privileged execution modes, for very understandable reasons.

                                                                        However, there’s an ancient and largely unmaintained tool called slirp that user mode Linux can interface with. It acts as a user-level TCP/IP stack and does not rely on any elevated permissions to run. This tool was first released in 1995, and its last release was made in 2006. This tool is old enough that compilers have changed so much in the meantime that the software has effectively rotten.

                                                                        It’s not immediately crashing, so I think it should be good!

                                                                        I have no idea how his Slackware system works fine with slirp but my Ubuntu and Alpine systems don’t, and why the binary he gave me also didn’t work; but I got something working and that’s good enough for me.

                                                                      1. 5

                                                                        Is it weird that I look forward to these releases but still mostly stay pinned to testing in my sources file?

                                                                        1. 3

                                                                          Not necessarily, because as soon as buster is stable, more updates should be expected, since the freeze will be over.

                                                                          1. 3

                                                                            Rule 1 of tracking testing: Don’t for the love of all that’s holy track testing in the weeks immediately following a release. Any other time, it’s probably more stable than anything else out there but things can get a bit hairy as all the pent up changes that have been held back for the stable release flood into the pool.

                                                                            1. 1

                                                                              I don’t know, I’ve been pinned to testing for years and I’ve never had a problem.

                                                                        1. 2

                                                                          Just don’t do this on any computer which ever runs untrusted code.

                                                                          Especially don’t do it if you browse the web on such a computer with Javascript turned on.

                                                                          1. 2

                                                                            Yes! Forgot to add the satire tag. Added. :-)

                                                                            1. 1

                                                                              I have computers that don’t ever run javascript. I’m bookmarking this. I haven’t heard of most of these arguments… Did the author miss any?

                                                                              The idea could be expanded to include gcc flags, right?

                                                                              1. 1

                                                                                I would suggest not and just trust the kernel maintainers. There may be very specific reasons for why the kernel is optimized the way it is. Sometimes over optimization actually introduces security vulns because compilers get too smart for their own good.

                                                                                1. 1

                                                                                  Oh, let me try again.

                                                                                  The kernel command line arguments shown in this post appear to deliberately disable important security features in favor of performance. (Are none of these ‘free’ to enable? I don’t know how much work the author put into selecting this specific list.)

                                                                                  Just for fun, using a computer disconnected from the internet… Can we push this idea further?

                                                                                  Are the binaries in popular contemporary linux distros compiled with compiler options that favor security over performance? That is, are there compile-time choices we can make to favor performance over security?

                                                                                  I’d like to imagine this line of thought could actually be meaningful in some hypothetical situation. Like using old hardware to play HD video (on a machine not connected to the internet) or something.

                                                                                  1. 1

                                                                                    Sure, link everything into the kernel and avoid syscall and context switch overhead.

                                                                                    1. 1

                                                                                      That sounds good. As I understand it, context switches are very expensive.

                                                                                      But, that’s a lot more work than changing some parameters, right? What would a utility like grep even look like after ‘linking everything into the kernel’?

                                                                                      tedu, are you talking about putting a bunch of kernel into a grep binary, or putting a bunch of grep application into the kernel?

                                                                                      (This line of conversation would rightly be classified as a thought experiment, right?)

                                                                                      1. 1

                                                                                        (This line of conversation would rightly be classified as a thought experiment, right?)

                                                                                        I suspect it would be classified as trolling.

                                                                                        But having a unibinary system would be fascinating, offer very little by way of runtime customization (let alone programming any compiled language), and have miserable separation between user accounts (if you maintain a mostly POSIX compatible interface).

                                                                                        1. 2

                                                                                          Unibinary. Ok, so, we’re talking about putting the functionality of applications into the kernel. Yes, the downsides you describe make sense. Though, customization is still possible through self-modification.

                                                                                          …I’m certainly not trying to have any kind of negative impact on anyone. I am, I’ll admit, trying to get something out of folks. I want to know how computers work. Actually, I guess I want to know how compilers work–in practice. I seem to know how compilers work in theory.

                                                                                          Moreover, I have believed for some time that my various CPUs spend a lot of time and heat on tasks that are somehow adjacent to whatever task I ask of them. More and more I am biased towards leaner systems that do less.

                                                                              2. 1

                                                                                That’s computer, right? How much of that command line even applies to my intel-free devices?

                                                                                1. 2

                                                                                  The spectre mitigations apply to everyone that does branch prediction.

                                                                              1. 5

                                                                                I was talking to a friend today who works in a high-assurance workplace (think defence / safety critical systems / nuclear engineering etc). They have a nifty Docbook based system where /all/ the documentation for their systems is contained within the source code & they can generate PDFs documenting the system (including state chart diagrams and all that kind of thing) from the sources which they can submit to their client / regulator whenever they want. I’ll have to see if they can document how they go about it.

                                                                                Setting this up was a huge improvement on their previous “do everything by hand in word documents” approach.

                                                                                1. 1

                                                                                  I assume that the PDFs contain nothing that isn’t just straight from the docbooks, would that be fair?

                                                                                  1. 1

                                                                                    I think they auto-generate statechart diagrams & that kind of thing from text in the source.

                                                                                1. 3

                                                                                  Now I’m going to have to dig out all my olf CSP references to compare & contrast.

                                                                                  1. 2

                                                                                    Try with SPIN. It’s based on CSP. Lots of academic and industrial use.

                                                                                    1. 5

                                                                                      I was the sole developer of FDR for a number of years :)

                                                                                      1. 2

                                                                                        That FDR? I didn’t even know about it for some reason! That’s pretty awesome. Thanks for the link. Even more awesome that it got parallelized.

                                                                                        1. 3

                                                                                          I was FDR2 for some time - FDR3/4 was mostly put written by a very talented postgrad at the department, but sizable chunks of that are mine too.

                                                                                        2. 1

                                                                                          omg you are super cool can I pick your brain on FDR and CSP please?

                                                                                          1. 1

                                                                                            Any time…

                                                                                    1. 34

                                                                                      I’m very strongly in favour of objective, civil, polite discourse. I don’t think anything is gained by penalising people if their tone is a little abrasive.

                                                                                      To be blunt (in full acceptance of the irony here) I would much rather that people post abrasive content than risk them self-censoring for fear of appearing “unkind”. I’m afraid that lobste.rs would cease being a haven for constructive discussion, and become some sort of Stepford-esque echo chamber instead.

                                                                                      1. 46

                                                                                        You can disagree with something without being a jerk about it. I think that’s something we should all aspire to.

                                                                                        1. 11

                                                                                          I agree (and think that “being a jerk” is quite a different thing to “being abrasive”, but maybe that’s just my reading of the terms).

                                                                                          However, I’ll take “abrasive and correct” over “nice but wrong” any day of the week, and I fear that encouraging downvotes over tone will result in the loss of some of the former.

                                                                                          1. 7

                                                                                            I’ll take “abrasive and correct” over “nice but wrong” any day of the week, and I fear that encouraging downvotes over tone will result in the loss of some of the former.

                                                                                            Fortunately, these are not the only two options. An ideal solution would discourage the “abrasive and correct” in favor of the “non-abrasive and correct.” In such an environment, “nice and wrong” comments are welcome because they spark informative discussions.

                                                                                            1. 2

                                                                                              Which is why a downvote, as they’re currently implemented, isn’t a good solution. Some sort of separate flag might be, though. Provide an easy way for readers to nudge posters to edit their posts for tone.

                                                                                              1. 1

                                                                                                “Nice and wrong” comments that require other people to expend effort to correct them are not kind.

                                                                                                1. 6

                                                                                                  The “incorrect” flag covers those, however.

                                                                                              2. 4

                                                                                                Perhaps the “unkind” vote should have zero affect on karma, but still deprioritize the comment? That way it’s less likely to generate feelings of defensive?

                                                                                                1. 4

                                                                                                  Being unkind seems very karma-related to me

                                                                                                  1. 4

                                                                                                    It could just be some CSS that provides visual feedback from the downvote action to the voter. Like a big red mechanical button that isn’t wired up to anything, but feels satisfying to press.

                                                                                                2. 6

                                                                                                  You certainly can. But anger and frustration, like other human emotions, have circumstances where they are objectively justified and reasonable. One would say that in software development, for example, we aren’t really facing the questions of life and death, and getting worked up about something like that is silly. I agree that a lot of anger in comment sections is, indeed, silly – but not all of it.

                                                                                                  Let me bring an example. Recently I found out that one company, which is developing a technology which have been my main “specialisation” since 2012, have decided to move around different modules and in the process deleted documentation for one of it - so from new version on, instead of automatically generated API specification, with all classes and methods, I would have to refer to guides, organised by topic. This change is completely unnecessary, and will make my day-to-day work much harder. I spend 8 hours a day, 5 days a week, at my job, and something that they have done will make me a little bit more miserable, every day. And they don’t even have any reason for it. And a forum post that I’ve written, with detailed explanation of the issue, and then tweeted at them, have gone without a single reply.

                                                                                                  Now, if I would encounter the person who was responsible for this decision here in the comments in a relevant thread, I think that venting my frustration, while staying civilised, would be an appropriate response. Don’t you?

                                                                                                  1. 6

                                                                                                    And yet, being direct is often seen as being unkind. I’m strongly against ranking people on kindness – in addition to being very vague, the norms are highly cultural.

                                                                                                    I sometimes wonder if we’d be better of getting rid of votes entirely, and rely on people using their words.

                                                                                                    1. 6

                                                                                                      highly cultural

                                                                                                      But surely we’re seeking to build a lobste.rs culture?

                                                                                                    2. 2

                                                                                                      100% agreed. In my experience you’re also more likely to effectively get your argument across. However, it is a skill that requires effort to learn and apply – at least in my case.

                                                                                                      1. 1

                                                                                                        but you can’t guarantee that people will interpret your politeness as such

                                                                                                      2. 38

                                                                                                        I would much rather that people post abrasive content than risk them self-censoring for fear of appearing “unkind”. I’m afraid that lobste.rs would cease being a haven for constructive discussion

                                                                                                        The opposite is also true: some people stop posting after too many negative interactions, which also reduces participation.

                                                                                                        You call it “censorship”. Frankly, I’m starting to strongly dislike this term. It’s carelessly thrown around far too often. Every community has social norms, and online communities are no exception. If I’m an asshole to my friends then at some point they’ll start shunning me. If I join a football club (or scout group, or choir, or whatever) and act like an asshole then sooner or later I’ll be asked not to come next week. Would you call this “censorship”? I wouldn’t.

                                                                                                        I stopped posting on /r/programming at reddit after being called a “moron”, “idiot”, “retard”, accused of having an IQ lower than 65, was told that I “fucking suck at making software (and I guess generally anything)”, was told that my opinion was “hates speech” in two separate recent incidents (both over a technical disagreement, wtf?!), and just general unconstructive/aggressive/belittling/etc. word choice.

                                                                                                        It’s not that I’m that sensitive, but if you spend a lot of time writing a weblog post, or make some software, and you get told any of the above (which are all real quotes) then that’s … not great. It’s not that I get angry or “offended”, but it’s also not fun and if it happens a few times I’ll stop coming back (as happened on /r/programming). I think most people participate in these communities just for the fun of it. Sure, you also learn new stuff, but I think fun is an important – if not the most important – part for many.

                                                                                                        Constructive discussion can only happen if everyone feels like they can participate without the fear of being mistreated (belittling, aggressive replies, insults, etc.) If there is such a fear, then I will guarantee you that some people simply won’t post at all.

                                                                                                        I’m not sure (yet) if a flag is a good idea here for other reasons (I’ll make another top-level comment about that), but I do (strongly) disagree with your sentiment.

                                                                                                        1. 18

                                                                                                          I definitely use this site less over time because of the loud frequent posters who often carelessly put down people with insensitive wording. When I was junior I was able to justify spending the emotional energy listening to technically correct jerks, but nowadays it’s pretty rare that I get anything other than anger out of their childish communication. I only come on lobste.rs now when I’ve got pretty high emotional buffers, because otherwise it’s likely to just make me feel worse.

                                                                                                          1. 7

                                                                                                            I’ve only been here for a few months, so I can’t comment on Lobste.rs specifically, but I can comment on two general observations:

                                                                                                            • Often >90% of the problems come from ~1% of the people.
                                                                                                            • As a community becomes larger, it becomes harder to manage because mods don’t see most of what’s going on (making it harder to identify patterns).

                                                                                                            In many ways it’s the same as traffic; if you drive or cycle home you may encounter 100 drivers, so if just 1% is reckless driver then you’ll meet one most days. Also, like traffic, it’s hard to completely remove these people unless they commit gross offences. You can break traffic laws and be reckless for pretty much your entire life, and suffer very few consequences.

                                                                                                            Most of us act like an asshole sometimes; I know I do; I have pretty strong feelings about certain political topics, and sometimes I just have a bad day. But I’m not consistently an asshole. I think they key here is not to look at individual comments too much, but rather at long-standing patterns. There are just a few mods here, and they probably don’t see most of what’s going on. So the ability to see things like “hey, this user is responsible for 28% of all unkind flags” is the critical bit.

                                                                                                            I don’t know if this needs to be tied to downvoting. Could just be a separate flag. I don’t think it matters too much, as long as there’s an admin panel to see an overview.

                                                                                                          2. 6

                                                                                                            This is exactly what happened to Slashdot too. The loudest, most aggressive users gradually took over the comment section, and the more rational voices left. That site is now a quagmire of hate speech. I think it is a good idea to get ahead of it on this site, because it could happen here too. I like the discussion environment of this site, and I don’t want to lose yet another community.

                                                                                                            I think that the idea of shadowbanning from reddit could be combined with the stack overflow style of flagging bad behavior. If nobody can see abusive or trollish comments, then they don’t accumulate comments and effectively don’t exist (i.e., not rewarding bad behavior). Those users will either correct their behavior or stop posting altogether.

                                                                                                            1. 3

                                                                                                              I’m sorry you experienced that behaviour. I have myself, largely for holding unpopular political opinions[1] . It’s even more fun when people attack those positions in a social situation, before realising that someone in the group actually holds them :)

                                                                                                              The behaviour you describe crosses way beyond “abrasive” to downright abusive. I’d be okay with a flat out ban in the case of someone who called another poster a retard, for example.

                                                                                                              By “abrasive” I mean posts that might be terse, strongly critical, or dismissive. That is, posts that have issues with tone. Things that could be charitably interpreted as well intentioned.

                                                                                                              [1] I guess you’d call them Objectivist, for want of a better term. Strongly socially and economically liberal. The former is common in Australian tech circles, the latter rare. People here usually assume party-political alignment, so if you’re say in favour of open immigration, they assume you’re also in favour of progressive taxation.

                                                                                                              1. 1

                                                                                                                The reddit example is of course much more extreme than anything I’ve seen here; but it does clearly illustrate the point that people can stop posting (“self-censor”) due to lack of moderation, too.

                                                                                                                By “abrasive” I mean posts that might be terse, strongly critical, or dismissive. That is, posts that have issues with tone. Things that could be charitably interpreted as well intentioned.

                                                                                                                A good rule-of-thumb is whether a comment makes you go sigh, “eh”, “pff”, or something similar, either by actually saying it or saying it “in your head”. You can be critical of what someone said and not evoke such a response. My previous comment was critical of your post, but I don’t think if evoked a “pff” response (or at least, I hope it didn’t!) but it’s not hard to imagine that it could with some stuff rephrased.

                                                                                                                I know this is murky and unclear, but that’s the way language works, especially in a global community with different cultures, etc.

                                                                                                                I think the key thing here is that “abrasiveness” accumulates. If you encounter an abrasive comment on occasion then that’s okay. Most people are abrasive some of the time (I know I am); that’s just the way things work. The problem is when people are abrasive most of the time, and you encounter abrasive everywhere you look.

                                                                                                                I don’t think singular abrasive comments are a problem, or that people should be punished for it. But if they’re constantly making them then there is a problem that should be addressed. Also see my other reply in this thread: https://lobste.rs/s/xnjo8g/add_downvote_reason_unkind#c_kqtuqr

                                                                                                                Analogy: Lobste.rs keeps track of “self promoters”; people who frequently post links to their own websites. Is this preventing people from submitting links to their own site? Not really; but it does help keep track of people who spam links too frequently. I think a potential “unkind flag” should work the same way.

                                                                                                              2. 2

                                                                                                                “Censorship” is certainly an overused weasel-word nowadays. Moderation is (generally) not censorship.

                                                                                                              3. 9

                                                                                                                Just to clarify, this kind of “bluntness” is of course perfectly acceptable. If you’d labeled my suggestion a “crap idea” on the other hand…

                                                                                                                1. 4

                                                                                                                  And that’s the rub, isn’t it? I have no problem with having my ideas called crap, but don’t like calling people “crap”. Others might be more sensitive than you, and then you have a ratchet that moves in only one direction, as people say less, challenge each other less, and so on.

                                                                                                                2. 4

                                                                                                                  Just to piggy-back on this a bit. I wholeheartedly applaud the heart behind this suggestion, I prefer that the tone be kept civil and polite here. However, kind/unkind might be a bit too subjective and might unwittingly stifle conversation. My concern is that for one person, a simple disagreement with an idea could be deemed “unkind” regardless of tone. My skin might be a little thinner, so my unkind trigger finger might be more prone to fire. I think troll covers abrasive behavior and perhaps some kind of flag could be used to alert moderators when issues arise and tone sinks too low in a conversation.

                                                                                                                1. 2

                                                                                                                  Let’s consider you’re a reckless dude who thought a swap partition is unnecessary or just forgot about it during installation. Worry not, at least if you’re on linux

                                                                                                                  # cd /
                                                                                                                  # dd if=/dev/zero of=/pagefile.sys bs=1G count=8
                                                                                                                  # chmod 600 /pagefile.sys
                                                                                                                  # mkswap /pagefile.sys
                                                                                                                  # swapon /pagefile.sys
                                                                                                                  # $EDITOR /etc/fstab #optionally
                                                                                                                  
                                                                                                                  1. 4

                                                                                                                    Hmm, I wonder where I have seen that filename before :)

                                                                                                                    This approach is quite nice because it makes it easy to resize the swap. One downside is that suspend-to-disk won’t work if the disk is encrypted.

                                                                                                                    1. 1

                                                                                                                      This is extra fun especially on CoW filesystems like ZFS: you’re out of memory and need to swap… and you reach into the filesystem to write… whoops the filesystem needs more memory for itself…

                                                                                                                      1. 1

                                                                                                                        Apparently it’s (probably) reliable if (and only if) you turn all the ZFS features off & make sure to pre-allocate the file on disc. So no compression, no checksums, no data cache. Turn on any of those and you’re going to get deadlocks under memory pressure though.

                                                                                                                        (The FreeBSD people have run into exactly the same problem for the same reasons.)

                                                                                                                        1. 1

                                                                                                                          Gp has an interesting point re:cow filesystems - and you probably shouldn’t use a zfs file for swap. Otoh zfs also is a volume manager - using a zvol should be fine. However, it appears Solaris special-cased swap (avoiding dead-lock, trading for the option of swap bit rot…). So it appears one would be better off with a separate partition for now:

                                                                                                                          https://github.com/zfsonlinux/zfs/issues/7734

                                                                                                                          1. 2

                                                                                                                            zvol swap on FreeBSD is not fine, I have deadlocked that :D

                                                                                                                      1. 39

                                                                                                                        We need a name for this pattern around network protocols: “Embrace, Capture, Break away, Lock-in”

                                                                                                                        • Embrace a communication standard
                                                                                                                        • Capture: attract a large user base
                                                                                                                        • Break away: break backward compatibility and/or provide a worse UX for those outside of your walled garden
                                                                                                                        • Lock-in: corner in the userbase

                                                                                                                        Google did this with Google Talk vs XMMP, email (try running your own mailserver), AMP, RSS…

                                                                                                                        1. 14

                                                                                                                          Email is still mostly unmolested if you understand the security and spam context; it’s not that google made it impossible to run your own smtp server, but in order to do so and not get flagged as spam, there are a lot of hoops to jump through. IMHO this is a net benefit, you still have small email providers competing against gmail, but much less spam.

                                                                                                                          1. 15

                                                                                                                            Email is mostly unmolested because it’s decentralized and federated, and a huge amount of communication crosses between the major players in the space. If Google decided they wanted to take their ball and go home, they would be cutting of all of Gmail, Yahoo mail, all corporate mail servers, and many other small domains.

                                                                                                                            If we want to make other protocols behave similarly, we need to make sure that federation isn’t just an option, but a feature that’s seamless and actively used, and we need a diverse ecosystem around the protocols.

                                                                                                                            To foster a diverse ecosystem, we need protocols that are simple and easy to implement, so that anyone can sit down for a week in front of a computer and produce a compatible version of the protocol from first-enough principles, and build a cooperating tool, to diffuse the power of big players.

                                                                                                                            1. 9

                                                                                                                              So how do you not get flagged for spam? I want to join you. I run my own e-mail server and have documented the spam issue here:

                                                                                                                              https://penguindreams.org/blog/how-google-and-microsoft-made-email-unreliable/

                                                                                                                              The only way to combat Google and Microsoft’s spam filters is sending my e-mail, texting my friend say, “Hey I sent you an e-mail. Make sure it’s not in your spam folder.” Usually if they reply, my e-mail will now get through .. usually. Sometimes it gets dropped again.

                                                                                                                              I have DKIM, DMARC and SPF all set up correctly. Fuck Gmail and fuck outlook and fuck all the god damn spammers that are making it more difficult for e-mail to just fucking work.

                                                                                                                              1. 3

                                                                                                                                Forgive the basic question: do you have an rDNS entry set for your IP address so a forward-confirmed reverse DNS test passes? I don’t see that mentioned by you in your blog post, though it is mentioned in a quote not specifically referring to your system.

                                                                                                                                It’s not clear who your hosting provider (ISP) is, though the question you asked them about subnet-level blocking is one you could answer yourself via third-party blacklist provider (SpamCop, Spamhaus, or many others of varying quality) and as a consequence work with them on demonstrable (empirical) sender reputation issues.

                                                                                                                                1. 8

                                                                                                                                  Yes I’ve been asked that before and haven’t updated the blog post in a while. I do have reverse DNS records for the single IPv4 and 2 IPv6 addresses attached to the mail server. I didn’t originally, although I don’t think it’s made that big a difference.

                                                                                                                                  I’ve also moved to Vultr, which blocks port 25 by default and requires customers explicitly request to get it unblocked; so hopefully that will avoid the noisy subnet problem so often seen on places like my previous host, Linode.

                                                                                                                                  I think a big factor is mail volume. Google and Microsoft seem to trust servers that produce large volumes of HAM and I know people at MailChimp that tell me how they gradually spin up newer IP blocks by slowly adding traffic to them. My volume is very small. My mastodon instance and confluence install occasionally send out notifications, but for the most part my output volume is pretty small.

                                                                                                                                  1. 8

                                                                                                                                    Email is inherently hard, especially spam filtering; Google and Microsoft just happen to be the largest email providers, so it appears to be a Google or Microsoft problem, but I don’t think it is.

                                                                                                                                    E-mail was once the pillar of the Internet as a truly distributed, standards-based and non-centralized means to communication with people across the planet.

                                                                                                                                    I think you’re looking through rose-tinted glasses a bit. Back in the day email was also commonly used to send out spam from hijacked computers, which is why many ISPs now block outgoing port 25, and many email servers disallow emails from residential IPs. Clearly that was suboptimal, too.

                                                                                                                                    Distributed and non-centralized systems are an exercise in trade-offs; you can’t just accept anything from anyone, because the assholes will abuse it.

                                                                                                                                    1. 4

                                                                                                                                      Cheap hosting is very hard to run a mailserver from because the IP you get is almost certainly tainted.

                                                                                                                                      Having valid rDNS, SPF & DMARC records helps.

                                                                                                                                2. 13

                                                                                                                                  It’s also not really a Google issue; many non-Google servers are similarly strict these days, for good reasons. It’s just that Google/Gmail is now the largest provider so people blame them for not accepting their badly configured email server and/or widely invalid emails.

                                                                                                                                  I’ve worked a lot with email in the last few years, and I genuinely and deeply believe that at least half of the people working on email software should be legally forbidden from ever programming anything related to email whatsoever.

                                                                                                                                  1. 2

                                                                                                                                    In other words, Google didn’t have to break email because email has been fundamentally broken since before they launched GMail.

                                                                                                                                    Worse, newer protocols like Matrix and the various relatives of ActivityPub and OStatus don’t fix this problem.

                                                                                                                                    1. 7

                                                                                                                                      Matrix, ActivityPub and OStatus don’t fix Email? Well it’s almost as if they are trying to solve other problems than internet mail.

                                                                                                                                      1. 3

                                                                                                                                        You completely and utterly missed the point.

                                                                                                                                        Mastodon, Synapse, and GNU Social all implement a mixture of blacklists, CAPTCHAs, and heuristics to lock out spambots and shitposters. The more popular they get, the more complex their anti-spam measures will have to get. Even though they’re not identical to internet mail (obviously), they still have the same problem with spambots.

                                                                                                                                        1. 11

                                                                                                                                          Those problems are at least partly self-inflicted. There’s nothing about ActivityPub which requires you to rehost all the public content that shows up. You can host your own local public content, and you can send it to other instances so that their users can see it.

                                                                                                                                          Rehosting publicly gives spammers a very good way to see and measure their reach. They can tell exactly when they’ve been blocked and switch servers. Plus all the legal issues with hosting banned content, etc.

                                                                                                                                          1. 3

                                                                                                                                            You’re acting as if that ONE problem (abusive use) is THE only problem and the rule and guide with which we should judge protocols.

                                                                                                                                            While a perfectly reasonable technocratic worldview, I think things like usability are also important :)

                                                                                                                                            1. 9

                                                                                                                                              In general, you’re right. A well-designed system needs to balance a lot of trade-offs. If we were having a different conversation, I’d be talking about usability, or performance, or having a well-chosen set of features that interact with each other well.

                                                                                                                                              But this subthread is about email, and abusive use is the problem that either causes or exacerbates almost every other problem in email. The reason why deploying an email server is such a pain is anti-spam gatekeeping. The reason why email gets delayed and silently swallowed is anti-spam filtering. The reason why email systems are so complicated is that they have to be able to detect spam. Anti-backscatter measures are the reason why email servers are required to synchronously validate the existence of a mailbox for all incoming mail, and this means the sending SMTP server needs to hold open a connection to the recipient while it sifts through its database. The reason ISPs and routers block port 25 by default is an attempt to reduce spam. More than half of all SMTP traffic is spam.

                                                                                                                                              If having lots of little servers is your goal, and you don’t want your new federated protocol to have control under a small number of giant servers, then you do need to solve this problem. Replicate email’s federation method, get emails emergent federation behavior.

                                                                                                                                      2. 5

                                                                                                                                        XMMP has a lot of legitimate issues. Try setting up a XMMP video chat between a Linux and macOS client. I’d rather lose my left arm than try doing that again.

                                                                                                                                        1. 7

                                                                                                                                          Desktop Jingle clients never really matured because it wasn’t a popular enough feature to get attention.

                                                                                                                                          These days I expect everyone just uses https://meet.jit.si because it works even with non-XMPP users and no client

                                                                                                                                          1. 0

                                                                                                                                            I just got jitsi working w/ docker-compose meet.dougandkathy.com – not headache free, but no way I could build it myself

                                                                                                                                          2. 1

                                                                                                                                            Audio, video and file transfer is still very unreliable on most IM platforms. Every time I want to make audio or video call with someone we had to try multiple applications/services and use the first one that works.

                                                                                                                                            1. 0

                                                                                                                                              Microsoft Teams does this pretty well, across many platforms. Linux support is (obviously, I guess) still a bit hacky, but apparently is possible to get to work as well.

                                                                                                                                          1. 6

                                                                                                                                            That repo is pure nostalgia fest for me, even though I skipped the 80s text adventure scene. I played a lot of the games that were released in the 90s on the back of Graham Nelsons work building a compiler for the z-machine & have all of these infocom games on a CDROM (released by Activision at some point) which is somewhere in the loft.

                                                                                                                                            All these games were being referenced by people within the scene all the time; it’s fantastic to have all this detail about them finally available to everyone. I hope they stay up & we shift the acceptable position on availability of copyrighted data for the sake of historical analysis away from ‘the copyright owner gets to insist that absolutely everything must stay locked up forever regardless of the wishes of everyone else’.

                                                                                                                                            1. 8

                                                                                                                                              Andrew Plotkin’s blogpost on the topic.

                                                                                                                                              Twitter thread on the release from Jason Scott of textfiles.org