1. 9

    Securing MTA must be a cursed job.

    Back in the old days we had near weekly RCEs in sendmail and exim and these days it’s OpenSMTPD with strong ties to the f’ing OpenBSD project. That’s the one project I expect an RCE the least from; much less two in as many months.

    Email is hard.

    1. 5

      It’s actually 3 — this one has two separate CVE’s in a single release, including a full local escalation to root on Fedora due to Fedora-specific bugs adding an extra twist (CVE-2020-8793).

      The other bug here (CVE-2020-8794) is a remote one in the default install; although the local user still has to initiate an action to trigger an outgoing connection to an external mail server of the attacker, so, I guess OpenBSD might not count it towards the remote-default count of just two bugs since years ago.

      1. 2

        I guess OpenBSD might not count it towards the remote-default count of just two bugs since years ago.

        I feel like that would be disingenuous. I realize it’s not enabled by default in a way that’s exploitable but in the default install there’s literally nothing running that’s even listening really (you can enable OpenSSH in a default install, I suppose); this is of course the correct way to configure things by default. However, the statement degenerates to “no remotely exploitable bugs in our TCP/IP stack and OpenSSH”…which is awesome, but…

        (Also, it’s easy to criticize: I’ve never written enterprise grade software used by millions.)

        1. 1

          Can you explain more about why you think that’s disingenuous? OpenBSD making this claim doesn’t seem different to me than folks saying that this new bug is remotely exploitable. It’s very specific and if something doesn’t meet the specific criteria then it doesn’t apply. Does that make sense?

          It is my opinion that the statement should be removed – not because it’s not accurate but because I just think it’s tacky.

          1. 4

            IMHO it’s disingenuous because it implies that there are only two remote holes in a heck of a long time on a working server. It’s like saying “this car has a 100% safety record in its default state,” that is, turned off.

            (I’m reminded of Microsoft bragging about Windows NT’s C2 security rating, while neglecting to mention that it got that rating only on a system that didn’t have a network card installed and its floppy drive glued shut.)

            I’m not sure if they include OpenSSH in their “default state” (I think it is enabled by default), but other than OpenSSH there’s nothing else running that’s remotely reachable. Most people want to use OpenBSD for things other than just an OpenSSH server (databases, mail servers, web servers, etc), and they might get an inflated sense of security from statements like that

            (Note that OpenBSD is remarkably secure and their httpd and other projects are excellent and more secure than most alternatives, but that’s not quite the point. Again, it’s easy for me to criticize, sitting here having not written software that has been used by millions.)

            1. 2

              I appreciate you taking the time to elaborate. I think the claim is tacky as it seems to be more provocative than anything else – whether true or not. I don’t think it’s needed because I think what OpenBSD stands for speaks for itself. I think I understand why the claim was used in the past but this conversation about it comes up every time there’s a bug – whether remote or not. The whole thing is played out.

              1. 2

                AFAIK OpenSMTPD is enabled by default, but does local mail delivery only with the default config. This makes the claim about “only 2 remote holes” still stand still, though I agree with your analysis of bullshit-o-meter of this slogan. But hey, company slogans are usually even more bullshit-ridden, so I don’t care.

          2. 1

            You’re saying a local user has to do something to make it remote? Can you explain how that makes it remote?

            1. 2

              One of the exploitation paths is parsing responses from remote SMTP servers, so you need to request that OpenSMTP connect out to an attacker-controlled server (e.g. by sending email).

              It looks like on some older versions there’s a remote root without local user action needed…

              1. 1

                I reckon I’ll go back and read the details again. However, if something requires that a local user do a very specific thing under very specific circumstances (attacker controlled server, etc.) in order to exploit – that does not jive with my definition of remote.

                1. 3

                  Apparently you can remotely exploit the server by triggering a bounce message.

          3. 2

            Step zero is don’t run as root and don’t have world writable directories.

            .

            .

            .

            Sorry, was I yelling?

            1. 4

              Mail is hard that way in that the daemon needs to listen to privileged ports and the delivery agent needs to write into directories only readable and writable by a specific user.

              Both of these parts require root rights.

              So your step zero is impossible to accomplish for an MTA. You can use multiple different processes and only run some privileged, but you cannot get away with running none of them as root if you want to work within the framework of traditional Unix mail.

              Using port redirection and virtual users exposing just IMAP you can work around those issues, but them you’re leaving the traditional Unix setup and you’re adding more moving parts to the mix (like a separate imap daemon) which might or might not bring additional security concerns

              1. 2

                At least on Linux there’s a capability for binding into privileged ports that is (the cap) not equivalent to root.

                1. 3

                  yes. or you redirect the port. but that still leaves mail delivery.

                  As I said in my original comment: email is hard and that’s ok. I take issue with people reducing these vulnerabilities (or any issue they don’t fully understand) to “just do X - it’s so easy” (which is a strong pointer they don’t understand the issue)

                  Which is why I sit in my rant about still using C for (relatively) new projects when safer languages exist, though - oh boy is it tempting to be dropping a quick “buffer overflows are entirely preventable in as-performant but more modern languages like rust. why did you have to write OpenSMPTD in C”, but I’m sure there were good reasons - especially for people as experienced and security focused as the OpenBSD folks.

                  1. 3

                    It’s hard if you impose the constraint that you need to support the classical UNIX model of email that was prevalent from the late 70s to the mid 90s. I was once very attached to this model but it’s based on UNIX file-system permissions that are hard to reason about and implement safely and successfully. The OpenSMTPD developers didn’t make these mistakes because they’re stupid, it’s really really hard. But it’s an unfortunate choice for a security focused system to chose to implement a hard model for email rather than making POP/IMAP work well, or some other approach to getting email under the control of a the recipient without requiring priviledges.

                2. 1

                  Not sure any of these are true, but more of a self-imposed traditional limitation.

                  Lower ports being bindable by root only could easily be removed; given linux has better security mechanisms to restrict lower port binding, like selinux, I’m not even sure why the kernel still imposes this moronic concept on people. Mail delivery (maildir, mbox, whatever zany construct) can also be done giving limited rw access to the specific user and the MDA. hell, MAIL on my system just points to /var/spool/mail which is owned by root anyhow.

                  1.  

                    selinux isn’t everywhere.

            1. -3

              About implemented useless features: “What is so wrong about […]”
              About non-implemented useless features: “Support has borders”
              We really don’t need to get too detailed to affirm that ShitDSystemD sucks, even because most people that says the contrary are not aware of the internal details (or do not care about technical quality at all)

              1. 5

                To not let people completely aimless about my statement (and let new downvotes consider using “unkind” instead of “troll”).

                Main Problems:
                • Too big
                • Is becoming a hard dependency of many projects

                Those are the biggest problems (the rest, such as bricking your BIOS, are bonus), because as we become more dependent of it, the harder it becomes to use alternatives (gl trying to re-implement the necessary interfaces, especially when it grows ad-infinitum); Otherwise people who still cares about quality could simply use an alternative.

                1. 10

                  Don’t you think that the reason why it’s becoming a hard dependency of many projects is because it’s actually pretty useful?

                  1. 1

                    Does a bad thing being useful mean it’s use should be supported?

                    1. 3

                      Bad is a moral judgement, and software doesn’t have moral character.

                      A more obvious form of the question is “Does a complex, buggy, opaque thing being useful mean it’s use should be supported?”.

                      This form of the question is unpopular with people who insist the answer should be “no” - possibly because it makes clear the reasons why it might sometimes be “yes”.

                      1. 2

                        Your question is too generic to answer ;), also I think it’s too generic to be adequate for our “should we use systemd’ problem.

                        • Should you use your car to travel? It contains complex, buggy, opaque and user-hostile systems (user-hostile when you’re trying to fix it yourself), it influences the environment in a negative way, it’s loud, kills people, and is designed to be replaced few years after buying. But because ambulances use it, people’s lives can be saved.

                        • Do you have the source code of the UEFI implementation of your motherboard? It’s complex, buggy, closed, nobody knows what it’s doing, and it’s running every time you power on your computer. But because it can be used to bootstrap your operating system, it allows you to connect with other people.

                        • Do you eat sugar / do you smoke / drink alcohol? It’s bad for your health, but it’s tasty / can help with social interaction.

                        1. 2

                          Wether or not you should use a car is a personnal concern. Some people don’t want to be bothered and need it (ambulances, police, etc..). The problem is that distros are getting rid of the bike lanes and sidewalks because they don’t want to be bothered with alternative transportation methods, as the car do it well. And me, a longboard enthusiast that don’t need a car at all to move around the city is forced into it, even though I neither need or want it.

                          1. 1

                            Wether or not you should use a car is a personnal concern.

                            I don’t agree. There are lots of people who would like to limit other people to have cars. Also lots of people who are bothered because 1 person has more than one car. Also, the environment that we live in is influenced by other people having cars (noise, pollution, sidewalks as parking spots, etc). So, I understand that me having a car can influence another person in a negative way. So since it influences another person, it’s not a personal concern. There’s also a non-zero risk that the car owner does some damage with it, since using a car involves manipulating a high energy object, that’s why the car insurance exists (disclaimer: I own a car and I’m not against using it).

                            The example with removing bike lanes and sidewalks seems adequate, but maybe the distro does it because the majority of people uses cars. So I’m not sure is it sensible to try to limit the optimization of the city for the majority, because a small percent of citizens desn’t want it. Also, removing sidewalks and bike lanes is not the goal here. Future features that will be possible to be implemented by optimizing the city for cars are the goals.

                            1. 2

                              the environment that we live in is influenced by other people having cars (noise, pollution, sidewalks as parking spots, etc). So, I understand that me having a car can influence another person in a negative way

                              This is where the metaphor stops applying. Of course an init system has zero impact on the environment, and does not directly impact other people. If we both use gentoo, and my system runs systemd while yours uses openrc, you won’t be bothered at all, no matter how bad I tune/use my system.

                              Back when systemd appeared, I was running archlinux, so I mostly remember the design choice for this particular distro. When they decided to make systemd the default init, many people asked why a tinkerer distro like this would enforce the init system on its users (it was possible at the time to replace the init system, SysV with openrc or runit fairly easily). The developers/maintainers stated that they chose systemd because it would unify how the services would be managed, so they would not have to maintain all the scripts for other daemon managers. They wanted a single way to write service files so they won’t be bothered with bugs in shell init scripts.

                              They did not remove the sidewalks and bike lanes because people used cars. They forced users into it so they won’t have to maintain the sidewalks and bike lanes. The choice didn’t come from the users at all, it cames from the distro makers.

                              edit: Found to official communication from 8 years ago! https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530 There were more than just the initscripts part obviously, as systemd solves real problems. I just remember getting into a discussion where the main argument was that maintaining init scripts was a waste of time for the devs because with systemd provides a unique interface for them, that distro can share together in a common effort (I still believe each distro maintain their own service files and don’t use a common source…).

                    2. 7

                      such as bricking your BIOS

                      This was buggy firmware that bricked itself when no EFI variables were set; such behaviour should be a factory reset instead. And it’d be exposed regardless of init system if you mounted the EFI variable filesystem.

                      1. 3

                        yep, but in SystemD case specifically, it needs the efivars to be mounted writable, according to Poettering because SystemD writes to it (with systemctl reboot --firmware)

                        1. 3

                          And that is systemd’s fault exactly how?

                          1. 3

                            Why does it need to write to the EFI filesystem every single boot?

                            No seriously, the whole point of having EFI on a separate filesystem is to protect the security and integrity of the boot process. systemd writes data to what should be a read-only partition unless you’re altering your bootloader.

                            1. 1

                              First, I think you’re mistaking EFI Filesystem (the FAT32 blob store with kernels) for EFIvars, NAND-backed kv configuration store. Systemd only writes to the latter when you explicitly invoke a reboot-to-firmware. And for that you need efivars mounted as rw.

                              EDIT: Also, traditionally, the former has always been mounted as rw somewhere under /boot, just like in BIOS days. And no one complains about it. And for EFIvars there are other customers too: gdisk, syslinux, grub2, efibootmgr, fwupd’s fwupdmgr and Gnome-Software frontends…

                              1. 1

                                If systemd needs to write to the EFI filesystem it should mount it read-write in a private namespace, not in the global namespace. I understand that namespaces are a relatively new feature but they’re not that new honestly.

                                1. 3

                                  That’s much more actionable feedback. Makes sense to me.

                                  1. 2

                                    If systemd needs to write to the EFI filesystem it should mount it read-write in a private namespace, not in the global namespace.

                                    Fine, but what about the half-dozen other clients, that expect writable efivars in the global namespace? Especially efibootmgr and fwupd.

                                    1. 1

                                      They should do the same IMO. This is an area you can permanently brick your machine, needing a little bit of inconvenience is a good thing

                                      1. 1

                                        Okay, that might work, can’t disagree here.

                          2. 2

                            I don’t see how that’s an excuse. It wouldn’t have happened if not for the actions of systemd’s developers.

                            1. 7
                              1. The EFI spec says you can reboot to firmware by wiping EFI vars.
                              2. Someone shipped hardware that is bricked if you follow the EFI spec.
                              3. Someone else shipped free software that follows the EFI spec.

                              How is it that the free software that follows the spec attracts more blame than the paid hardware that doesn’t?

                      1. 8

                        Inside of the home directory a file ~/.identity […]

                        How about no?

                        Landing systemd-homed adds over twenty-one thousand lines of code to systemd.

                        All written in one of the most dangerous and unsafe languages ever invented.

                        1. 13

                          All written in one of the most dangerous and unsafe languages ever invented.

                          Just like the vast majority of the code in Unix/Linux…

                          1. 12

                            I think it’s reasonable to accept the state of existing applications, but at the same time not wanting to add more gasoline to the tire fire.

                            (Another example: My home directory is read-only. Existing applications continue to work, but new applications better not dump their stuff into $HOME, because that just won’t work.)

                            1. 2

                              What’s the best language to replace C? Rust? I’ve read a lot about how the multi-platform support simply isn’t there yet.

                              1. 9

                                To be frank, systemd’s one and only platform is Glibc/Linux, well supported by rust.

                                1. 2

                                  This is true, systemd doesn’t really target anything other than Linux, and x86 is probably the most dominant platform.

                                2. 7

                                  Rust multi-platform support in general is very good. The whole toolchain is build with cross-compilation in mind and it avoids hard to rely on specifics.

                                  But the set of platforms Linux has been ported to through GCC and that LLVM (and thus Rust) has support for are not the same. Also, some smaller platforms needs some practical care. e.g. we track ~5-10 bugs on OpenWRT that have more to do with specifics of their SDK/library layout then anything language related. Someone with OpenWRT knowledge will probably get that fixed in a week or so. Some of them are filed by me, but I only used OpenWRT at a glance and we don’t do “amateur fixes”. It’s not like Rust programs don’t run on OpenWRT, but they are a little too big, which just isn’t acceptable on that platform.

                                  So… Yessish, Rust is an option, if you are willing to help the project out a little.

                                  Also, when systemd started, Rust was an experimental language, so I would call support abysimal ;).

                                  1. 2

                                    Thanks a lot for this clarification.

                            2. 7

                              Yeah we finally establish standards for not littering home directories with dotfiles beyond what we absolutely necessarily need and systemd just goes and shits all over that with yet another path directly under $HOME.

                              1. 5

                                It’s pretty much the “our rules are for other people to follow” crowd, same as Flatpak (”.var is fine, because we decided it’s fine, and if you don’t like it we will just add it to the XDG spec to say it’s fine”).

                            1. 8

                              No joke, I’m terrified that some major player in online music like Spotify, CD Baby, or Bandcamp will go out of business, and that thousands of bands will instantly vanish as if they never existed.

                              I was foolishly ready to just go all-in on Spotify, until one day Dashboard Confessional’s entire back catalog vanished, with no explanation. That music was foundational to my childhood! Not being able to listen to it was completely unacceptable. It eventually returned, still with no explanation, but my trust did not. It’s one thing if a band never makes it onto Spotify for some reason, like Tool was missing for years, but it’s quite another thing when e.g. Lizzo’s early songs disappear without a trace from my playlists that I’ve carefully curated. Why can’t you listen to early Lizzo? It’s not like she’s unpopular! Where is Batches & Cookies? What do I do with this hole in my mix?

                              Now I’m trying to back up all of my old CDs before they expire: https://www.vice.com/en_us/article/mg9pdv/the-hidden-phenomenon-that-could-ruin-your-old-discs Unfortunately, some of them already have scratches. I was trying to back up some of my LPs too, but that was much more of a chore and the recordings didn’t turn out that great.

                              I’m trying to buy copies of the new music I care about, but it’s getting hard these days. Big names don’t have their music on Bandcamp, and there aren’t many other options for buying lossless files (which I insist upon, since preservation is my goal). Sometimes it’s still possible to buy CDs, but small bands don’t print them because the economics don’t make sense anymore. My insignificant band made the mistake of printing 1000 CDs in 2010, since that was the minimum amount to get a reasonable price on “real CDs”, and the odds that we ever sell them all is essentially zero. A run of 100 CD-Rs is doable, but those aren’t nearly as durable as pressed CDs, so why bother, if longevity is a priority?

                              People need to own the books, music and other art that is an important part of their inner lives.

                              1. 3

                                No joke, I’m terrified that some major player in online music like Spotify, CD Baby, or Bandcamp will go out of business, and that thousands of bands will instantly vanish as if they never existed.

                                A key point here is that Bandcamp at least sells you the totally unencumbered bits in whatever format you want. I download all my purchases in lossless. BIG fan of that platform.

                                If they ever go away, you at least don’t lose the music you’ve already bought!

                                1. 3

                                  Big names don’t have their music on Bandcamp

                                  This is mostly true, but I was delighted recently to realize that Death Cab for Cutie is on there. Some of the older Decemberists albums, too. If you’re looking for a big-name band, it’s a reasonable assumption that they aren’t on Bandcamp, but it never hurts to check.

                                  1. 2

                                    I think more and more artists are starting to see the benefits of Bandcamp.

                                  2. 2

                                    one day Dashboard Confessional’s entire back catalog vanished, with no explanation. That music was foundational to my childhood! Not being able to listen to it was completely unacceptable. It eventually returned, still with no explanation, but my trust did not. It’s one thing if a band never makes it onto Spotify for some reason, like Tool was missing for years, but it’s quite another thing when e.g. Lizzo’s early songs disappear without a trace from my playlists that I’ve carefully curated. Why can’t you listen to early Lizzo? It’s not like she’s unpopular! Where is Batches & Cookies? What do I do with this hole in my mix?

                                    It’s no consolation, but the internet is on position that songs disappearing from streaming services are only fault of the record companies that upload them in the first place.

                                    1. 3

                                      But there’s no transparency! There’s no changelog of “Oh yeah, here we had to take down old Lizzo songs b/c her previous record company hates her now, but they’ll be back once her old contract runs out.” It could be the record company, or it could be technical difficulties, or it could be Lizzo doesn’t like the way she used to sound and doesn’t want you to hear her old stuff. It’s a total mystery. I think that bothers me at least as much as the music disappearing.

                                      1. 2

                                        It’s no consolation, but the internet is on position that songs disappearing from streaming services are only fault of the record companies that upload them in the first place.

                                        Very true. Don’t kill the messenger and get angry at the streaming service, BUT it still does highlight the fact that music you care about could easily sink beneath the waves and become inaccessible to you.

                                        So the question then is “How much do you care?”

                                        For a lot of people the answer is “Not enough” but for some of us it’s worth the few hundred dollars and man days to fix the problem in a sustainable way.

                                    1. 2

                                      Ericsson has offices in all over the world, including China, but most of the major decisions still happen in Sweden/Europe.

                                      1. 3

                                        Doesn’t mean much if technical implementation is developed someplace else.

                                        1. 3

                                          a lot of technical implementation is also in Europe, but sometimes in cheaper labor european countries. Also, they usually work with very “waterfall” style of development, where smaller offices just implement some big vision of a big office

                                        2. 2

                                          Same for Nokia.

                                        1. 11

                                          For the systemd units we ship ourself or use in production for Arch Linux, I’ve made this WIP page describing a more variety of hardening measures.

                                          https://wiki.archlinux.org/index.php/Security_package_guidelines#Systemd_services

                                          1. 1

                                            I wish more arch packages would use these.

                                            UNIT                                 EXPOSURE PREDICATE HAPPY
                                            NetworkManager.service                    7.7 EXPOSED   🙁    
                                            accounts-daemon.service                   9.6 UNSAFE    😨    
                                            alsa-state.service                        9.6 UNSAFE    😨    
                                            auditd.service                            9.5 UNSAFE    😨    
                                            bluetooth.service                         6.8 MEDIUM    😐    
                                            bolt.service                              5.2 MEDIUM    😐    
                                            colord.service                            8.8 EXPOSED   🙁    
                                            dbus.service                              9.6 UNSAFE    😨    
                                            dm-event.service                          9.5 UNSAFE    😨    
                                            emergency.service                         9.5 UNSAFE    😨    
                                            fwupd.service                             7.4 MEDIUM    😐    
                                            gdm.service                               9.7 UNSAFE    😨    
                                            getty@tty1.service                        9.6 UNSAFE    😨    
                                            lvm2-lvmetad.service                      9.5 UNSAFE    😨    
                                            lvm2-lvmpolld.service                     9.5 UNSAFE    😨    
                                            polkit.service                            9.6 UNSAFE    😨    
                                            rescue.service                            9.5 UNSAFE    😨    
                                            rngd.service                              8.6 EXPOSED   🙁    
                                            rtkit-daemon.service                      7.1 MEDIUM    😐    
                                            shadow.service                            9.6 UNSAFE    😨    
                                            systemd-ask-password-console.service      9.3 UNSAFE    😨    
                                            systemd-ask-password-wall.service         9.4 UNSAFE    😨    
                                            systemd-coredump@0.service                3.1 OK        🙂    
                                            systemd-initctl.service                   9.3 UNSAFE    😨    
                                            systemd-journald.service                  4.3 OK        🙂    
                                            systemd-logind.service                    2.7 OK        🙂    
                                            systemd-timedated.service                 2.1 OK        🙂    
                                            systemd-timesyncd.service                 2.0 OK        🙂    
                                            systemd-udevd.service                     7.0 MEDIUM    😐    
                                            thermald.service                          9.6 UNSAFE    😨    
                                            udisks2.service                           9.6 UNSAFE    😨    
                                            upower.service                            2.1 OK        🙂    
                                            user@1000.service                         9.4 UNSAFE    😨    
                                            wpa_supplicant.service                    9.6 UNSAFE    😨    
                                            
                                            1. 2

                                              reply

                                              Most of them come from upstream it seems although a valid one is the shadow service https://git.archlinux.org/svntogit/packages.git/tree/trunk/shadow.service?h=packages/shadow

                                              And do note that the exposure rating systemd-analyze security is a bit strange. For example PrivateNetwork= is rated as 0.5 points but NetworkManager relies on network to be available.

                                              But there is certainly a lot to improve and some upstreams happily merge system hardening patches!

                                          1. 2

                                            I use AdGuard on all my devices that support it.

                                            1. 1

                                              That’s the second translation extension which I use that was “attacked” by Mozilla. The other one was S3.Translator after a request to collect statistics of use.

                                              1. 3

                                                That’s removal from addons.mozilla.org. Is S3.Translator also blocked for side-loading?

                                                  1. 3

                                                    FWIW we have been delisted (wiped from the face of the AMO, automatic updates disabled, but people can install/still use the extension) rather than blocked (forcibly disabled on every users’ machine).

                                                    1. 2

                                                      Jesus. If Mozilla prevents me from using tridactyl, then it’s likely that I move back to Chrome. Thank you for dealing with it. It sure looks frustrating.

                                                1. 8

                                                  Wow, chown() on login. I trust that he’s right that chown()’ing the home dir recursively will happen extremely rarely, probably never for most people, and that the kernel would eventually add features to make it completely unnecessary. However, a (not so) quick find ~ | wc -l tells me I have 19 517 661 (19.5 million) entries which would have to be individually chown()’d. He says recursive chown is fast, but I imagine it’d take quite a while for me. Just listing all the files and piping to wc -l took 6 minutes.

                                                  1. 2

                                                    Maybe UID/GID namespaces could be utilized instead? One on-disk ID, preferably random-ish, one synthesized per system.

                                                    That smells of WNT though.

                                                    1. 1

                                                      Just listing all the files and piping to wc -l took 6 minutes

                                                      Did you do that >/dev/null? Otherwise, the bottleneck was probably your terminal emulator :)

                                                      But yeah, if we’re talking single mount per home dir, I think the UID should be a mount option.

                                                      1. 2

                                                        As ClashTheBunny said, wc soaks up everything and the terminal emulator doesn’t print anything. The full command was sudo find ~ | pv | wc -l (sudo because some yocto stuff has weird permissions), where pv prints stats (total bytes read, bytes/sec, elapsed time) every second. pv should affect anything, since pv </dev/zero >/dev/null runs at 10 GiB/s.

                                                        Now, with my CPU running at full speed (the previous times was from a downclocked CPU because I have TLP set to downclock when I’m on battery) and with warm disk caches, time sudo find ~ >/dev/null completes in 22 seconds and time sh -c 'sudo find ~ | pv | wc -l' takes 25 seconds. While those times are far more reasonable, they’re still fairly long (and I bet running chown on each entry would take significantly longer).

                                                        1. 1

                                                          When you pipe to wc there is only summary output at the end, so it was all disk IO.

                                                      1. 2

                                                        This is interesting, in a RISKS Digest sort of way:

                                                        LLVM will now remove stores to constant memory (since this is a contradiction) under the assumption the code in question must be dead. This has proven to be problematic for some C/C++ code bases which expect to be able to cast away ‘const’. This is (and has always been) undefined behavior, but up until now had not been actively utilized for optimization purposes in this exact way. For more information, please see: bug 42763 and post commit discussion.

                                                        The post-commit discussion links to code in BusyBox’s ash which would fail if built by a clang based on this llvm version. I suppose the RISK is a build breaking with a newer clang, people cursing and going back to the older clang instead of fixing their code, and missing out on improved bug reporting or similar.

                                                        1. 1

                                                          Interesting indeed, to be honest, I’m not totally sure of the value of an optimization like that. Is there any case that would actually improve performance and not just break things?

                                                          1. 1

                                                            Less code → room for other code in L1i cache

                                                            1. 2

                                                              But why would any correct code be writing to const memory? It doesn’t make sense to optimize incorrect code anyway.

                                                        1. 3

                                                          A lot of talk about Google. People forget that the only (profitable) Google business is ads. Once it is gone, everything else is gone too.

                                                          1. 2

                                                            Play Store? GCP?

                                                          1. 2

                                                            Buy a domain name and then do everything that people are saying here. Fastmail allows custom domains. For each new registration, create new email name. For example: for Hilton.com, when registering provide hilton@yourdomain.com. In fastmail you can catch all the names. That will allow to understand who sold your email later.

                                                            1. 6

                                                              I think that it is valuable to own your own domain, and I endorse this advice, but I do have a caveat to add to it.

                                                              Generally speaking, using a custom domain for your email adds attack surface to any account linked to your email address. See this story about the Twitter handle @N. You very likely want to have a non-custom-domain email which you use for account recovery purposes.

                                                              For me, the biggest thing about moving away from Google products is not actually that it loses the features or the network effects (Google is prone to shutting down everything I like, anyway), but that it loses the resilience against social engineering. Nothing like that is ever perfect, but at least you should try to minimize how many separate companies’ customer service process are part of your attack surface, and pick them carefully.

                                                              I’m using my Google Employee hat here not to give these words greater weight, but to disclose my bias.

                                                              1. 4

                                                                In addition, the ICANN recommends registrants use an external address for administrative domain contacts: https://www.icann.org/en/system/files/files/sac-044-en.pdf

                                                                1. 1

                                                                  This is exactly the kind of thing I was looking for, thank you for posting!

                                                                2. 1

                                                                  Thank you for pointing this out. Unless I’m missing some piece in the chain, that would be: 1. The registrar 2. The DNS provider and 3. the email service, right? (I’m under the impression that with FastMail, they can provide both 2 and 3, but I’d have to check)

                                                                  1. 1

                                                                    That agrees with my analysis, yes.

                                                                3. 3

                                                                  That will allow to understand who sold your email later.

                                                                  More importantly, it’ll make your identities across different websites unlinkable. Or at least harder to link.

                                                                  1. 1

                                                                    Would + tags (like you+hilton@example.net) help with that or are spammers getting smart and stripping them out?

                                                                    1. 2

                                                                      Some services [mistakenly] consider a plus character not valid for use in an email address.

                                                                      1. 1

                                                                        I’ve seen spammers that know about catch-all domains and are striping even the unique part. Oh well.

                                                                    1. 2

                                                                      Playing board games and preparing to DM a Dungeons & Dragons campaign.

                                                                        1. 17

                                                                          Update: There will be a dot release shortly, but only so we affect less people. If you don’t take or get the dot release, you’ll still be able to see and verify the new certificate (updated through kinto, for remote settings updates).

                                                                          More updates will happen in https://discourse.mozilla.org/t/certificate-issue-causing-add-ons-to-be-disabled-or-fail-to-install/39047

                                                                          1. 2
                                                                            429 Too Many Requests
                                                                            nginx
                                                                            
                                                                            1. 1

                                                                              Yikes. Look at Bugzilla in the meantime then :(

                                                                          1. 3

                                                                            That’s a very nice essay, but—what do you suggest as an alternative?

                                                                            I don’t mean this in a negative way, at all, if you make a new os with a new paradigm that looks cool, I will happily switch to it.

                                                                            1. 2

                                                                              If it were that easy to give a good alternative, I guess we wouldn’t be in the situation about having to think about one.

                                                                              My best guess, and what I would like to urge people to do, is to try and think contrary to established metaphors and designs. I’ve recently been thinking about a system that tries to meld operating system, programming language and compiler more or less in to one component, but it’s too vauge to to put it into words properly. Most alternatives will be failures, but hopefully controlled failures we will be able to learn from.

                                                                              1. 3

                                                                                I’ve recently been thinking about a system that tries to meld operating system, programming language and compiler more or less in to one component

                                                                                Sounds a bit like templeos. See http://www.codersnotes.com/notes/a-constructive-look-at-templeos/

                                                                                1. 2

                                                                                  There’s Synthesis OS, written back in 92 that today would be considered a “just-in-time” operating system. And it could run SunOS binaries faster than SunOS (on the same hardware).

                                                                                  1. 1

                                                                                    some pointers, might be invalid: Pharo, Emacs, Inferno,

                                                                                    1. 1

                                                                                      Urbit has ambitions of melding those things, as well as a universal concept of network identity and ownership thereof.

                                                                                      1. 1

                                                                                        I liked the write-up and the interesting historical tidbits in there.

                                                                                        I strongly agree with thinking contrary to established design - many established patterns might actually have quite low ceilings. Merging the OS, compiler and PL etc. could make sense (that’s what Smalltalk looks like). Much of the tedium and weakness in current systems is at the integration points - perhaps the dividing the ‘whole computing system’ into these specific sub-concepts needs to be rethought?

                                                                                        In a similar vein, see https://eighty-twenty.org/2016/05/05/unix-is-incorrectly-factored and https://programmingmadecomplicated.wordpress.com/2017/08/12/there-is-only-one-os-and-its-been-obsolete-for-decades/

                                                                                    1. 49

                                                                                      It is Firefox’s moral obligation to stop adding anti-features (DRM, Pocket, all the start page spam).

                                                                                      1. 24

                                                                                        For Pocket and the start-page stuff, look at it this way: Browsers are very complex systems, and therefore very expensive to build and maintain. Chrome’s funding comes from Google’s datamining of its users, which I think we can agree is a downside, but Firefox’s funding… also comes from Google’s datamining of its users. If you don’t think Firefox should be supported, that’s fine, and if you want to support Firefox yourself through a monthly donation or just allowing ads on the start page, that’s even better, but I don’t think it’s fair to expect Google’s users pay for Firefox so you can have it for free.

                                                                                        As for DRM, Firefox may be the #2 browser vendor but it is not an immovable object. When an irresistible force comes along sometimes the wisest course of action is to get out of the way and live to fight another day, rather than get crushed to death on the spot. If your only regular user is Richard M. Stallman, you may have the moral high ground but you’ve forfeited your chance to improve society in general.

                                                                                        1. 11

                                                                                          Just a reminder that Google has created the DRM spec that Firefox was forced to adopt (forced in the sense that a browser that is blocked by Netflix would lose users, and influence, and thus leaving only DRM-supporting browsers on the market anyway).

                                                                                          Chrome and Safari have DRM enabled by default, without warning.

                                                                                          In Firefox at least it’s an optional component that you can choose not to download and you’re warned whenever a site tries to use it. So if you take a stance against DRM, then Firefox in the default configuration (i.e. without DRM) is still a great choice.

                                                                                          Mozilla has bought Pocket, so it’s no longer a 3rd party, but a fancy Firefox bookmark tool. It doesn’t use any noticeable resources if you don’t use it, and you can disable it if you can’t stand having an icon.

                                                                                          1. 9

                                                                                            Pocket is easy to disable, and the DRM is needed to play videos on many web sites. Without DRM, Firefox wouldn’t have a large enough user base to make the project sustainable. And remember that the Firefox code base feeds a number of downstream free-software projects

                                                                                            If you don’t want features like Pocket or DRM in the browser executable, there are lots of Firefox forks to choose from, which contain only free software. I personally use Fennec F-Droid on my mobile.

                                                                                            1. 8

                                                                                              And the DoH deal with cloudflare..

                                                                                              1. 2

                                                                                                My ISP actively redirects failed DNS lookups to advertisement pages. Cloudflare is probably going to be a little nicer (at least for now…)

                                                                                              2. 3

                                                                                                I’m not sure it is actually Mozilla’s moral obligation. But there’s quite a bit of crap (Pocket, sponsored stories on the start page and silently installing extensions in the background to promote TV shows are what I remember, but I’m sure there is more) going on with Firefox that stopped me to switch back from Chromium for quite a while.

                                                                                                1. 23

                                                                                                  Perfect being the enemy of good applies here.

                                                                                                  I quite dislike it when people use minor issues as an excuse not to do anything about major issues. Is Firefox perfect? No. Is it nearly as problematic as Chrome? No.

                                                                                                  There will never be a platonic ideal mass-adopted web browser. We can choose between good and bad though.

                                                                                                  1. 2

                                                                                                    “Problematic” is the salient term here.

                                                                                                    The issues raised in the linked article and elsewhere are valid, yes, but they do not affect the end user at this time.

                                                                                                    I personally prefer using Chrome for the simple reason that it works on all my devices and keeps browsing history, passwords etc in sync. Does Firefox do that? Maybe. Is it worth it for me to switch and try to research it? No.

                                                                                                    A less diverse browser ecosystem is probably bad in the long run, but just as with real ecosystems, the damage is so diffuse in both time and space that any action an individual can take is basically useless.

                                                                                                    Personally I’d prefer if Firefox was much better than Chrome (just like Chrome was much better than IE/Firefox when it debuted). That would make the switch much easier for more people.

                                                                                                    1. 11

                                                                                                      I personally prefer using Chrome for the simple reason that it works on all my devices and keeps browsing history, passwords etc in sync. Does Firefox do that? Maybe. Is it worth it for me to switch and try to research it? No.

                                                                                                      For completeness sake: yes, it has. It’s called Firefox Sync and is a neat product. You can even have your own sync server if you prefer.1 Firefox for mobile is a really good product on all devices and Firefox Klar (the “quickly search something in a private fashion” is a really good addition).

                                                                                                      1. 5

                                                                                                        Firefox Klar

                                                                                                        A.k.a. Firefox Focus outside of Germany.

                                                                                                      2. 0

                                                                                                        …any action an individual can take is basically useless.

                                                                                                        In other words: why bother voting anyway? It’s not like my vote means anything… /s

                                                                                                        1. 0

                                                                                                          You’re taking my words out of context.

                                                                                                          I’m saying that in relationship to the environment we live in - carbon in the atmosphere, pressure on water tables and arable land, fish in the ocean, political systems corrupted by resource extraction.

                                                                                                          If I decide to buy a car with a better mileage, or not have a car at all, the decision can be negated by a family moving somewhere where they need 2 cars instead of one, or a well-targeted ad campaign for a gas-guzzler in China.

                                                                                                          A vote in a well-run democracy is the opposite of that. If you don’t screw up the process, you can be reasonably sure that your vote will be present on a tally-board on the evening news in a few days.

                                                                                                          1. 1

                                                                                                            I don’t think context matters in this instance. If you think your actions only matter if they have a visible effect on the news… there must be something I’m missing, because I don’t know what to tell you. That’s not logic, that’s instant gratification.

                                                                                                            1. 1

                                                                                                              If you think your actions only matter if they have a visible effect on the news…

                                                                                                              I am not thinking that, you are misreading my position. I’m happy to clarify though.

                                                                                                              That’s not logic, that’s instant gratification.

                                                                                                              I’m not sure what you mean here. Again, I feel we’re talking at cross purposes.

                                                                                                              1. 1

                                                                                                                I apologize, I know I wouldn’t enjoy being misstated. Please do clarify.

                                                                                                                1. 1

                                                                                                                  I’ll try.

                                                                                                                  In my first comment on this thread, I made the analogy between the browser ecosystem and the wider IRL ecosystem that exists on this planet.

                                                                                                                  My basic point was that any action I take as an individual has very little to no impact whatsoever in a positive direction in any of these ecosystems. I made this comment after my (unstated) reflection on the limitations of private, volunteer action in regards to environmentalism. I believe the only viable way to lessen the impact of humanity on the environment are communal - restrictions on certain products, taxation of consumption of certain goods.

                                                                                                                  Scaling that down to the browser ecosystem, I don’t believe that supporting one “free to play”, ad-supported browser over another will significantly help keeping the internet free and open.

                                                                                                                  You stated in your comment:

                                                                                                                  In other words: why bother voting anyway? It’s not like my vote means anything… /s

                                                                                                                  (sarcasm tag noted!)

                                                                                                                  A counter-example:

                                                                                                                  Voters in Sweden have enabled a party (the Green party) to enter government, and leverage this power to ensure that gasoline taxes in Sweden are around 50% of the cost of a liter of fuel at the pump.

                                                                                                                  Thus the result of an election led indirectly to a policy that in its turn incentivizes consumers to choose cars with better gas mileage. This in its turn contributes to Sweden trying to reach its internationally agreed-upon carbon emission target.

                                                                                                      3. 1

                                                                                                        To me it’s not about being perfect. It is about what fit my needs and principles better. And if I have to choose between a software that talks about caring about its users and being privacy oriented while obviously doing the opposite (I.e. silently installing extensions in the background without my consent or integrating code that leads to selling my browsing data — remember the Cliqz integration?) and one that has not done so (I’m talking about chromium here), the choice is easy to me.

                                                                                                        In my opinion Chromium is technically the better browser while being more transparent about privacy. That’s what counts to me. I don’t care about perfection.

                                                                                                  1. 5

                                                                                                    I’m at my family’s home, so the setup is a little spartan yet cluttered https://imgur.com/a/XHdQidI (and I apologize for the uneven framing)

                                                                                                    1. 3

                                                                                                      Ah Agricola, nice.

                                                                                                    1. 13

                                                                                                      I may as well join in.

                                                                                                      I’ve had a light conversation with SirCmpwn before and he doesn’t care for macros either, which I find foolhardy, but I’ll focus on just this article.

                                                                                                      The inertia of “what I’m used to” comes to a violent stop when they try to use Go. People affected by this frustration interpret it as a problem with Go, that Go is missing some crucial feature - such as generics. But this lack of features is itself a feature, not a bug.

                                                                                                      I use a number of wildly different languages, including Common Lisp, APL, and most recently Ada; each of these languages is lacking things the other has, but it also vastly more suited to other tasks than the rest. I’ve never used Go. Unlike these three languages I’ve mentioned, which have perfectly good reasons for lacking whatever it is they lack, Go very often has poor reasons or perhaps even no reasons, although I don’t skulk around the mailing lists or whatnot.

                                                                                                      For a good example, take a look at this; it’s my understanding Go lacked a proper mechanism for determining time and many people critiqued this, but daddy Google didn’t care until someone important was hit by it. This is a good example of the problems caused by a language that is not only uncustomizable by the users, but is designed by people who don’t care and won’t care. Unless you’re someone, Google doesn’t care about what you think and the language certainly doesn’t, considering it is designed at every point to take away programmer choice.

                                                                                                      Go strikes me as one of the most conservative programming languages available today. It’s small and simple, and every detail is carefully thought out. There are very few dusty corners of Go - in large part because Go has fewer corners in general than most programming languages.

                                                                                                      This isn’t equivalent to a language that is good for writing programs in. Ofttimes, a lack of edge cases in the world of the language doesn’t correspond to a lack of edge cases in real use. Take a look at Ada for a counterexample; the rules may not have a nice technical explanation, but the corresponding real world explanation is very simple, because it’s usually to prevent some manner of error.

                                                                                                      I feel that this applies to generics. In my opinion, generics are an imperfect solution to an unsolved problem in computer science.

                                                                                                      Dynamic typing as in Lisp is one solution. Ada has a nice generic system, but again, Ada was designed not for theoretical prettiness, but to actually make large systems easier to write without flaws, so generics were of course there because otherwise you get people copying and pasting code, which makes maintenance and everything else harder because you can’t tell if one of the copies is wrong or otherwise changed easily or quickly.

                                                                                                      I used to sneer at the Go maintainers alongside everyone else whenever they’d punt on generics. With so many people pining after it, why haven’t they seen sense yet? How can they know better than all of these people?

                                                                                                      Have you ever considered these people don’t know better than anyone else. Have you considered that Go is just an extension of the UNIX and C religion and people like Rob Pike are just playing their part as a priest over scared people who don’t know any better and want a panacea and a movement to join?

                                                                                                      I don’t think programming languages should compete with each other in an attempt to become the perfect solution to every problem. This is impossible, and attempts will just create a messy kitchen sink that solves every problem poorly.

                                                                                                      I’d prefer to think that’s common sense. APL and its family is the clear choice for array problems, but will fall flat against many other types of problems. What is Go actually good for? I find that poor languages, typically ALGOL clones, tend to differentiate themselves on purpose rather than anything intrinsic. You see this with Perl being a ’‘scripting’’ language, Ruby being for ’‘web services’’, Python being ’‘glue code’’, and, what, Go being for ’‘scalable programs with a focus on internet-connected services’’? The key detail to observe is these languages are all rather the same and, utterly lacking originality, attempt to dominate in a particular usage, because that’s the only way they can really be differentiated.

                                                                                                      If you disagree with this, compare Perl to PHP to Go to Python and compare those differences to those between comparing Common Lisp to Forth to APL to Ada.

                                                                                                      If you’re fighting Go’s lack of generics trying to do something Your Way, you might want to step back and consider a solution to the problem which embraces the limitations of Go instead. Often when I do this the new solution is a much better design.

                                                                                                      I felt something similar when I was writing an Ada program and, wanting to use the package system properly, was forced to structure my program in a different, albeit natural and better way. Tell me if there’s a document that lists all of Go’s design decisions and why they were taken, or am I only going to find the typical UNIX and C response of ’‘We know better. It’s better this way. Don’t consider other ways. Our way is the one true way.’’?

                                                                                                      So it’s my hope that Go will hold out until the right solution presents itself, and it hasn’t yet. Rushing into it to appease the unwashed masses is a bad idea.

                                                                                                      Go was designed for the ’‘unwashed masses’’, I mean those ’‘not capable of understanding a brilliant language, but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt.’’, straight from Rob Pike’s mouth. Go is designed to use programmers as unintelligent implementing machines, which is why it’s so opinionated. Its opinions have little or nothing to do with good programs and apparently everything to do with preventing the damage any single fool they want to use can cause or, worse, prevent a new hire who isn’t a fool from writing a good program that makes them more valuable than their peers and harder to fire. There’s no macros in Go, only the same thing, everywhere, no matter how poorly suited it is to the problem. If everyone writes Go the same, it’s easy to fire and interchange employees without friction.

                                                                                                      I could keep going on about how Go is just a continuation of UNIX, C, and so also Plan9, UTF-8, and whatever else those malign idiots create, but I believe this gets the point across well enough. The only ’‘philosophy’’ these things espouse is that the computer isn’t a tool for leveraging the mind.

                                                                                                      1. 10

                                                                                                        ’‘We know better. It’s better this way. Don’t consider other ways. Our way is the one true way.’’

                                                                                                        Hilariously, a pro-Go commenter just said to me that Go is an anti-“we know better” language.

                                                                                                        Go is just an extension of the UNIX and C religion

                                                                                                        And yet it goes against everything in the actual modern Unix world. Go likes static linking (because Linux distros), has custom syscall wrappers, a custom assembler (!), custom calling convention and weird stack setup… As a result, calling non-Go code requires either overhead (cgo) or ridiculous hacks (c2goasm), LD_PRELOAD hooks don’t work, and porting the main official Go implementation to a new OS/CPUarch combo is utter hell.

                                                                                                        1. 9

                                                                                                          Go being for ’‘scalable programs with a focus on internet-connected services’’?

                                                                                                          Two comments: the two wins go has over other languages is

                                                                                                          • (1) build/link - that its build system is fast, and it produces reasonably small static binaries, suitable for deploying into containers. This requires a fair bit of fiddling in other languages and with fairly large binaries in the outcome. Not infeasible, but certainly more than plug and play.

                                                                                                          • (2) it aligns with the sensibilities of python and ruby programmers in general, but in a typed manner, so improved maintainability with a fairly simple semantic.

                                                                                                          I’m not a go fan, but these are key good things for go.

                                                                                                          Rather write in something like Haskell or Common Lisp, but ce la vie…

                                                                                                          1. 9

                                                                                                            I had you until the last paragraph. What the heck do you find bad in UTF-8?

                                                                                                            1. 1

                                                                                                              I don’t want that to turn into its own discussion, but I have reasons aplenty and I’ll list all those that currently come to mind.

                                                                                                              Firstly, I have my own thoughts about machine text. I find the goal of Unicode, being able to have all languages in one character set, to be fundamentally misguided. It’s similar to the general UNIX attitude: ’‘Should we have the ability to support multiple standards and have rich facilities for doing so transparently? No, we should adopt a single, universal standard and solve the problem that way. The universal standard is the one true way and you’re holding back progress if you disagree!’’.

                                                                                                              Operating systems can support multiple newline conventions, as VMS did, and it would be trivial to have a format for incorporating multiple character sets into a single document without issue, but that’s not what is done. Instead, Unicode is forced and there are multiple Unicode encodings. Unicode is also filled with dead languages, emojis, and graphics-building characters, the latter being there, I think in part, because GUIs under UNIX are so poor and so turning the character set into the GUI toolkit is such an easy ’‘solution’’. I’m fully aware the other reasoning is likely to encompass graphics from other character sets, however.

                                                                                                              UTF-8 is a large, variable-length character set that can have parsing errors, which I find unacceptable. It’s backwards compatible with ASCII, which I also dislike, but at least ASCII has the advantage of being small. UTF-8 takes pains to avoid containing the zeroeth character, so as to avoid offending C’s delicate sensibilities, since C is similarly designed to not accommodate anything and expect everything to accommodate it instead. It is as if Ken Thompson thought: ’‘I haven’t done enough damage.’’

                                                                                                              UTF-8 disadvantages other languages, such as Japanese and Chinese (This isn’t even mentioning the Eastern character controversy.), by being larger than a single-minded encoding, leading several such peoples to prefer their own custom encodings, anyway. You can only add UTF-8 support to a program transparently in trivial cases, as anything more such as a text editor will break in subtle ways.

                                                                                                              There’s also that Unicode makes the distinction between characters, graphemes, and other such things that turn a simple problem into an unmanageable one. I use Common Lisp implementations that support Unicode characters, but don’t actually support Unicode, because there are so many combining characters and other such things that have no meaning to Common Lisp and so can’t be implemented ’‘correctly’’, as they would violate the semantics of the language.

                                                                                                              There are other reasons I can list, but this is sufficient.

                                                                                                              1. 10

                                                                                                                multiple standards and have rich facilities for doing so transparently

                                                                                                                Well, looks like getting everyone to agree on a way of selecting encodings turned out to be way harder than getting everyone to agree on one encoding :)

                                                                                                                And sure — we have Content-Type: what/ever;charset=MyAwesomeEncoding on the web, we can have file formats with specified encodings inside, but there’s nothing you can do about something as fundamental as plain text files. You could never get everyone to agree to use something like extended FS attributes for this, and to make it work when moving a file across filesystems… that’s just not happening.

                                                                                                                format for incorporating multiple character sets into a single document without issue

                                                                                                                Again, some format that software has to agree on. Plain, zero-metadata text fields and files are a thing that’s not going away, as much as you’d like it to.

                                                                                                                UTF-8 disadvantages other languages, such as Japanese and Chinese

                                                                                                                They often include ASCII pieces like HTML tags, brand names, whatever; you should use an actual compressor if you care about the size so much; and every character in these languages conveys more information than a Latin/Greek/Cyrillic/etc character anyway.

                                                                                                                1. 7

                                                                                                                  It seems like you don’t actually know what UTF-8 is. UTF-8 is not Unicode. Rob Pike did not design Unicode, or have anything really do to with Unicode. Those guys designed UTF-8, which is an encoding for Unicode, and it’s an encoding that has many wonderful properties.

                                                                                                                  One of those properties is backwards compatibility. It’s compatible with ASCII. You ‘dislike’ this, apparently. Why? It’s one of the most important features of UTF-8! It’s why UTF-8 has been adopted into network protocols and operating systems seamlessly and UTF-16 hasn’t.

                                                                                                                  UTF-8 doesn’t ‘disadvantage’ other languages either. It doesn’t ‘disadvantage’ Japanese or Chinese at all. Most web pages with Japanese and Chinese text are smaller in UTF-8 than UTF-16, despite the actual Japanese and Chinese text taking up 3 bytes instead of 2, because all the other bytes (metadata, tags, etc.) are smaller.

                                                                                                                  The fact is that anyone that says that Unicode ‘makes the distinction between characters, graphemes, and other such things that turn a simple problem into an unmanageable one’ doesn’t know what they’re talking about. Unicode did not create those problems, Unicode simply represents that problem. That problem exists regardless of the encoding. Code units, code points, characters, graphemes.. they’re all inherently different things.

                                                                                                                  Unicode does not have any GUI characters.

                                                                                                                  1. 2

                                                                                                                    Could you maybe elaborate the following quote?

                                                                                                                    UTF-8 disadvantages other languages, such as Japanese and Chinese (This isn’t even mentioning the Eastern character controversy.)

                                                                                                                    1. 6

                                                                                                                      I reckon it refers to the controversial Han unification, which was in China’s favour.

                                                                                                                    2. 1

                                                                                                                      It’s similar to the general UNIX attitude: ’‘Should we have the ability to support multiple standards and have rich facilities for doing so transparently? No, we should adopt a single, universal standard and solve the problem that way. The universal standard is the one true way and you’re holding back progress if you disagree!’’.

                                                                                                                      What precisely does UNIX force you into? Are you sure this isn’t also the LISP attitude as well? For example, Lispers usually sternly glare over the interwebs if you dare you use anything but EMACS and SLIME.

                                                                                                                      Operating systems can support multiple newline conventions, as VMS did, and it would be trivial to have a format for incorporating multiple character sets into a single document without issue, but that’s not what is done.

                                                                                                                      You’re confusing multiple newlines in a single character encoding with multiple newlines across character encodings. You say that it would be trivial to have multiple character sets in a single document, but you clearly have not tried your hand at the problem, or you would know it to be false.

                                                                                                                      Give me twenty individual sequences of bytes that are all ‘invalid’ in twenty different character encodings, and then give me 200 individual sequences of bytes that are all ‘invalid’ in 200 different character encodings. Otherwise there is ambiguity on how to interpret the text and what encoding is used.

                                                                                                                      This problem can be seen by the people who are trying to revamp the c2 wiki. Reworking it has stalled because there are around 150 files with multiple different character encodings, and they cannot be identified, separated, and unified by the machine.

                                                                                                                      Unicode is also filled with dead languages, […]

                                                                                                                      Right, because Unicode is supposed to be a superset of all encodings. The fact it supports languages that are not used anymore is a feature, not a bug. It is important to people working in linguistics (you know, that field outside of computer science…) that any computer encoding format has a method of displaying the text that they are working with. This is important to language archival efforts.

                                                                                                                      UTF-8 disadvantages other languages, such as Japanese and Chinese (This isn’t even mentioning the Eastern character controversy.

                                                                                                                      This is outright false, but someone else has already mentioned that.

                                                                                                                      I use Common Lisp implementations that support Unicode characters, but don’t actually support Unicode, because there are so many combining characters and other such things that have no meaning to Common Lisp and so can’t be implemented ’‘correctly’’, as they would violate the semantics of the language.

                                                                                                                      Unicode allows language implementations to disallow some sets of characters for ‘security’ reasons: http://www.unicode.org/reports/tr31/

                                                                                                                      This entire rant remined me of Steve Yegge’s post “Lisp is not an acceptable Lisp”:

                                                                                                                      But what’s wrong with Common Lisp? Do I really need to say it? Every single non-standard extension, everything not in the spec, is “wrong” with Common Lisp. This includes any support for threads, filesystem access, processes and IPC, operating system interoperability, a GUI, Unicode, and the long list of other features missing from the latest hyperspec.

                                                                                                                      Effectively, everything that can’t be solved from within Lisp is a target. Lisp is really powerful, sure, but some features can only be effective if they’re handled by the implementation.

                                                                                                                  2. -4

                                                                                                                    I could keep going on about how Go is just a continuation of UNIX, C, and so also Plan9, UTF-8, and whatever else those malign idiots create

                                                                                                                    The difference is that Plan9, UNIX, C, UTF-8 and Go are all absolutely wonderful and have all had far more positive effect on the world than anything you will ever create. That’s not because they got lucky, it’s because they were designed by people that actually understand what makes things successful.

                                                                                                                    1. -5

                                                                                                                      He’s just a butthurt Lisper who’s mad his elegant, beautiful language is ignored by people who actually get stuff done. UNIX-haters and that.