1. 1

    I do not like the click-baity title, but I kept it the way it was. There are some interesting extensions listed that I never heard of before.

    1. 3

      It looks like there are some “interesting” extension nobody else has heard of before.

      Like that Web Security addon, which appears to send all your navigation data to their servers over plain http, using some homegrown “crypto” to obfuscate the details. According to their privacy policy, they build a profile for advertising purposes.

    1. 3

      The problem turns out to be some obscure FUSE mounts that the author had lying around in a broken state, which subsequently broke the kernel namespace system. Meanwhile, I have been running systemd on every computer I’ve owned in many years and have never had a problem with it.

      Does this not seem a bit melodramatic?

      1. 9

        From the twitter thread:

        Systemd does not of course log any sort of failure message when it gives up on setting up the DynamicUser private namespace; it just goes ahead and silently runs the service in the regular filesystem, even though it knows that is guaranteed to fail.

        It sounds like the system had an opportunity to point out an anomaly that would guide the operator in the right direction, but instead decided to power through anyways.

        1. 8

          A lot like continuing to run in a degraded state is a plague that affects distributed systems. Everybody thinks it’s a good idea “some service is surely better than no service” until it happens to them.

          1. 3

            At $work we prefer degraded mode for critical systems. If they go down we make no money, while if they kind of sludge on we make less but still some money while we firefight whatever went wrong this time.

            1. 8

              My belief is that inevitably you could be making $100 per day, would notice if you made $0, but are instead making $10 and won’t notice this for six months. So be careful.

              1. 4

                We have monitoring and alerting around how much money is coming in, that we compare with historical data and predictions. It’s actually a very reliable canary for when things go wrong, and for when they are right again, on the scale of seconds to a few days. But you are right that things getting a little suckier slowly over a long time would only show up as real growth not being in line with predictions.

            2. 2

              I tend to agree that hard failures are nicer in general (especially to make sure things work), but I’ve also been in scenarios where buggy logging code has caused an entire service to go down, which… well that sucked.

              There is a justification for partial service functionality in some cases (especially when uptime is important), but like with many things I think that judgement calls in that are usually so wrong that I prefer hard failures in almost all cases.

              1. 1

                Running distributed software on snowflake servers is the plague to point out.

                1. 1

                  Everybody thinks it’s a good idea “some service is surely better than no service” until it happens to them.

                  So if the server is over capacity, kill it and don’t serve anyone?

                  Router can’t open and forward a port, so cut all traffic?

                  I guess that sounds a little too hyperbolic.

                  But there’s a continuum there. At $work, I’ve got a project that tries to keep going even if something is wrong. Honest, I’m not sure I like how all the errors are handled. But then again, the software is supposed to operate rather autonomously after initial configuration. Remote configuration is a part of the service; if something breaks, it’d be really nice if the remote access and logs and all were still reachable. And you certainly don’t want to give up over a problem that may turn out to be temporary or something that could be routed around… reliability is paramount.

                  1. 2

                    And you certainly don’t want to give up over a problem that may turn out to be temporary

                    I think that’s close to the core of the problem. Temporary problems recur, worsen, etc. I’m not saying it’s always wrong to retry, but I think one should have some idea of why the root problem will disappear before retrying. Computers are pretty deterministic. Transient errors indicate incomplete understanding. But people think a try-catch in a loop is “defensive”. :(

              2. 4

                So you never had legacy systems (or configurations) to support? I read Chris’ blog regularly, and he works at a university on a heterogeneous network (some Linux, some other Unix systems) that has been running Unix for a long time. I think he started working there before systemd was even created.

                1. 3

                  Why do you say that the FUSE mounts were broken? As far as we can see they were just set up in a uncommon way https://twitter.com/thatcks/status/1027259924835954689

                  1. 3

                    It does look brittle that broken fuse mounts prevent the ntpd from running. IMO the most annoying part is the debugability of the issue.

                    1. 2

                      Yes, it seems melodramatic, even to my anti-systemd ears. It’s a documentation and error reporting problem, not a technical problem, IMO. Olivier Lacan gave a great talk last year about good errors and bad errors (https://olivierlacan.com/talks/human-errors/). I think it’s high time we start thinking about how to improve error reporting in software everywhere – and maybe one day human-centric error reporting will be as ubiquitous as unit testing is today.

                      1. 2

                        In my view (as the original post’s author) there are two problems in view. That systemd doesn’t report useful errors (or even notice errors) when it encounters internal failures is the lesser issue; the greater issue is that it’s guaranteed to fail to restart some services under certain circumstances due to internal implementation decisions. Fixing systemd to log good errors would not cause timesyncd to be restartable, which is the real goal. It would at least make the overall system more debuggable, though, especially if it provided enough detail.

                        The optimistic take on ‘add a focus on error reporting’ is that considering how to report errors would also lead to a greater consideration of what errors can actually happen, how likely they are, and perhaps what can be done about them by the program itself. Thinking about errors makes you actively confront them, in much the same way that writing documentation about your program or system can confront you with its awkward bits and get you to do something about them.

                    1. 4

                      This is really interesting to get an idea of how people are taking advantage of BSD! I now have a much nicer idea of why people are going to it (and am a bit tempted myself). That feeling of having to go through ports and simply not having 1st-class support for some software seems… rough for desktop usage though

                      1. 3
                        1. 1

                          I mean “someone talks to me about an application and I’m interested in trying it out on my system”?

                          I feel like the link to the CVE database is a bit of an unwarranted snipe here. I’m not talking too much about security updates, just “someone released some software and didn’t bother to confirm BSD support so now I’m going to need to figure out which ways this software will not work”.

                          To be honest I don’t really think that having all userland software come in via OS-maintained package managers is a great idea in the first place (do I really need OS maintainers looking after anki?). I’m fine downloading binaries off the net. Just nicer if they have out of the box support for stuff. I’m not blaming the BSDs for this (it’s more the software writer’s fault), just that it’s my impression that this becomes a bit of an issue if you try out a lot of less used software.

                          1. 4

                            As an engineer that uses and works on a minority share operating system, I don’t really think it’s reasonable to expect chiefly volunteer projects to ship binaries for my platform in a way that fits well with the OS itself. It would be great if they were willing to test on our platform, even just occasionally, but I understand why they don’t.

                            Given this, it seems more likely to expect a good experience from binaries provided by somebody with a vested interest in quality on the OS in question – which is why we end up with a distribution model.

                            1. 2

                              Yep, this makes a lot of sense.

                              I’m getting more and more partial to software relying on their host language’s package manager recently. It’s pretty nice for a Python binary to basically always work so long as you got pip running properly on your system, plus you get all the nice advantages of virtual environments and the like letting you more easily set things up. The biggest issue being around some trust issues in those ecosystems.

                              Considering a lot of communities (not just OSes) are getting more and more involved in distribution questions, we might be getting closer to getting things to work out of the box for non-tricky cases.

                              1. 8

                                software relying on their host language’s package manager

                                In general I’m not a fan. They all have problems. Many (most?) of them lack a notion of disconnected operation when they cannot reach their central Internet-connected registry. There is often no complete tracking of all files installed, which makes it difficult to completely remove a package later. Some of the language runtimes make it difficult to use packages installed in non-default directory trees, which is one way you might have hoped to work around the difficulty of subsequent removal. These systems also generally conflate the build machine with the target machine (i.e., the host on which the software will run) which tends to mean you’re not just installing a binary package but needing to build the software in-situ every time you install it.

                                In practice, I do end up using these tools because there is often no alternative – but they do not bring me joy.

                                Operating system package managers (dpkg/apt, rpm/yum, pkg_add/pkgin, IPS, etc) also have their problems. In contrast, though, these package managers tend to at least have some tools to manage the set of files that were installed for a particular package and to remove (or even just verify) them later. They also generally offer some first class way to install a set of a packages from archive files obtained via means other than direct access to a central repository.

                                1. 3

                                  For development I use the “central Internet-connected registry.”, for production I use DEB/RPM packages in a repository:

                                  • forces you to limit the number of dependencies you use, otherwise too much work to package them all;
                                  • force you to choose high quality dependencies that are easy to package or already packaged;
                                  • makes sure every dependency is buildable from source (depending on language);
                                  • have an “offline” copy of the dependencies, protect against “left-pad” issues;
                                  • run unit tests of the dependencies during package build, great for QA!;
                                  • have (PGP) signed packages that uses the distribution’s tools to verify.

                                  There are probably more benefits that escape me at the moment :)

                        2. 1

                          That feeling of having to go through ports and simply not having 1st-class support for some software seems… rough for desktop usage though

                          What kind of desktop software do you install from these non-OS sources?

                          1. 2

                            Linux is moving more and more towards Flatpak and Snap for (sandboxed) application distribution.

                            1. 2

                              I remember screwing around with Flathub on the command line in Fedora 27, but right now on Fedora 28, if you enable Flatpak in the Gnome Software Center thingy, it’s actually pretty seamless - type “Signal” in the application browser, and a Flatpak install link shows up.

                              With this sort of UX improvements, I’m optimistic. I feel like Fedora is just going to get easier and easier to use.

                        1. 8

                          Speaking as a C programmer, this is a great tour of all the worst parts of C. No destructors, no generics, the preprocessor, conditional compilation, check, check, check. It just needs a section on autoconf to round things out.

                          It is often easier, and even more correct, to just create a macro which repeats the code for you.

                          A macro can be more correct?! This is new to me.

                          Perhaps the overhead of the abstract structure is also unacceptable..

                          Number of times this is likely to happen to you: exactly zero.

                          C function signatures are simple and easy to understand.

                          It once took me 3 months of noodling on a simple http server to realize that bind() saves the pointer you pass into it, so makes certain lifetime expectations on it. Not one single piece of documentation I’ve seen in the last 5 years mentions this fact.

                          1. 4

                            It once took me 3 months of noodling on a simple http server to realize that bind() saves the pointer you pass into it

                            Which system? I’m pretty sure OpenBSD doesn’t.

                            https://github.com/openbsd/src/blob/4a4dc3ea4c4158dccd297c17b5ac5a6ff2af5515/sys/kern/uipc_syscalls.c#L200

                            https://github.com/openbsd/src/blob/4a4dc3ea4c4158dccd297c17b5ac5a6ff2af5515/sys/kern/uipc_syscalls.c#L1156

                            1. 2

                              Linux (that’s the manpage I linked to above). This was before I discovered OpenBSD.

                              Edit: I may be misremembering and maybe it was connect() that was the problem. It too seems fine on OpenBSD. Here’s my original eureka moment from 2011: https://github.com/akkartik/wart/commit/43366d75fbfe1. I know it’s not specific to that project because @smalina and I tried it again with a simple C program in 2016. Again on Linux.

                                1. 1

                                  Notice that I didn’t implicate the kernel in my original comment, I responded to a statement about C signatures. We’d need to dig into libc for this, I think.

                                  I’ll dig up a simple test program later today.

                                  1. 2

                                    Notice that I didn’t implicate the kernel in my original comment, I responded to a statement about C signatures. We’d need to dig into libc for this, I think.

                                    bind and connect are syscalls, libc would only have a stub doing the syscall if anything at all since they are not part of the standard library.

                            2. 2

                              Perhaps the overhead of the abstract structure is also unacceptable..

                              Number of times this is likely to happen to you: exactly zero.

                              I have to worry about my embedded C code being too big for the stack as it is.

                              1. 1

                                Certainly. But is the author concerned with embedded programming? He seems to be speaking of “systems programming” in general.

                                Also, I interpreted that section as being about time overhead (since he’s talking about the optimizer eliminating it). Even in embedded situations, have you lately found the time overheads concerning?

                                1. 5

                                  I work with 8-bit AVR MCUs. I often found myself having to cut corners and avoid certain abstractions, because that would have resulted either in larger or slower binaries, or would have used significantly more RAM. On an Atmega32U4, resources are very limited.

                              2. 1

                                Perhaps the overhead of the abstract structure is also unacceptable..

                                Number of times this is likely to happen to you: exactly zero.

                                Many times, actually. I see FSM_TIME. Hmm … seconds? Milliseconds? No indication of the unit. And what is FSM_TIME? Oh … it’s SYS_TIME. How cute. How is that defined? Oh, it depends upon operating system and the program being compiled. Lovely abstraction there. And I’m still trying to figure out the whole FSM abstraction (which stands for “Finite State Machine”). It’s bad enough to see a function written as:

                                static FSM_STATE(state_foobar)
                                {
                                ...
                                }
                                

                                and then wondering where the hell the variable context is defined! (a clue—it’s in the FSM_STATE() macro).

                                And that bind() issue is really puzzling, since that haven’t been my experience at all, and I work with Linux, Solaris, and Mac OS-X currently.

                                1. 1

                                  I agree that excessive abstractions can hinder understanding. I’ve said this before myself: https://news.ycombinator.com/item?id=13570092. But OP is talking about performance overhead.

                                  I’m still trying to reproduce the bind() issue. Of course when I want it to fail it doesn’t.

                              1. 37

                                I’ve been very happy with pass, a command-line tool that stores passwords and notes in a git repository. Being a directory of text files, it’s easy to use standard command-line tools on or tinker with programmatically. There’s a thriving ecosystem of plugins, tools, and clients.

                                I also use autopass for autofilling in X applications. As time goes in, I fill in more and more autotype fields to check ‘remember me’ boxes and other non-standard fields. It’s really convenient. (One annoyance is that if any password files are not valid YAML, autopass errors to stdout without opening a window, so I hit my hotkey and nothing happens.)

                                1. 11

                                  One more vote for pass, i’ve been a happy user for years now. Was missing a proper browser extension for it so I built one: Browserpass. It’s no longer maintained by me due to lack of time, but the community is doing a far better job at maintaining it than I possibly could so that’s all good!

                                  1. 10

                                    Pass looks pretty neat, but the reason I stick with KeePass(XC) is that Pass leaks metadata in the filenames - so your encryption doesn’t protect you from anyone reading the name of every site you have an account with, which is an often overlooked drawback IMO.

                                    1. 5

                                      Your filenames don’t have to be meaningful though. It would be relativity trivial to extend pass to use randomly generated names, and then use an encrypted key->value file to easily access the file you want.

                                      On the other hand, if someone already has that access to your device, accessing ~/.mozilla/firefox/... or analogous other directories with far more information is just as trivial, and has probably more informational value.

                                      1. 3

                                        Then youre working around a pretty central part of pass’s design, which I don’t really like. It should be better by default.

                                        wrt your second point, if you give up when they can read the filesystem, why even encrypt at all? IMO the idea is you should be able to put your password storage on an untrusted medium, and know that your data are safe.

                                        1. 12

                                          if you give up when they can read the filesystem, why even encrypt at all?

                                          Because in my opinion, there’s a difference between a intruder knowing that I have a “mail” password, and them actually knowing this password.

                                    2. 5

                                      The QR code feature of pass is neat for when you need to login on a phone.

                                      1. 2

                                        Huh, you made me read the man page and learn about this - it’s really cool! What’s your usage like for this though? Just use any barcode reader and then copy paste in the password box?

                                        1. 1

                                          A barcode reader I trusted, but yeah - its a good hack because I usually have my laptop which has full disk encryption.

                                          1. 2

                                            Yeah, when you said that all I could think of was the barcode scanner that I used to use where it would store the result of each barcode scanned in a history file… Not ideal :)

                                      2. 2

                                        Seems like the android version’s maintainer is giving up. (Nice, 80k lines of code in just one dep…)

                                        The temptation to nih it is growing stronger but I don’t have enough time :(

                                      1. 8

                                        KeePass has clients that work the 3 operation systems in question, and I’ve had good luck using Syncthing to share the password file between computers, but the encryption of the database means that any good sync utility can work with it.

                                        1. 4

                                          I KeePassX together with SyncThing on multiple Ubuntus and Androids for two years now. By now I have three duplicate conflict files which I keep around because I have no idea what the difference between the files is. Once I had to retrieve a password from such conflict file as it was missing in the main one.

                                          Not perfect, but works.

                                          Duclare, using ssh instead of SyncThing would certainly work since the database is just a file. I prefer SyncThing because of convenience.

                                          1. 2

                                            Duclare, using ssh instead of SyncThing would certainly work since the database is just a file.

                                            Ideally it’d be automated and integrated into the password manager though. Keepass2android does support it, but it does not support passwordless login and don’t recall it ever showing me the server’s fingerprint and asking if that’s OK. So it’s automatically logging in with a password to a host run by who knows. Terribly insecure.

                                            1. 1

                                              I had the same situation. 3 conflict files and merging is a pain. I’ve switched to Pass instead now.

                                            2. 2

                                              I use Keepass for a few years now too. I tried other Password managers in the meantime but I never got quite satisfied, not even pass though that one was just straight up annoying.

                                              I’ve had a few conflicts over the years but usually Nextcloud is rather good at avoiding conflicts here and KPXC handles it very well. I think Syncthing might casue more problems as someone else noted, since nodes might take a while to sync up.

                                            1. 4

                                              I personally find this API fairly frustrating. This call can have three different semantics, depending on the values and context in which you call it. This contributes to complexity.

                                              I notice that in the userspace diff, the “lock unveil” functionality is never used, even in cases where unveil is added to the pledge string. As far as I understand it, this means that if an attack obtained code execution, they’d simply be able to undo the unevil with unveil("/", "rwx"). That’s unintuitive and likely to be a regular source of programming errors.

                                              Grabbing some comments I made on IRC last night on how I’d persue this API:

                                              22:10:09 <Alex_Gaynor> If I was doing this API, I'd probably do `sandbox_context *sandbox_context_create(void)` and then a bunch of `sandbox_context_add_X(sandbox_context *, ...)` with appropriate signatures, and then a `sandbox_context_apply(sandbox_context *)` and basically a default `sandbox_context` had no permissions, and then you can add back whatever you want, and calling `sandbox_apply` a second time on a process killed the process or something
                                              22:10:34 <Alex_Gaynor> (Or maybe was allowed, as long as the permissions were a strict subset of what was already applied)
                                              22:12:33 <Alex_Gaynor> Oh, and they should add a platform-specific `posix_spawn_...` thing to take a `sandbox_context` so that it's applied right at `exec`, before any user code runs.
                                              22:13:23 <Alex_Gaynor> Basically the two properties I've found useful in sandboxing are: a) It should be extremely easy to see what capabilities your process has, you want them all in one place, and defaulted to "nothing" so basically the permissions are what you have written down, b) It should be extremely easy to draw a perimeter around what your process already does, and slowly wittle it down by basically deleteing "adds".
                                              
                                              1. 2

                                                I notice that in the userspace diff, the “lock unveil” functionality is never used, even in cases where unveil is added to the pledge string.

                                                I only saw two or three diffs where it isn’t clear if the unveil pledge is later revoked. All others that add unveil to pledge also have a pledge without unveil soon after the unveil calls. It’s possible the two or three cases also have it but it’s just not visible in the diff.

                                                So in these programs, that attack does not work unless you get your RCE during the initialization phase.

                                                Even if unveil was never locked, it can still protect against all-too-common path traversal style bugs (especially in web crapps) that leak data without RCE.

                                              1. 2

                                                What if I don’t want to align to the adjacent line?

                                                1. 1

                                                  Well, then you have to add a blank line in between.

                                                1. 4

                                                  Having PRIMARY and CLIPBOARD is a good thing and once you get used to it, it’s like having two clipboards.

                                                  Shame he never tells how to actually use them both. Afaict only the primary selection is usable with the default binds.

                                                  XTerm.VT100.translations: #override \n\
                                                          Ctrl Shift <Key>C: copy-selection(CLIPBOARD) \n\
                                                          Ctrl Shift <Key>V: insert-selection(CLIPBOARD)
                                                  

                                                  Now if only I could get all the other software to support them both as well.

                                                  EDIT: Another tip. If you find the font sizes available in the menu to be ridiculous, they’re pretty easy to change.

                                                  XTerm*faceSize1: 8
                                                  XTerm*faceSize2: 10
                                                  XTerm*faceSize3: 13
                                                  XTerm*faceSize4: 16
                                                  XTerm*faceSize5: 20
                                                  XTerm*faceSize6: 26
                                                  

                                                  faceSize1 corresponds to “Unreadable.”

                                                  Now would someone give me key binds to decrease/increase font size? :-)

                                                  1. 6

                                                    You might want to read X Selections, Cut Buffers, and Kill Rings for how to use the PRIMARY and CLIPBOARD selections in X Windows.

                                                    1. 1

                                                      It doesn’t, and can’t really explain how to use them because there is no way to use them in X. Instead, you have to use them in applications running under X and each application does its own thing. I still don’t know if there’s a way to copy to clipboard in xterm without creating a custom bind.

                                                      1. 0

                                                        Does it also work with X?

                                                        1. 1

                                                          If by “X” you mean “the graphical interface that runs on Linux” then yes, it works, because that is X Windows.

                                                          1. -1

                                                            Eh, the developers would disagree, but what do they know?

                                                            1. 1

                                                              Where did this X Windows meme even start?

                                                              Some lamer back in 1995 thinking it sounded cool and having it go viral on Usenet?

                                                              1. 2

                                                                Where did this X Windows meme even start?

                                                                I don’t know. Probably people who think it’s the X-TREME version of Microsoft Windows.

                                                                1. 1

                                                                  It’s mentioned in The Unix-Haters Handbook as a reliable tool for getting Unix weenies angry.

                                                                  1. 1

                                                                    I’m pretty sure “X Windows” is much older than that (as is MS Windows). I vaguely recall reading about “X Windows” in Byte magazine in 1993 or so.

                                                                    The comp.windows.x newsgroup goes back to at least 1987 (https://groups.google.com/forum/message/raw?msg=comp.windows.x/TtNRIfTKqsw/i7hzWBiDfkgJ), a month after X11 was created. They even refer to it as “x-windows”.

                                                                    1. 1

                                                                      Could it have been a different implementation? Cuz I remember doing the RTFM thing way back when, and it was very clear about not being “X Windows”, though didn’t specify why.

                                                                      Sorry if this is explained in the link. Can’t be arsed with Google. Usenet used to come without opt-in spying.

                                                                  2. 1

                                                                    Well, excuse me for using outdated terminology then. Would if have been better had I said “You might want to read X Selections, Cut Buffer, and Kill Rings for how to use the PRIMARY and CLIPBOARD selections in X”?

                                                                    1. 1

                                                                      Not outdated, just incorrect.

                                                            2. 3

                                                              Now would someone give me key binds to decrease/increase font size? :-)

                                                              i have the following

                                                              *VT100*translations: #override \
                                                                  Meta <Key> minus: smaller-vt-font() \n\
                                                                  Meta <Key> plus: larger-vt-font() \n\
                                                                  Super <Key> minus: smaller-vt-font() \n\
                                                                  Super <Key> plus: larger-vt-font() \n\
                                                              

                                                              and either meta/super keys work as expected.

                                                            1. 5

                                                              I’m playing around with libtls (per advice. I’ve already proved to myself that it can be used in an event based server, and now I’m playing around with trying to get it integrated into our network flow, which in this case means writing a Lua wrapper for it. [1]

                                                              [1] There are two Lua modules for libtls that I’ve found, but neither one meets my criteria, namely, using the call back mechanism to control the network. The changes are extensive enough that I find it easier to write my own version.

                                                              1. 1

                                                                Please write about your experience with libtls once you know how it pans out :)

                                                              1. 9

                                                                Many of the author’s experiences speaking with senior government match my own.

                                                                However, there’s one element that I think is very easily lost in this conversation, and which I want to highlight: there is no group I spend more time trying to convince of the importance of security than other software engineers.

                                                                Software engineers are the only group of people I’ve ever had push back when I say we desperately need to move to memory safe programming languages. All manner of non-engineers, when I’ve explained the damages wrought by C/C++, and how nearly every mass-vulnerability they know about has a shared root cause, generally understand why this is an important problem, and want to discuss ideas about how do we resolve this.

                                                                Engineers complain to me that rewriting things is hard, and besides if you’re disciplined in writing C and use sanitizers and fuzzers you’ll be ok. Rust isn’t ergonomic enough, and we’ve got a really good hiring pipeline for C++ engineers.

                                                                If we want to build software safety into everything we do, we need to get engineers on board, because they’re the obstacle.

                                                                1. 11

                                                                  People don’t even use sanitizers and fuzzers, so I’m not sure why you would expect them to rewrite in Rust. It’s literally 1000x less effort.

                                                                  As far as I can tell, CloudFlare’s CloudBleed bug would have been found if they compiled with ASAN and fed about 100 HTML pages into it. You don’t even have to install anything; it’s built right into your compiler! (both gcc and Clang)

                                                                  I also don’t agree that “nearly every mass vulnerability has a shared root cause”. For example, you could have written ShellShock in Rust, Python, or any other language. It’s basically a “self shell-code injection” and has very little to do with memory safety (despite a number of people being confused by this.)

                                                                  The core problem is the sheer complexity and number of lines of unaudited code, and the fact that core software like bash has exactly one maintainer. There are actually too many people trying to learn Rust and too few people maintaining software that everybody actually uses.

                                                                  In some sense, Rust can make things worse, because it leads to more source code. We already have memory-safe languages: Python, Ruby, JavaScript, Java, C#, Erlang, Clojure, OCaml, etc.

                                                                  Software engineers should definitely spend more time on security, and need to be educated more. But the jump to Rust is a non-sequitur. Rust is great for kernels where the above languages don’t work, and where C and C++ are too unsafe. But kernels are only a part of the software landscape, and they don’t contain the majority of security bugs.

                                                                  I would guess that most data breaches these days have nothing to do with memory safety, and have more to do with bugs similar to the ones in the OWASP top 10 (e.g. XSS, etc.)

                                                                  https://www.owasp.org/images/7/72/OWASP_Top_10-2017_%28en%29.pdf.pdf


                                                                  Edit: as another example, Mirai has nothing to do with memory safety:

                                                                  https://en.wikipedia.org/wiki/Mirai_(malware)

                                                                  All it does it try default passwords, which gives you some idea of where the “bar” is. Rewriting software in Rust has nothing to do with that, and will actually hurt because it takes effort and mindshare away from solutions with a better cost/benefit ratio. And don’t get me wrong, I think Rust has its uses. I just see people overstating them quite frequently, with the “why don’t more people get Rust?” type of attitude.

                                                                  1. 2

                                                                    There were languages like Opa that tried to address what happened on web app side. They got ignored just like people ignore safety in C. Apathy is the greatest enemy of security. It’s another reason we’re pushing the memory-safe, higher-level languages, though, with libraries for stuff likely to be security-critical. The apathetic programmers do less damage on average that way. Things that were code injections become denial of service. That’s an improvement.

                                                                  2. 2

                                                                    not only software engineers, almost the entire IT industry has buried it’s head in the sand and is trying desperately hard to hide from the problem, because “security is too hard”. We are pulling teeth to get people to even do the minimal upgrades to things. I recently had a software vendor refusing to support anything other than TLS 1.0. After many exchanges back and forth, including an article from Microsoft(and basically every other sane person) saying they were dropping all support of older TLS protocols because of their insecurity, they finally said, OK we will look into it. I’m sure we all have stories like this.

                                                                    If you can’t even bother to take the minimum of steps to upgrade your security stacks after more than a decade,(TLS1.0 released in 1999 and TLS 1.2 is almost exactly a decade old now) because it’s “too hard”, trying to get people to move off of memory unsafe languages like C/C++ is a non-starter.

                                                                    But I agree with you, and the author.

                                                                    1. 2

                                                                      I would like to use TLS 1.3 for an existing product. It’s in C and Lua. The current system is network driven using select() (or poll() or epoll() depending upon the platform). The trouble I’m having is finding a library that is easy, or even a bit complicated but sane to use. The evented nature means I an notified when data comes in, and I want to feed this to the TLS library instead of having the TLS library manage the sockets for me. But the documentation is dense, the tutorials only cover blocking calls, and that’s when they’re readable! Couple this with the whole “don’t you even #$@#$# think of implementing crypto” that is screamed from the roof tops and no wonder software engineers steer away from this crap.

                                                                      I want a crypto library that just handles the crypto stuff. Don’t do the network, I already have a framework for that. I just need a way to feed data into it, and get data out of it, and tell me if the certificate is good or not. That’s all I’m looking for.

                                                                      1. 2

                                                                        OpenBSD’s libtls.

                                                                        1. 2

                                                                          TLS 1.3 is not quite ready for production use, unless you are an early adopter like Cloudfare. Easy to use API’s that are well-reviewed are not there yet.

                                                                          Crypto Libraries: OpenBSD’s libtls like @kristapsdz mentioned, or libsodium/nacl or OpenSSL. If it’s just for your internal connections and don’t actually need TLS, just talking to libsodium or NaCL for an encrypted stream of bytes is probably your best bet, using XSalsa20+Poly1305. See: https://latacora.singles/2018/04/03/cryptographic-right-answers.html

                                                                          TLS is a complicated protocol(TLS1.3 reduces a LOT of complexity, it’s still very complicated).

                                                                          If you are deploying to Apple, Microsoft or OpenBSD platforms, you should just tie to the OS provided services, that provide TLS. Let them handle all of that for you(including the socket). Apple and MS platforms have high-level API’s that will do all the security crap for you. OpenBSD has libtls.

                                                                          On other platforms(Linux, etc), you should probably just use OpenSSL. Yes it’s a fairly gross API, but it’s pretty well-maintained nowadays(5 years ago, it would not qualify as well maintained.). The other option is libsodium/NaCL.

                                                                          1. 1

                                                                            Okay, fine. Are there any crypto libraries that are easy to use for whatever is current today? My problem is: a company that is providing us information today via DNS has been invaded by a bunch of hipster developers [1] who drunk the REST Kool-Aid™ so I need a way to make an HTTPS call in an event driven architecture and not blow our Super Scary SLAs with the Monopolistic Phone Company (which would case the all-important money to flow the other way), so your advice to let OS provided TLS services control the socket is a non-starter.

                                                                            And for the record, the stuff I write is deployed to Solaris. For reasons that exceed my pay grade.

                                                                            So I read the Cryptographic Right Answers you linked to and … okay. That didn’t help me in the slightest.

                                                                            The program I’m working on is in C, and not written by me (so it’s in “maintenance mode”). It works, and rewriting it from scratch is probably also a non-starter.

                                                                            Are you getting a sense of the uphill battle this is?

                                                                            [1] Forgive my snarky demeanor. I am not happy about this.

                                                                            Edit: further clarification on what I have to work with.

                                                                            1. 1

                                                                              I get it, it sucks sometimes. I’m guessing you are not currently doing any TLS at all? So you can’t just upgrade the libraries you are currently using for TLS, whatever they are.

                                                                              In my vendor example, the vendor already implemented TLS (1.0) and then promptly stopped. They have never bothered to upgrade to newer versions of TLS. I don’t know the details of their implementation, obviously, since it’s closed-source; but unless they went crazy and wrote their own crypto code, upgrading their crypto libraries is probably all that’s required. I’m not saying it’s necessarily easy to do that, but this is something everyone should do at least once every decade, just to keep the code from rotting a terrible death anyways. TLS 1.2 becomes a decade old standard next month.

                                                                              I don’t work on Solaris platforms (and haven’t in at least a decade, so you are probably better off checking with other Solaris people). Oracle might have a TLS library these days, I have no clue. I tend to avoid Oracle land whenever possible. I’m sorry you have to play in their sandbox.

                                                                              I agree the Crypto right-answers page isn’t useful for you, since you just want TLS, It’s target is for developers who need more than TLS. I used it here mostly as proof of why I recommended XSalsa20+Poly1305 for symmetric encryption. Again, you know you need TLS, so it’s a non-useful document for you at this point.

                                                                              Event driven IO is possible with OpenSSL, but it’s not super easy see: https://www.openssl.org/docs/faq.html#PROG11. Then again, nothing around event driven IO is super easy. Haproxy and Nginx both manage to do it, and are both open-source implementations of TLS, so you have working code you can go examine. Plus it might give you access to developers who have done event driven IO with TLS. I haven’t ever written that implementation, so I can’t help with those specifics.

                                                                              OpenSSL is working on making their API’s easier to use, but it’s a long, slow haul, but it’s definitely a known problem, and they are working on it.

                                                                              As for letting the OS do the work for you, you are correct there are definitely use-cases where it won’t work, and it seems you fit the bill. For most applications, letting the OS do it for you is generally the best answer, especially around Crypto which can be hard to get right, and of course only applies to the platforms that offer such things(Apple, MS, etc). Which is why I started there ;)

                                                                              Anyways, good luck! Sorry I can’t just point to a nice easy example, for you. Maybe someone else around here can.

                                                                              1. 1

                                                                                I’m not even using TCP! This is all driven with UDP. TCP complicates things but is manageable. Adding a crap API between TCP and my application? Yeah, I can see why no one is lining up to secure their code.

                                                                                1. 1

                                                                                  I think there is a communication issue here.

                                                                                  The vendor you are connecting with over HTTPS supports UDP packets on a REST API interface? really? Crazier things have happened I guess.

                                                                                  I think what you are saying is you are doing DNS over UDP for now, but are being forced into HTTPS over TCP?

                                                                                  DNS over UDP is very far away from a HTTPS rest API.

                                                                                  Anyways, for being an HTTPS client, against a HTTPS REST API over TCP, you have 2 decent options:

                                                                                  Event driven/async: use libevent, example code: https://github.com/libevent/libevent/blob/master/sample/https-client.c

                                                                                  But most people will be boring, and use something like libcurl (https://curl.haxx.se/docs/features.html) and do blocking I/O. If they have enough network load, they will setup a pool of workers.

                                                                                  1. 2

                                                                                    Right now, we’re looking up NAPTR records over DNS (RFC-3401 to RFC-3404). The summary is that one can query name information for a given phone number (so 561-555-5678 is ACME Corp.). The vendor wants to switch to a REST API and return JSON. Normally I would roll my eyes at this but the context I’m working in is more realtime—as in Alice is calling Bob and we need to look up the information as the call is being placed! WE have a hard deadline with the Monopolistic Phone Company to provide this information [1].

                                                                                    We don’t use libevent but I’ll look at the code anyway and try to make heads and tails.

                                                                                    [1] Why are we querying a vendor this for? Well, it used to be in house, but now “we lease this back from the company we sold it to - that way it comes under the monthly current budget and not the capital account.” (at least, that’s my rational for it).

                                                                                    1. 2

                                                                                      Tell me how it goes. Fwiw, you might want to take a quick look at mbed TLS. Sure it wants to wrap a socket fd in its own context and use read/write on it, but you can still poll that fd and then just call the relevant mbedtls function when you have data coming in. It does also support non-blocking operation.

                                                                                      https://tls.mbed.org/api/net__sockets_8h.html#a2ee4acdc24ef78c9acf5068a423b8c30 https://tls.mbed.org/api/net__sockets_8h.html#a03af351ec420bbeb5e91357abcfb3663

                                                                                      https://tls.mbed.org/api/structmbedtls__net__context.html

                                                                                      https://tls.mbed.org/kb/how-to/mbedtls-tutorial (non-blocking io not covered in the tutorial but it doesn’t change things much)

                                                                                      I’ve no experience with UDP (yet – soon I should), but if you’re doing that, well, mbedtls should handle DTLS too: https://tls.mbed.org/kb/how-to/dtls-tutorial (There’s even a note relevant to event based i/o)

                                                                                      We use mbedtls at work in a heavily event based system with libev. Sorry, no war stories yet, I only got the job a few weeks ago.

                                                                                      1. 1

                                                                                        Right, let’s add MORE latency for a real-time-ish system. Always a great idea! :)

                                                                      1. 15

                                                                        Seemed too good to be true.

                                                                        I tried infer on our standard build environment but it depends on newer libs than you get on rhel/centos7. So I moved on to the docker image but it ran out of memory while building infer. So I moved on to using the prebuilt binaries in a custom docker image running ubuntu, and after getting the deps right, I can finally run infer on a trivial one-line C program. Unfortunately the thing segfaults if I try to use it on real code.

                                                                        1. 5

                                                                          The most glaring omission on the post is Infer from Facebook. I woud rate Infer as the most impressive open source C/C++ static analyzer, by far.

                                                                          1. 3

                                                                            ugh, I’ve been trying to package it for arch and it’s such a pain in the ass. It uses a bunch of ocaml libraries that didn’t previously have packages and it bundles a custom version of clang with its own modifications and extensions. Oh, and due to requiring a custom clang, builds can be over half an hour before anything goes wrong.

                                                                            1. 2

                                                                              Whoa, if that thing does what it says on the tin, I’m super interested.

                                                                              I hope it does.

                                                                              Cppcheck did not.

                                                                              EDIT: A nasty nest of segfaults is all I can get out of it. Maybe I’ll check back next year.

                                                                            1. 20

                                                                              Kinesis Advantage. I’ve been using them for almost twenty years, and other than some basic remapping, I don’t customize.

                                                                              1. 2

                                                                                Ditto, I’m at a solid decade. I cannot recommend them enough.

                                                                                1. 2

                                                                                  Also Kinesis Advantage for over a decade. On the hardware side I’ve only mapped ESC to where Caps Lock would be. On the OS side I’ve got a customized version of US Dvorak with scandinavian alphabet.

                                                                                  I’d like to try a maltron 3d keyboard with integrated trackball mouse. It’s got better function keys too, and a numpad in the middle where there’s nothing except leds on the kinesis.

                                                                                  1. 2

                                                                                    Me too. I remap a few keys like the largely useless caps-lock and otherwise I don’t program it at all. It made my wrist pain disappear within a couple weeks of usage though.

                                                                                    1. 2

                                                                                      My only “problem” with the Kinesis, and it’s not even my problem, was that the office complained about the volume of the kicks while I was on a call taking notes.

                                                                                      So I switch between the Kinesis and a Apple or Logitech BT keyboard for those occasions.

                                                                                      1. 1

                                                                                        You can turn the clicks off! I think the combo is Prgm-\

                                                                                        1. 2

                                                                                          Yeah, its not that click, it’ the other one from the switches :-)

                                                                                          I can be a heavy typer and for whatever reason, these keys stand out more than I expected to others behind the microphone.

                                                                                      2. 2

                                                                                        I prefer the kinesis freestyle2. I like the ability to move the two halves farther apart (broad shoulders) and the tilt has done wonders for my RSI issues.

                                                                                        1. 2

                                                                                          similar, largely I like that I can put the magic trackpad in between the two halves and have something that feels comparable to using the laptop keyboard. I got rid of my mouse years ago but I’m fairly biased on a trackpad’s potential.

                                                                                          I’ve sometimes thought about buying a microsoft folding keyboard and cutting/rewiring it to serve as a portable setup. Have also thought of making a modified version of the nyquist keyboard to be a bit less ‘minimal’ - https://twitter.com/vivekgani/status/939823701804982273

                                                                                      1. 12

                                                                                        Anybody with a brain knows that open plan offices are just plain bad in just about every way - except one - they’re dirt freaking cheap, which is why they’re everywhere, and that’s unlikely to change anytime soon.

                                                                                        I’ve worked in this business long enough to remember when, even as a lowly sysadmin, I had either my own office or a shared office with one other person.

                                                                                        Those were the days :)

                                                                                        1. 1

                                                                                          I’ve worked in this business long enough to remember when, even as a lowly sysadmin, I had either my own office or a shared office with one other person.

                                                                                          I’ve worked in this business for three weeks and I’m hacking away alone in what used to be the boss’ office.

                                                                                          The plan is to set up my very own remote office too.

                                                                                        1. 21

                                                                                          Stylus is using the same theme database without collecting your history:

                                                                                          1. 7

                                                                                            +1

                                                                                            But the problem is: how to ensure that Stylus (or any alternative) won’t become the next “Stylish”?

                                                                                            1. 7

                                                                                              I’ve written a couple of my own extensions, partly for this reason. For certain complicated or common needs (like ad-blocking) I have no choice but to find an extension I trust and use it. But in other cases I just end up writing my own because I can’t find something that doesn’t feel sketchy.

                                                                                              Ironically, one of my extensions was recently removed from the Firefox store because there was some incidental code in a dependency (that isn’t used at runtime) that makes a network request.

                                                                                              1. 1

                                                                                                I’ve written a couple of my own extensions, partly for this reason.

                                                                                                This is the “hacker’s approach” that I prefer.
                                                                                                Everyone should be able to hack software for his own need.

                                                                                                For certain complicated or common needs (like ad-blocking) I have no choice but to find an extension I trust and use it.

                                                                                                Well, actually you can also review them, if the sources are available.

                                                                                                1. 6

                                                                                                  Well, actually you can also review them, if the sources are available.

                                                                                                  Certainly an important part of the process, but both major browsers push updates to extensions silently, and there’s no guarantee that the code my browser runs is the same code that was in the OSS repository. It’s a crap situation all-around, really.

                                                                                                  1. 4

                                                                                                    This is the “hacker’s approach” that I prefer.

                                                                                                    I prefer it too, but as far as I can tell webextensions goes out of its way to make this tedious and annoying.

                                                                                                    I’ve tried building webextensions from source, and as far as I can tell there is no way to permanently install them. You can only install them for a single session at a time. (Hopefully there’s a workaround someone can suggest, but I didn’t find one at the time.) It was pretty appalling from a hackability/software-freedom perspective, so I was pretty surprised to see it coming from Mozilla.

                                                                                                    1. 2

                                                                                                      Idk about mozilla, but I made my own permanently installed extension for an appliance with chromium. Precisely to avoid the risk of updates or unavailability due to internet outages.

                                                                                                2. 4

                                                                                                  Consumers should demand that extensions don’t improperly use personal info, and that the browser vendors only allow extensions that adhere to these rules.

                                                                                                  1. 17

                                                                                                    Consumers should demand that extensions don’t improperly use personal info

                                                                                                    Do you know any consumer that want extensions to sell their personal info?
                                                                                                    I mean, it’s like relying on consumers’ demand for pencils that do not explode.

                                                                                                    Yes, they might ask for it… if only they knew they should!
                                                                                                    (I’m not just sarcastic: perfect symmetric information is the theoretical assumption of free market efficiency)

                                                                                                    1. 2

                                                                                                      I was being half sarcastic. Marketing is basically information arbitrage, after all.

                                                                                                      But as a practical matter I believe voluntary regulation is the way forward for this. Laws are struggling to catch up, although it would be interesting to see how GDPR applies here.

                                                                                                      1. 5

                                                                                                        I believe voluntary regulation is the way forward for this.

                                                                                                        Gentlemen agreements work in a world of gentlemen.
                                                                                                        In a world wide market cheating is too easy. It’s too easy to hide.

                                                                                                        GDPR reception shows how much we can trust companies “voluntary regulations”.

                                                                                                        Laws are struggling to catch up

                                                                                                        True. This is basically because many politics rely on corporate “experts” to supply for their ignorance.

                                                                                                    2. 3

                                                                                                      In theory the permissions system should govern this. For example, I can imagine a themeing extension needing permission to access page content; but it should be easy to make it work without any external communication, e.g. no network access, read-only access to its own data directory (themes could be separate extensions, and rely on the extension manager to copy them into place), etc.

                                                                                                      1. 2

                                                                                                        It can leak data to its server by modifying just css, not even touching DOM, by adding background images for example. I don’t know if it’s even possible to design browser extensions system so extension effects are decently isolated.

                                                                                                        However, these exfiltration hacks might attract attention easier than plain XHR.

                                                                                                        1. 1

                                                                                                          Hmm, yes. I was mistakenly thinking of a theme as akin to rendering given HTML to a bitmap; when in fact it’s more like a preprocessor whose result is sent to the browser engine. With no way of distinguishing between original page content and extension-provided markup, you’re right that it’s easy to exfiltrate data.

                                                                                                          I can think of ways around this (e.g. setting a dirty bit on anything coming from the theme, or extending cross domain policies somehow, etc.) but it does seem like I was being a bit naive about how hard it would be.

                                                                                                    3. 2

                                                                                                      Theoretically, you could audit the GitHub repo (https://github.com/openstyles/stylus) and build it yourself. Unfortunately that doesn’t seem too feasable.

                                                                                                      1. 1

                                                                                                        For this reason I install the absolute minimum extensions. I usually only have privacy badger installed as I’m fairly sure the EFF won’t sell out.

                                                                                                    1. 1

                                                                                                      On the EINTR/retry issue, it’s not clear to me why a simple ‘return to userspace and retry the instruction’ is harder to implement.

                                                                                                      There is kernel-side state associated with the fd (seek position, network buffers etc) but these need to be maintained anyway for an EINTR return.

                                                                                                      Put another way - what work can the kernel avoid in the ‘return EINTR and let the userspace application call the system call again’ scenario which is required in the ‘return to userspace at PC-1’?

                                                                                                      1. 5

                                                                                                        There are two big issues: changing entry conditions, and blocking after interrupts.

                                                                                                        Blocking after interrupts is fairly obvious when you think about it. If you are blocking on some syscall in your main thread, receive a signal, and your syscall is restarted automatically, you cannot respond to that signal in your main thread at all. You’d just keep blocking. If the signal was SIGINT and you would want to cleanly shut down, you can’t. You’d be stuck until your syscall unblocks, which could never happen.

                                                                                                        Changing entry conditions are much more tricky. For example, some syscalls have timeouts—the amount of time they should block before returning regardless of their success or failure. If you start a syscall with a 10 second timeout, what happens if that syscall is cancelled and restarted 5 seconds in? If the same arguments are used, it would be as if the timeout had been 15 seconds. If you receive 1 signal per second indefinitely, the syscall will block forever. Unlikely but possible.

                                                                                                        When you call something with a 10 second timeout, you actually mean 10 seconds from now. To restart these syscalls, the kernel would need to preserve that start time entry condition. That’s fairly doable, but there are other entry conditions that aren’t doable at all. If you’re writing size-formatted output to the console, and you receive SIGWINCH, you don’t want to proceed with the write. Instead you need to reformat the output to match the new console dimensions. The kernel certainly can’t do that for you.

                                                                                                        There are so many reasons you might want to change syscall parameters after an interrupt, or do something else entirely. The kernel can’t know all of them. And designing an interface to conveniently accommodate all of them is a lot harder. Thus the worse-is-better solution: never assume what a program wants to do after an interrupt.

                                                                                                        1. 2

                                                                                                          Signals, I think, is now fairly well accepted as one of the “ugly” parts of posix.

                                                                                                          signalfd was an attempt at fixing it…. Here is more discussion of that..

                                                                                                          https://ldpreload.com/blog/signalfd-is-useless

                                                                                                          Having spend the last week battling the fine fine fine corner cases of signal handling….

                                                                                                          Sigh.

                                                                                                          I wish linux had something better.

                                                                                                          1. 1

                                                                                                            Thanks for this.

                                                                                                            If you are blocking on some syscall in your main thread, receive a signal, and your syscall is restarted automatically, you cannot respond to that signal in your main thread at all. You’d just keep blocking.

                                                                                                            I’m still not getting this one. I’d envisage:

                                                                                                            • application calls blocking read()
                                                                                                            • application receives SIGHUP
                                                                                                            • kernel sees incoming signal and stops doing read()
                                                                                                            • kernel calls up into user spaces to run signal handler in user context for SIGHUP. As far as application goes, it’s still doing read(), but signals can happen any time anyway, so no problem here?
                                                                                                            • kernel restarts read()

                                                                                                            If you’re writing size-formatted output to the console, and you receive SIGWINCH, you don’t want to proceed with the write.

                                                                                                            I’m not sure I agree. Given that arriving signals are inherently racy, I think it could be considered to also be correct to re-run the system call without the application making a new choice based on the new information. (The system call could easily have completed before the signal arrived - and the application should be prepared for that eventuality).

                                                                                                            When you call something with a 10 second timeout, you actually mean 10 seconds from now. To restart these syscalls, the kernel would need to preserve that start time entry condition.

                                                                                                            This is a good point. However, the optimist in me would like to think this is always solvable with API design. (In the timeout case, this would involve absolute timeout rather than relative).

                                                                                                            1. 2

                                                                                                              Your scenario doesn’t work, because signal handlers are extremely restricted. Signal handlers must be reentrant with respect to all other signal handlers, meaning they can’t allocate memory, can’t use most syscalls, can’t use any libc functions that set errno, and can’t non-atomically modify non-local variables.

                                                                                                              For this reason, signal handlers usually set a global volatile flag and return immediately, allowing the main thread to catch EINTR, check the flag, and handle the signal without restriction.

                                                                                                              In the SIGWINCH example, doing what you suggest causes significant visual tearing, even though it’s technically correct. But that assumption about racy signals only works if all syscalls are guaranteed to complete.

                                                                                                              However, the optimist in me would like to think this is always solvable with API design.

                                                                                                              Perhaps. Until such an API actually exists, we must write interruptible code somehow.

                                                                                                              (In the timeout case, this would involve absolute timeout rather than relative).

                                                                                                              What absolute time? The system clock can change. The clock that can’t is the monotonic raw clock, which simply ticks upwards from an unspecified time. I don’t think an API that takes a deadline with respect to a meaningless clock beats an API that requires restarting on interrupt.

                                                                                                              1. 1

                                                                                                                Your scenario doesn’t work, because signal handlers are extremely restricted.

                                                                                                                Yes they are, but I don’t think that stops the kernel from invoking them and then restarting the system call afterwards.

                                                                                                                In the SIGWINCH example, doing what you suggest causes significant visual tearing

                                                                                                                Which could already occur. We’re just (slightly) increasing the window in which delivery of the signal will cause it.

                                                                                                                What absolute time?

                                                                                                                The one with the same semantics as a relative timeout - i.e. monotonic. That is the behaviour you would get if you specified “10 seconds from now”.

                                                                                                                1. 2

                                                                                                                  Your scenario doesn’t work, because signal handlers are extremely restricted.

                                                                                                                  Yes they are, but I don’t think that stops the kernel from invoking them and then restarting the system call afterwards.

                                                                                                                  What I think you’re missing here is that since you can’t do much in a signal handler besides setting a flag for the main thread to check, you must have a way to interrupt/cancel/unblock whatever system call is blocking the main thread so that the main thread can start doing whatever the signal called for. If the kernel automatically restarts the system call, the main thread obviously can’t go check that flag and react accordingly.

                                                                                                                  1. 2

                                                                                                                    Yes, you’re right, thank you. But in that case, the interrupted return is the feature, not the bug?

                                                                                                                    The whole ‘worse is better’ thing is cast as “return EINTR” being an undesirable property.

                                                                                                                    1. 2

                                                                                                                      Yes indeed, it is a feature. But it is a feature that complicates life for every call that doesn’t care and doesn’t want to be interrupted.

                                                                                                                      1. 2

                                                                                                                        OK, but it isn’t anything to do with Worse is Better, right? The article characterises it as a deficiency of implementation.

                                                                                                                        In fact, it’s a feature you need if you want to abort a blocking operation.

                                                                                                                        afaics, the only problem is the default. You probably want the SA_RESTART behaviour by default (and perhaps that should be per-fd, rather than per-signal).

                                                                                                                        But I don’t think the characterisation in the article is fair, unless I’ve missed something.

                                                                                                                        1. 1

                                                                                                                          Worse is better means choosing a design that’s worse in some cases because at least it works for all cases. EINTR absolutely embodies that description. It’s a dead simple way to make sure you can always handle interrupts, but extremely tedious to work with in the common case.

                                                                                                                          1. 2

                                                                                                                            That was my understanding of the pt made in the article and the received wisdom regarding this article.

                                                                                                                            However, I thought the conclusion of the discussion above was that it wasn’t any easier than not handling the interrupt. In fact, the interrupted behaviour is required (it is a feature) in many cases.

                                                                                                                            I still fail to see how it is easier for the kernel to return EINTR rather than restart the syscall. (Apart from the API issue mentioned regarding entry conditions, e.g. relative/absolute times).

                                                                                                                            It might help if someone can outline the desired behaviour. It isn’t “restart syscall”, since that has been disposed of as undesirable above.

                                                                                                                            I think my point is: “this article says the Unix EINTR approach is a short cut in kernel implementation which has imposed a cost on userspace since then”. However:

                                                                                                                            a) I can’t see the shortcut being taken (no one has pointed out how it is easier to restart a read() than return EINTR) b) arguments above are for EINTR being a good thing

                                                                                                                            Is there a 3rd way (other than ‘abort read() early and return EINTR’ or ‘restart read()’) which I’m missing here?

                                                                                                                            1. 2

                                                                                                                              I’m totally out of my element here, but did not EINTR evolve over years? Like, it started really cheap and not useful, but it became better (and more complex) over time? Is this what the author referred to when they said that unix/c improved from 50% to 90%?

                                                                                                            2. 0

                                                                                                              Thus the worse-is-better solution: never assume what a program wants to do after an interrupt.

                                                                                                              One might argue that interrupts are a broken IPC system. Errno definitely is broken.

                                                                                                              However syscalls can fail (as any other computation).

                                                                                                          1. 3

                                                                                                            Can’t say I like arch that much. The wiki is indeed a damn good resource. But using the OS involves too much config wank and “what broke now” after updating a pile of packages you didn’t want to update but had to because you needed to install one new package.. I’d also prefer not to have to hunt down all the packages I need to get a usable system up and running. Also I never liked hunting down aur pkgbuilds. Why not just a repo?

                                                                                                            Pacman also never made that much sense to me. Why “sync”? Why do I “sync” to search? And why do I also “sync” to install? apt install, dnf install, and pkg_add in particular make sense. pacman -Sy and -Ss and -S, not so much.

                                                                                                            I use it and I hate it.

                                                                                                            1. 2

                                                                                                              Yeah, I’m experiencing a similar thing. Arch works the best of the distros I’ve tried for what I’m doing (running a VM server with video passthrough) and I use it for daily development aside from that, but I would really like something more declarative like Nix. I don’t feel like I can stand using rolling for all my packages anymore.

                                                                                                            1. 2

                                                                                                              … and I guess this is why software is so terrible.

                                                                                                              1. 7

                                                                                                                Funny thing is, though – software that is not subject to the commercial incentives is still terrible.

                                                                                                                1. 1

                                                                                                                  How about all the free / open source infrastructure (from kernels thru daemons thru libs thru languages & compilers) the commercial sector builds their projects upon? Sure, these too are supported by corps with commercial incentives. But they’re not the “fast and cheap” apps that make it or don’t.

                                                                                                                  I think we get pretty good software from people who do it for the love of doing it and are paid to keep working on it. Maybe it’s not perfect, but there’s a fair amount of software that doesn’t make me hate it all.

                                                                                                                  1. 5

                                                                                                                    I’m with jfb. Most OSS software has poor UI, poor documentation, poor security, and so on. It’s crap. Even if better than proprietary average, that wouldn’t be saying much since so much of it is crap. Software being crap is the default. Another phrasing is Big Ball of Mud.

                                                                                                                    1. 5

                                                                                                                      Part of the problem is that software quality is an aesthetic judgement, a multi-dimensional one, and so people’s views of what makes software “good” are necessarily going to vary.

                                                                                                                      1. 2

                                                                                                                        Well, maybe, maybe not. It’s definitely subjective in terms of what calls what people will make on it. There are objective attributes we can go by, though. Traditionally, those included things like size of modules, coupling, amount of interactions, what features were tested, how often they fail, how severe failures are, ease of making a change, ease of training new users if UX, and so on. I think the specific values considered acceptable will vary considerably project to project for good reasons. We often assess the same stuff in each one, though. Suggests some things are more objective than others.

                                                                                                                        1. 3

                                                                                                                          I largely agree with this, yes.

                                                                                                                    2. 1

                                                                                                                      I think they’re almost uniformly terrible, too.

                                                                                                                      1. 1

                                                                                                                        Do you have any examples of software you like? I see your POV a bit and was wondering if you had anything you liked.

                                                                                                                        1. 3

                                                                                                                          I liked the original Interface Builder a lot. I was also a fan of the classic Mac OS for a while. I enjoy Squeak. djb’s software is uniformly good. I use Emacs and I love it but that love is tempered by a strong dislike for emacs lisp itself. But as an environment, I couldn’t possibly surrender it.

                                                                                                                          I admire a lot more software than I like – OpenBSD, for instance.

                                                                                                                          ETA: Postgres, of course, is a very good piece of software.

                                                                                                                          ETA’: I really, really like http://reederapp.com, an iOS/OS X RSS reader.

                                                                                                                1. 2

                                                                                                                  Personally I think these small language are much more exciting than big oil tankers like Rust or Swift.

                                                                                                                  I’m not familiar with either of those languages, but any idea what the author means by this? I thought Rust has been picking up quite a bit recently.

                                                                                                                  1. 10

                                                                                                                    I understood the author to be talking about the “size” of the language, not the degree of adoption.

                                                                                                                    I’m not sure that I personally agree that C is a small language, but many do belive that.

                                                                                                                    1. 3

                                                                                                                      Your involvement with rust will bias your opinion - rust team hat would be appropriate here :)

                                                                                                                      1. 11

                                                                                                                        He is right though. C’s execution model may be conceptually simple but you may need to sweat the implementation details of it, depending on what you’re doing. This doesn’t make C bad, it just raises the bar.

                                                                                                                        1. 9

                                                                                                                          I had that opinion before Rust, and I’m certainly not speaking on behalf of the Rust team, so in my understanding, the hat is very inappropriate.

                                                                                                                          (I’m also not making any claims about Rust’s size, in absolute terms nor relative to C)

                                                                                                                          1. 4

                                                                                                                            Or you can just test his claim with numbers. A full, C semantics is huge compared to something like Oberon whose grammar fits on a page or two. Forth is simpler, too. Whereas, Ada and Rust are complicated as can be.

                                                                                                                            1. 5

                                                                                                                              I agree that there are languages considerably smaller than C. In my view, there is a small and simple core to C that is unfortunately complicated by some gnarly details and feature creep. I’ve expressed a desire for a “better C” that does all we want from C without all the crap, and I sincerely believe we could make such a thing by taking C, stripping stuff and fixing some unfortunate design choices. The result should be the small and simple core I see in C.

                                                                                                                              When comparing the complexity of languages, I prefer to ignore syntax (focusing on that is kinda like bickering about style; yeah I have my own style too, and I generally prefer simpler syntax). I also prefer to ignore the standard library. What I would focus on is the language semantics as well as the burden they place on implementation. I would also weigh languages against the features they provide; otherwise we’re talking apples vs oranges where one language simply makes one thing impossible or you have to “invent” that thing outside the language spec. It may look simpler to only present a floating 64-bit point numeric type, but that only increases complexity when people actually need to deal with 64-bit integers and hardware registers.

                                                                                                                              That brings us to Oberon. Yes, the spec is short. I guess that’s mostly not because it has simple semantics, but because it lacks semantics. What is the range of integer types? Are they bignums, and if so, what happens you run out of memory trying to perform multiplication? Perhaps they have a fixed range. If so, what happens when you overflow? What happens if you divide by zero? And what happens when you dereference nil? No focking idea.

                                                                                                                              The “spec” is one for a toy language. That is why it is so short. How long would it grow if it were properly specified? Of course you could decide that everything the spec doesn’t cover is undefined and maybe results in program termination. That would make it impossible to write robust programs that can deal with implementation limitations in varying environments (unless you have perfect static analysis). See my point about apples vs oranges.

                                                                                                                              So the deeper question I have is: how small can you make a language with

                                                                                                                              1. a spec that isn’t a toy spec
                                                                                                                              2. not simply shifting complexity to the user
                                                                                                                              3. enough of the same facilities we have in C so that we can interface with the hardware as well as write robust programs in the face of limited & changing system resources

                                                                                                                              Scheme, Oberon, PostScript, Brainfuck, etc. don’t really give us any data points in that direction.

                                                                                                                              1. 5

                                                                                                                                So the deeper question I have is: how small can you make a language with

                                                                                                                                1. a spec that isn’t a toy spec
                                                                                                                                2. not simply shifting complexity to the user
                                                                                                                                3. enough of the same facilities we have in C so that we can interface with the hardware as well as write robust programs in the face of limited & changing system resources

                                                                                                                                Scheme, Oberon, PostScript, Brainfuck, etc. don’t really give us any data points in that direction.

                                                                                                                                Good question. There are few languages with official standards (sorted by page count) that are also used in practice (well.. maybe not scheme ;>):

                                                                                                                                1. Scheme r7rs - 88 pages - seems to be only language without useful standard library
                                                                                                                                2. Ruby 1.8 - 341 pages
                                                                                                                                3. Ada 95 - 582 pages
                                                                                                                                4. Fortran 2008 - 621 pages - seems to be only language without useful standard library
                                                                                                                                5. C11 - 701 pages
                                                                                                                                6. EcmaScript - 885 pages
                                                                                                                                7. Common Lisp - 1356 pages
                                                                                                                                8. C++17 - 1623 pages

                                                                                                                                I know that page count is poor metric, but it looks like ~600 pages should be enough :)

                                                                                                                                1. 3

                                                                                                                                  Here are the page counts for a few other programming language standards:

                                                                                                                                  1. PL/I General purpose subset 443 pages
                                                                                                                                  2. Modula-2 800 pages - base - 707 pages, generics - 45 pages, objects - 48 pages
                                                                                                                                  3. Ada 2012 832 pages
                                                                                                                                  4. Eiffel 172 pages
                                                                                                                                  5. ISO Pascal 78 pages
                                                                                                                                  6. Jovial J73 168 pages
                                                                                                                                  1. 2

                                                                                                                                    I know that page count is poor metric, but it looks like ~600 pages should be enough :)

                                                                                                                                    Given that N1256 is 552 pages, yeah, without a doubt.. :-)

                                                                                                                                    The language proper, if we cut it off starting at “future language directions” (then followed by standard library, appendices, index, etc.) is only some 170 pages. It’s not big, but I’m sure it could be made smaller.

                                                                                                                                  2. 2

                                                                                                                                    I’ve expressed a desire for a “better C” that does all we want from C without all the crap, and I sincerely believe we could make such a thing by taking C, stripping stuff and fixing some unfortunate design choices. The result should be the small and simple core I see in C.

                                                                                                                                    That might be worth you writing up with hypothetical design. I was exploring that space as part of bootstrapping for C compilers. My design idea actually started with x86 assembler trying to design a few, high-level operations that map over it which also work on RISC CPU’s. Expressions, 64-bit scalar type, 64-bit array type, variables, stack ops, heap ops, expressions, conditionals, goto, and Scheme-like macros. Everything else should be expressable in terms of the basics with the macros or compiler extensions. The common stuff gets a custom, optimized implementation to avoid macro overhead.

                                                                                                                                    “ What I would focus on is the language semantics as well as the burden they place on implementation. “

                                                                                                                                    Interesting you arrived at that since some others and I talking verification are convinced a language design should evolve with a formal spec for that reason. It could be as simple as Abstract, State Machines or as complex as Isabelle/HOL. The point is the feature is described precisely in terms of what it does and its interaction with other features. If one can’t describe that precisely, how the hell is a complicated program using those same features going to be easy to understand or predict? As an additional example, adding a “simple, local change” show unexpected interactions or state explosion once you run the model somehow. Maybe not so simple or local after all but it isn’t always evident if just talking in vague English about the language. I was going to prototype the concept with Oberon, too, since it’s so small and easy to understand.

                                                                                                                                    “but because it lacks semantics.”

                                                                                                                                    I didn’t think about that. You have a good point. Might be worth formalizing some of the details to see what happens. Might get messier as we formalize. Hmm.

                                                                                                                                    “So the deeper question I have is: how small can you make a language with”

                                                                                                                                    I think we have answers to some of that but they’re in pieces across projects. They haven’t been integrated into the view you’re looking for. You’ve definitely given me something to think about if I attempt a C-like design. :)

                                                                                                                            2. 4

                                                                                                                              He also says that the issues with memory-safety in C are overrated, so take it with a grain of salt.

                                                                                                                              1. 13

                                                                                                                                He is not claiming that memory safety in general is not an issue in C. What he is saying is that in his own projects he was able to limit or completely eliminate dynamic memory allocation:

                                                                                                                                In the 32 kloc of C code I’ve written since last August, there are only 13 calls to malloc overall, all in the sokol_gfx.h header, and 10 of those calls happen in the sokol-gfx initialization function

                                                                                                                                The entire 8-bit emulator code (chip headers, tests and examples, about 12 kloc) doesn’t have a single call to malloc or free.

                                                                                                                                That actually sounds like someone who understands that memory safety is very hard and important.

                                                                                                                                1. 3

                                                                                                                                  Not at all the vibe I got from it.

                                                                                                                                2. 4

                                                                                                                                  I’m not familiar with either of those languages, but any idea what the author means by this?

                                                                                                                                  I’m also way more interested in Zig than I am in Rust.

                                                                                                                                  What I think he’s saying is that the two “big” languages are overhyped and have gained disproportionate attention for what they offer, compared to some of the smaller projects that don’t hit HN/Lobsters headlines regularly.

                                                                                                                                  Or maybe it’s a statement w.r.t. size and scope. I don’t know Swift well enough to say if it counts as big. But Rust looks like “Rubyists reinvented C++ and claim it to be a replacement for C.” I feel that people who prefer C are into things that small and simple. C++ is a behemoth. When your ideal replacement for C would also be small and simple, perhaps even more so than C itself, Rust starts to seem more and more like an oil tanker as it goes the C++ way.

                                                                                                                                  1. 3

                                                                                                                                    I agree with your point on attention. I just wanted to say maybe we should get a bit more credit here:

                                                                                                                                    “compared to some of the smaller projects that don’t hit HN/Lobsters headlines regularly.”

                                                                                                                                    Maybe HN but Lobsters covers plenty oddball languages. Sometimes with good discussions, too. We had authors of them in it for a few. I’ve stayed digging them up to keep fresh ideas on the site.

                                                                                                                                    So, we’re doing better here than most forums on that. :)

                                                                                                                                    1. 2

                                                                                                                                      Sure! Lobsters is where I first learned about Zig. :-)