Threads for kevincox

  1. 1

    How does this compare to Magic Wormhole?

    1. 11

      Firefox Send is “async”. The file is uploaded first, then the server “buffers” it until the downloader comes by. This enables a nicer workflow but has some downsides (mostly server costs).

      Magic Wormhole is Peer-to-Peer which removes the need for size limits and can have lower end-to-end latency and higher performance if both ends have a good connection (although it also means that both ends have to suffer if one end has a bad connection). But both users need to be online at the same time.

      1. 2

        Wormhole is single-use, whereas Send can be accessed by several people without generating a new link.

        1. 1

          Magic Wormhole has no size limits, which is nicer.

          ffsend might be a little easier to use? Ymmv.

        1. 3

          /tmp/ape

          I hope this isn’t executing some random file out of /tmp if it happens to exist.

          1. 2

            Author here. Here’s the context you didn’t quote. APE first tries to use the ape program off your $PATH. If it isn’t there, then it tries to extract the embedded one to $TMPDIR/ape since it’s defined by POSIX and used by systems like MacOS to create secure user-specific directories. If $TMPDIR isn’t defined, then finally, as a last resort, it extracts to /tmp/ape. This creates a potential race condition where multiple users try to run an APE binary at the same time. So their sysadmin should install the ape program systemwide to fix that.

            1. 1

              I don’t see how this addresses the problem though. Are you saying that it will never use an existing /tmp/ape? If that is the case why use a predictable name instead of something random that doesn’t have the race condition? If it will reuse an existing /tmp/ape then if a malicious user creates a malicious /tmp/ape another user will try to run it.

              1. 2

                That’s a problem with Unix distros rather than APE. The POSIX standard recommendation is to use $TMPDIR. It’s not APE’s responsibility to secure your computer. There’s very little we can do if your distro doesn’t follow standards. We have however always been clear and transparent in documentation. The APE source code has paragraphs talking about this topic. The concern has been raised multiple times on GitHub. If you think there’s something more we could be doing that’s actionable, let me know and I’ll take it into consideration.

                1. 1

                  What standard are you talking about? I don’t recall any standard that suggest that it is safe to execute files out of $TMPDIR.

                  I would also be interested if you have links to previous discussions or documentation.

                  1. 2

                    The Open Group Base Specifications Issue 7, 2018 edition IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008) talks at length in multiple places about TMPDIR and how it addresses concerns about race conditions between multiple users. Concerns about whether or not it’s safe to run programs you’ve downloaded off the internet is a non-technical issue that’s orthogonal to this discussion.

                    1. 2

                      Just naming a spec isn’t a strong argument. But since apparently I’m bored I decided to grep the version of that spec I could find and only see two notable references to TMPDIR. I don’t know if I have a partial or wrong version.

                      Neither of these seem to imply that TMPDIR should be per-user or otherwise safe to execute files from. In fact the second one explicitly says that “Implementations shall ensure that temporary files, when used by the standard utilities, are named so that different utilities or multiple instances of the same utility can operate simultaneously without regard to their working directories, or any other process characteristic other than process ID.” which doesn’t appear to be the case with a static name. Although this is talking more about standard utilities than the properties of the directory itself.

                      But even if there was something in the standard I don’t think it is very useful if no one follows it and AFAICT all major distros have a common TMPDIR that is world writable but sticky. So if you are executing a file out of it that you can’t prove that your user wrote you are gonna have a bad time.

                      1. 1

                        Neither of these seem to imply that TMPDIR should be per-user

                        Could you explain to me how a multi-user directory defined as an environment variable would work?

                        1. 2

                          Forgive me for misunderstanding the conversation, but my assumption in reading the thread is that the question is actually:

                          why aren’t you using mktemp(1) or similar?

                          1. 1

                            Because ${TMPDIR:-/tmp}/ape.$$ effectively does the same thing except with shell builtins. Fast performance is the most important thing. It then atomically renames the file so there’s no race conditions and you only need to store one copy. Not wasting disk space with a zillion $TMPDIR/ape.9QaW9BO1NB files is very important. It’s not possible to program an executable to delete itself when it’s done running. It’s also not acceptable to have executables delete themselves while they’re running. Maximum transparency is a goal of the project.

                            1. 3

                              $TMPDIR is not guaranteed to be user / process specific, and may be /tmp so the concern is the classic “time of check to time of use” vulnerability. I think that’s a valid concern here.

                              The major problem is that I may not be aware that a program I am using is executed via ape. This means that I might not know to install /usr/bin/ape beforehand leaving me open to exploitation.

                              Also, I’m surprised that you wouldn’t be able to register an atexit that:

                              • forks
                              • setsid
                              • rm argv[0]
                              1. 2

                                We can address your concern by making sure the APE loader is included in Unix distros by default. So far operating systems have had a positive reaction to this new format and have gone to great lengths to help us out. For example, the last two years we spent upstreaming changes to zsh, fish, FreeBSD, and NetBSD are what helped us get this far. Back when I started doing this, there was even a meeting with the POSIX committee (Austin Group) about whether or not the rules against executing shell scripts containing binary code should be lifted, in order to permit formats like this. The answer was yes. So we moved forward. This whole topic of “what if /tmp is compromised by an adversary” never really came up. But then again, back then, the favored approach was to self-modify the binary in place. That approach should address your concerns too. However it made other communities like the NixOS crowd unhappy. So what we’re doing is providing a variety of APE utilities and strategies so you can choose the one that works best for you!

                          2. 1

                            I don’t understand the question.

                            What I meant is that if all users have the same directory in TMPDIR then you have to worry about untrusted files in that directory. If each user had a unique TMPDIR that only they could write to them they can trust it’s contents (to some degree).

              2. 1

                /tmp can also be mounted as noexec in some cases, in which case this may not work. Finding someplace where you are guaranteed to be able to write on every system can be quite difficult (/dev/shm looks like a candidate althought that one I really don’t see why you’d ever want to execute things from), but some XDG env vars might help., e.g. XDG_RUNTIME_DIR if set.

                1. 3

                  IIUC the only path that is actually “gaurenteed” to be executable is $HOME/.local/bin as it says “… executable files may be written”. The spec doesn’t say anything specific about exec permissions.

                  But $XDG_RUNTIME_DIR does say “The directory MUST be owned by the user, and he MUST be the only one having read and write access to it” which should mitigate the security problems here.

                  1. 1

                    Would be nice if ape didn’t have to rely on a shell script to start itself, but just execute it directly (I think after you started it at least once it may rewrite itself so it doesn’t have to, but then you lose the “cross OS” portability part).

                    1. 1

                      Author here. You can make your APE binaries directly executable by “assimilating” them. One of the features of the shell script is a ./foo.com --assimilate flag. It causes your binary to be modified in-place to be ELF or Mach-O depending on the system.

                  2. 1

                    /tmp can also be mounted as noexec in some cases

                    If a system administrator has made that choice, then the APE format is designed to respect their decision by not working around it. What APE binaries do instead is print an error. That way the user can make a decision about whether or not he/she wants to install the ape loader systemwide.

                1. 4

                  I wish they spent half this much energy making sure that they didn’t serve CAPTCHAs to read-only cachable requests such as JS files and RSS feeds by default.

                  1. 5

                    The author frames this as a problem with WordPress (the article concludes with WordPress is still vulnerable with the latest version, 6.0. )

                    However, I think that this is a problem with Certificate Transparency;

                    A very common situation is that a user will first configure a web host with a certificate, often using automated tools like Certbot, and then start the installation of a web application like WordPress. The certificate will show up shortly after in one of the public Certificate Transparency logs; this process takes only a few minutes.

                    That makes sense; if you don’t configure the web host with a certificate, you can’t connect to it. Of course you could use HTTP for the installation, but then you’d open a whole different can of security problems. Personally I set up hosts with a self-signed certificate first, and then opt for a publicly trusted certificate when I’m done, but I can see how this could be a bit too advanced for most hobbyists.

                    Let’s Encrypt gives you free certificates, but if you use it, it will announce your hostname to the world. Hey, here is a new web server, maybe you can compromise it before the admin finishes setting it up. The solution would be to make Certificate Transparency more limited; it could require a token in DNS before you can query it. Putting the responsibility for this securty problem on WordPress is the world upside down.

                    But in the meantime, the only way for an admin to be safe is to use wildcard certificates. But that requires DNS challenges which are harder to set up than HTTP challenges, so we will have to live with this problem for a while.

                    1. 39

                      I don’t buy this at all. For one thing, it’s really very easy to solve this within the parameters I think you’re implying: just put WP on an unpredictable path, or gate it behind HTTP basic auth.

                      But more generally I think any security model based on hosting a public server that people can’t find is just broken. Domain names can be discovered in many ways, of which certificate transparency logs are just one newer and particularly easy example. It’s a race against time at best, and an unwinnable race against time against a determined attacker.

                      1. 2

                        I don’t buy this at all. For one thing, it’s really very easy to solve this within the parameters I think you’re implying: just put WP on an unpredictable path, or gate it behind HTTP basic auth.

                        I don’t see how any if this works without WordPress making installs more complicated. Some setup either happens before you upload files, or you get cloud WordPress involved.

                        1. 11

                          This isn’t complicated. If you install it from source, there should be no admin password until you set it in the config (along with the DB connection info and a bunch of other stuff you already have to set). If a hosting company installs it for you, they should assign a random password and give it to you out of band.

                          1. 3

                            IIRC, the traditional wordpress install process involves you configuring the DB connection info in the installer, and the installer writing the config for you - so in theory, you don’t currently need to make any manual config edits.

                            1. 1

                              Presumably they don’t want you to put the admin password in a file … But the point about connection information for the database is completely valid.

                              1. 1

                                Right, I was thinking it’s just an initial password you’d have to change during the web setup process when you set up the real admin account. (Another improvement I’d make is forcing an admin user to be created rather than having an “admin” account exist at all.)

                            2. 4

                              making installs more complicated

                              Even with a key, opening a locked door is harder than opening an unlocked one; that’s not a great argument for leaving your doors unlocked.

                              Wordpress could require you to set an “initial setup password” at install time, or require that initial setup be done on the loopback address (you can ssh-forward a port to connect), or any number of other methods.

                              1. 4

                                Wordpress could require you to set an “initial setup password” at install time, or require that initial setup be done on the loopback address (you can ssh-forward a port to connect), or any number of other methods.

                                And yet, all of these things are significant barriers to install, something WordPress has optimized over it’s 20ish year history. Should they do something? Yes. Are they likely to do something that complicates the install process too much more than they already have to? Pretty unlikely.

                                So, let’s see in order of “these are mitigations I can think of off the top of my head* with difficulty between 1 (easy) and 5 (easy) for J. Basic User:

                                1. Have the user setup http basic auth before install: 5
                                2. Require the user edit a file before uploading it to the website that has an install password: 3
                                3. Move the install to a random directory for setup, and then move it back / symlink it / whatever: 4
                                4. Require registering the domain with WordPress.com and somehow validating it when you login via WordPress.com: 2
                                5. Recommend people use web hosts that have 1-click WordPress setups, and make them deal with it: 1
                                6. Do nothing: 1

                                Not sure of the install statistics, but I’d guess that 1-click setups are probably fairly popular, so the “experts” that are offering them should be able to take on most of the burden. Then, maybe the edit a file before uploading is good enough for the bulk of the other users who are going so far as to set it up themselves, completely… Seems reasonable to me.

                          2. 17

                            The actual problem is:

                            These installers usually come without any authentication. Whoever accesses a web host with a WordPress installer can finish the installation.

                            There was a discussion here last week that went into “secure by default.” Security is improved by not having to turn security features on in the first place. Usually that’s because users may forget or put off turning them on. In this case it’s because of a race condition.

                            A program should not ship with fixed, well-known credentials to administer it. This has caused so many problems in the past, such as millions of easily-hackable MongoDB installations. Raspberry Pi just updated their installers to stop creating a default “pi” admin account.

                            1. 2

                              I think the problem here is that you unzip the downloaded release onto your shared hosting account, but then you need “somehow” to supply it with unique credentials just for you. Since it’s all about ease of use, editing a file remotely might be too much to ask (especially if it requires you to get the syntax right), this is not really an option.

                              It doesn’t really have fixed credentials BTW; it just presents you with an installation wizard which sets up the admin user for you with a password of your own choosing.

                              1. 2

                                If you can unzip the files remote editing seems like a pretty low bar. Maybe they can make a “config generator” that produces a second zip that sets a password (that is also shown on the website). If you want to be really fancy just inject a random password on the fly into the main zip. (Computationally easy but removes the ability to use a basic HTTP CDN)

                                But in my mind just unzipping an archive into your webserver shouldn’t allow remote code execution. That seems like the wrong tradeoff to me. The unzipped files should be “unarmed” by default.

                                1. 1

                                  I mean, you could have a setting “mode = uninitialized” in the zip, then on startup, WP sees that, and dumps a password into startup-password.txt on the machine in a private location accessible to SFTP but not WWW, and then WP deletes that file the first time someone logs in. There are plenty of ways to do it. You just have to care about being the largest source of exploits on the web today and not shrug it off.

                              2. 5

                                How is it ever acceptable for it to start a publically connectible instance without a password? It should just create a password on set-up and print that to console or to a file, the hosters can then take that and for example email it to the customer or display it in their management console or whatever.

                                1. 3

                                  oh it’s definitely a problem with wordpress, or more specifically:

                                  […] that an installation will usually be performed quickly and thus it should pose little risk […]

                                  Here is the mistake. Don’t put an open system on the public internet and expect it to be safe.

                                  Now how do you make it secure without making the setup complicated is the question, as stated many times below.

                                  1. 2

                                    The solution would be to make Certificate Transparency more limited; it could require a token in DNS before you can query it.

                                    Offering a way to turn off CT is like offering a backdoor for encryption. No matter how much you claim that only the “good guys” will be able to use it, sooner or later it is inevitable that the “bad guys” will be able to use it to.

                                    Meanwhile this is Wordpress’ problem. The security of the installer is entirely dependent on nobody else knowing there’s a Wordpress installer running on a given URL; that’s simply not workable as a security model, and honestly hasn’t been for years and years. CT is just the thing that’s making you aware of it, but if you think the “bad guys” don’t or won’t ever have other ways to scan newly-registered or newly-provisioned stuff, well, you’re just flat wrong. People are going to figure out ways to do this, and every time a vector opens up for it Wordpress will be broken.

                                    So this absolutely is Wordpress’ problem (and any other software that has the same “security” model). There are tons of ways to fix this — injecting a randomly-generated password into the download and making the user copy it from the page wouldn’t even be that hard! — and it’s time for WP to adopt one of them.

                                    1. 1

                                      The solution would be to make Certificate Transparency more limited; it could require a token in DNS before you can query it.

                                      So basically I’m translating this as:

                                      “it would require the DNS admin to do operations before the unsecure wordpress can be exploited”.

                                      How would this solve the real problem of wordpress being unsecure by default?

                                      Putting the responsibility for this securty problem on WordPress is the world upside down.

                                      It’s just 2 separate things:

                                      • discoverability of an unsecure system (cert transparency is just one way to find them)
                                      • leaving a system vulnerable

                                      Fixing one won’t fix the other.

                                      1. 2

                                        discoverability of an unsecure system (cert transparency is just one way to find them)

                                        It’s a very effective one, I would argue it’s even the most effective one by far. Other methods (continuous scanning) take a lot more effort and are a lot harder to do covertly. Especially if you want to hit the time window between uploading and setting the admin password, which will typically be well under an hour.

                                        So basically I’m translating this as:

                                        “it would require the DNS admin to do operations before the unsecure wordpress can be exploited”.

                                        DNS admins do typically not allow zone transfers to anyone, try running dig AXFR lobste.rs @ns1.dnsimple.com, yet crt.sh shows me that there was an l.lobste.rs at some point. Why should I be allowed to query this, and why is it not possible to opt-out from that? So why the difference? Why do we still block zone transfers as a security measure (don’t show more data than you have to), yet we’re fine with bad actors subscribing to CT feeds telling them which hostnames are new and possibly an easier target.

                                        Setting a token in your TXT record, such as the sha512 hash of a secret string, and requiring to provide the secret string when querying CT logs for your domain, would improve security of CT a lot.

                                        But making CT opt-in might be even better, as CT does not really offer much unless you’re one of the big ones. Most reports I’ve seen from people monitoring CT for their domain, is figuring out that their cloud provider switched CAs, such as recently when Cloudflare started signing using two different CAs (one wonders why people who use CT don’t use CAA records). But I have yet to see a small site use CT to find out about a rogue CA (not just bad practices, actual exploitable security problems), and be able to do something about it.

                                        1. 6

                                          Making CT opt-in… doesn’t really work. The point, for better or worse, of CT as a security model is that all CAs are observable and their mistakes can be picked up in third-party audits. If there’s any mechanism for secret certificates to be issued, CT instantly loses most of its value.

                                          Also, CT is designed to provide an efficient means for third parties to check that the CT log services themselves are serving the same log to everyone. But that requires CT log services to serve the same log to everyone. If we accept the existence of entries that the server is not prepared to reveal to everyone, we’re also accepting a reduction in the level of confidence we can have in CT itself.

                                          Incidentally, most domains are discovered by passive DNS monitors at some point, and this data is generally available for sale (to security companies, but you can count on bad actors finding a way). How quickly that happens will depend on various factors, but it can be comparable to CT—so while it’s obvious that CT makes this kind of attack a lot easier, I don’t think it was ever safe. CT’s contribution might even be a positive one, if it ultimately leads to more effort invested in making the secure way easy.

                                          In any case, having lost the obscurity in a security-through-obscurity setup, I think it’s probably wrong to focus on trying to get the obscurity back.

                                    1. 2

                                      I hate the argument that there should be a place for only “system indicators”. Just because you make the shell doesn’t make you special. There should be no reason that using GNOME with NetworkManager should get you a wifi icon but using something else should. I don’t like the the DE makers are getting all high and mighty with this feature that they make for themselves but can’t trust others with.

                                      1. 7

                                        I’ve basically stopped trying to use the desktop versions of electron apps like discord; I just use the website. For that matter, I have the server portion of element-web running in a docker container on my laptop just so I don’t have to run an electron process for it.

                                        There’s obviously something compelling about being able to package up a web app as a “desktop app,” but I’m wondering if the desktop browsers supporting the various technologies that go under the heading of PWAs wouldn’t be a better fit.

                                        This doesn’t necessarily go for things like vscode, which actually do things that aren’t just talk to a web server. But even there, it’d be nice not to have to spawn a whole separate browser.

                                        1. 2

                                          That’s not helpful for many of us as the web app lacks the single most important feature - push to talk.

                                          1. 3

                                            You can set up a local mic mute push-to-talk hotkey which solves this issue and works for all programs on your computer.

                                            1. 1

                                              honestly it boggles my mind that people sell microphones that don’t have this implemented in hardware. what the hell people.

                                            2. 1

                                              I’m not sure such a feature could work in Wayland, as it requires a global hotkey. Perhaps it would be ideal to have it as a system function instead, where your microphone device is muted unless you hold the key.

                                          1. 5

                                            I feel the same way. I want to like formatters but at the end of the day I find that human logic is really the best. Formatters tend to focus on unimportant things like line-length. What I care about is line complexity, I want to keep each line having one, maybe two ideas. No formatter can understand that.

                                            I don’t mind some basic formatting such as brace style and fixing indentation but after much more than that I find it is putting things onto one line when it shouldn’t or vice-versa. I think things like clang-format have tried to take on too much and really make all code look the same. But I’ve never been confused by code because it used a+b rather than a + b.

                                            1. 2

                                              Interesting article. I almost skipped over it because of its name mentioning a legacy technology (like if it had talked about TLS by mentioning “SSL certificates”).

                                              Does anybody know if broadcasting Atom feeds has any impact on Search Engine Optimization?

                                              1. 1

                                                RSS seems to be the colloquial term for feeds in general. And “RSS Feeds” gets people thinking about the right thing whereas “Feeds” is a little too generic.

                                                Google at least will subscribe to your feed and treat it similarly to a sitemap. I don’t know if this directly affects your ranking, I would assume not or at least not much.

                                              1. 3

                                                Multiplicities are hard if you don’t study your theory.

                                                • One: T
                                                • Zero or one: Maybe
                                                • Zero or many: List

                                                And now you combine them with tuples.

                                                • One or two: (T, Maybe)
                                                • At least three: (T, T, T, List)
                                                • Two to five: (T, T, Maybe, Maybe, Maybe)

                                                Of course, you can throw dependant types and interfaces to it to make it nicer.

                                                1. 6

                                                  Laurence Tratt has been publishing PLT papers for almost two decades, I think he knows the theory.

                                                  1. 4

                                                    Yes, but are there any programming languages that make structures like this easy to work with?

                                                    For example if you have “at least one” and (T, List) and you want to insert into the front of the list it is a bit awkward. I think you could probably make a generic type that made this easy to work with (or even adding a min parameter to existing list types) but I wouldn’t want to pass around funny tuples.

                                                    1. 1

                                                      We use the NonEmpty type in Haskell all the time, it has mostly the same interface as list. IMO the basic list of structures you need for 99% of cases representing relationships between data are:

                                                      • a
                                                      • Maybe a
                                                      • [a]
                                                      • NonEmpty a

                                                      Beyond that is quite rare, and at that point you should really be using custom types that only allow the valid states.

                                                  1. 6

                                                    There are multiple points here I disagree with:

                                                    1. Go’s and Zig’s defer are rather different beasts Go runs defered statements at the end of the function, Zig at the end of scope. ant to lock a mutex insife a loop? Can’t use Go defer for that..
                                                    2. destructors can’t take arguments or return values While most destructions only release acquired resources, passing an argument to a defered call can be very useful in many cases
                                                    3. hidden code all defer code is visible in the scope. Look for all lines starting with defer in the current scope and you have all the calls. Looking for destructors means looking how drop is implemented for all the types in the scopes.
                                                    1. 11

                                                      Go’s and Zig’s defer are rather different beasts Go runs defered statements at the end of the function, Zig at the end of scope. ant to lock a mutex insife a loop? Can’t use Go defer for that..

                                                      This distinction doesn’t really matter in a language with first-class lambdas. If you want to unlock a mutex at the end of a loop iteration with Go, create and call a lambda in the loop that uses defer internally.

                                                      destructors can’t take arguments or return values

                                                      But constructors can. If you implement a Defer class to use RAII, it takes a lambda in the constructor and calls it in the destructor.

                                                      hidden code all defer code is visible in the scope

                                                      I’m not sure I buy that argument, given that the code in defer is almost always calling another function. The code inside the constructor for the object whose cleanup you are defering is also not visible in the calling function.

                                                      1. 4

                                                        hidden code all defer code is visible in the scope

                                                        I’m not sure I buy that argument, given that the code in defer is almost always calling another function. The code inside the constructor for the object whose cleanup you are defering is also not visible in the calling function.

                                                        The point is that as a reader of zig, you can look at the function and see all the code which can be executed. You can see the call and breakpoint that line. As a reader of c++, it’s a bit more convoluted to breakpoint on destructors.

                                                        1. 2

                                                          you can look at the function and see all the code which can be executed.

                                                          As someone that works daily with several hundred lines functions, that sounds like a con way more than a pro.

                                                        2. 1

                                                          But constructors can.

                                                          This can work sometimes, but other times packing pointers in a struct just so you can drop it later is wasteful. This happens a lot with for example the Vulkan API where a lot of the vkDestroy* functions take multiple arguments. I’m a big fan of RAII but it’s not strictly better.

                                                          1. 1

                                                            At least in C++, most of this all goes away after inlining. First the constructor and destructor are both inlined in the enclosing scope. This turns the capture of the arguments in the constructor into local assignments in a structure in the current stack frame. Then scalar replacement of aggregates runs and splits the structure into individual allocas in the first phase and then into SSA values in the second. At this point, the ‘captured’ values are just propagated directly into the code from the destructor.

                                                          2. 1

                                                            If you want to unlock a mutex at the end of a loop iteration with Go, create and call a lambda in the loop that uses defer internally.

                                                            Note that Go uses function scope for defer. So this will actually acquire locks slowly then release them all at the end of function. This is very likely not what you want and can even risk deadlocks.

                                                            1. 1

                                                              Is a lambda not a function in Go? I wouldn’t expect defer in a lambda to release the lock at the end of the enclosing scope, because what happens if the lambda outlives the function?

                                                              1. 1

                                                                Sorry, I misread what you said. I was thinking defer func() { ... }() not func() { defer ... }().

                                                                1. 2

                                                                  Sorry, I should have put some code - it’s much clearer what I meant from your post.

                                                          3. 5

                                                            The first point is minor, and not really changing the overall picture of leaking by default.

                                                            Destruction with arguments is sometimes useful indeed, but there are workarounds. Sometimes you can take arguments when constructing the object. In the worst case you can require an explicit function call to drop with arguments (just like defer does), but still use the default drop to either catch bugs (log or panic when the right drop has been forgotten) or provide a sensible default, e.g. delete a temporary file if temp_file.keep() hasn’t been called.

                                                            Automatic drop code is indeed implicit and can’t be grepped for, but you have to consider the trade-off: a forgotten defer is also invisible and can’t be grepped for either. This is the change in default: by default there may be drop code you may not be aware of, instead of by default there may be a leak you may not be aware of.

                                                            1. 3

                                                              destructors can’t take arguments or return values. While most destructions only release acquired resources, passing an argument to a deferred call can be very useful in many cases.

                                                              Yes, more than useful:

                                                              • Zero-cost abstraction in terms of state: A deferred call doesn’t artificially require objects to contain all state needed by their destructors. State is generally bad, especially references, and especially long lived objects that secretly know about each other.
                                                              • Dependencies are better when they are explicit: If one function needs to run before another, letting it show (in terms of what arguments they require) is a good thing: It makes wrong code look wrong (yes, destruction order is a common problem in C++) and prevents it from compiling if you have lifetimes like Rust.
                                                              • Expressiveness: In the harsh reality we live in, destructors can fail.

                                                              I think the right solution is explicit destructors: Instead of the compiler inserting invisible destructor calls, the compiler fails if you don’t. This would be a natural extension to an explicit language like C – it would only add safety. Not only that: It fits well with defer too – syntactic sugar doesn’t matter, because it just solves the «wrong default» problem. But more than anything, I think it would shine in a language with lifetimes, like Rust, where long lived references are precisely what you don’t want to mess with.

                                                              1. 2

                                                                You could run an anonymous function within a loop in Go, just to get the per-loop defer. Returning a value in a defer is also possible.

                                                                func main() (retval int) {
                                                                    for {
                                                                        func() {
                                                                            // do stuff per loop
                                                                            defer func() {
                                                                                // per loop cleanup
                                                                            }()
                                                                        }()
                                                                    }
                                                                    defer func() {
                                                                        retval = 42
                                                                    }()
                                                                    return
                                                                }
                                                                
                                                              1. 3

                                                                I completely agree this is a downside of defer, but I do think there are a few (contingent on the task in hand) reasons to prefer it over RAII. One thing that’s pretty nice about Zig code is that it’s really easy to reason about its performance due to things being pretty explicit, and RAII does erode this slightly as there is (if I understand correctly) potentially arbitrary amounts of code being executed when exiting some scope, so you have to read a lot more code before fully understanding the performance of a leaf function. I admit this is not really a real issue for most programmers most of the time though. Another more common case where defer is really nice is when calling C libraries, which there are of course a huge number of. defer in Zig lets you use those libraries and have fairly nice resource clean-up happening without needing to build and maintain any wrappers. RAII also seems like it pushes you down the exceptions route for handling errors during initialisation and clean-up, and exceptions certainly have their downsides.

                                                                1. 4

                                                                  It would be intetto have a language with “explicit RAII”. You can define a destructor, but instead of calling it for you the compiler just complains if you forget to call it. This way it would be visible in your code (likely with a defer statement) but hard to forget.

                                                                  when calling C libraries

                                                                  When interfacing with a C library in C++ or Rust one of the first things I tend to do is make small wrappers for resources that implement RAII for them. I find they pays off very quickly in terms of productivity and correctness. Even if I don’t wrap much else this is very useful.

                                                                  1. 1

                                                                    Heyyy, similar idea! See above.

                                                                    1. 2

                                                                      I read that comment difficulty. I read that comment as code such as:

                                                                      autoclose File f = open(...)
                                                                      

                                                                      I am talking about “regular” RAII as seen in C++ or Rust except that it fails to compile if you forget it. It isn’t opt-in because it is required but the caller still needs to write cleanup code (well call the cleanup function). For example.

                                                                      f = open(...)
                                                                      f.write("foo")
                                                                      // Error here: "f implements Drop and must be cleaned up.
                                                                      

                                                                      Of course you could combine these ideas because something like an autoclose attribute would manage the cleanup and prevent the error from firing.

                                                                1. 1

                                                                  I have to say, this post didn’t give me the best impression of the author. They are clearly new to Rust and don’t understand why a lot of decisions were made but are strongly asserting about their opinions without comparing them to the actual reasons that things are that way. Of course lots of the complaints are valid but still the way it was presented and that these interactions didn’t strike me as the best attitude.

                                                                  A C function call is by definition “unsafe”, I shouldn’t have to tell the Rust compiler that.

                                                                  This is completely missing the point. Of course the compiler knows, that is how it can raise an error. This is letting your teammates know that this code is potentially dangerous and needs to be treated with more care when reviewing and modifying it.

                                                                  But yes, I can imagine that if you are dealing with a lot of C FFI it can feed tedious. Generally the best approach here is to isolate C FFI into a small region of your code and write safe wrappers around commonly used functions. Of course if you are consuming a huge API service this also gets tedious fast.

                                                                  1. 8

                                                                    It is pretty damming to the Go language that you can’t use any existing code. Just about every other language provides relatively straightforward (if not seamless) interop with any other C ABI. Only in Go have I heard such consistent and negative opinions on the FFI. Java and JNI are close, but it still seems better received and in that case at least there is a decent reason because ruins your portability once you add native code.

                                                                    The fact that someone would recomend “reimplementing a large piece of C code in Go” instead of just binding to it is exposing a huge downside of the language.

                                                                    1. 4

                                                                      The fact that someone would recomend “reimplementing a large piece of C code in Go” instead of just binding to it is exposing a huge downside of the language.

                                                                      The main reason is so you “get” effortless portability as a result. I can only think of zig where you get some out-of-the-box portability without re-writing your C in zig (since it ships with libc for various platforms/archs and has all the toolchain nicely setup).

                                                                      1. 2

                                                                        i immediately thought of zig’s self contained compiler when i saw this post… and i recall things being posted to show how you can integrate zig cc in w/ go/cgo to have portable c compilers

                                                                        seems like it would be a good thing for these project maintainers to get on board with…

                                                                        1. 7

                                                                          I wrote a blog post where I cross-compiled with Zig the CGo SQLite library for all the major OSs without too much fuss.

                                                                          https://zig.news/kristoff/building-sqlite-with-cgo-for-every-os-4cic

                                                                      2. 3

                                                                        I can’t wait for Go to have an FFI some day!

                                                                        1. 1

                                                                          As mentioned above, I believe this to be simply untrue: Go has an FFI today and it’s called cgo. What is it about cgo that does not make it an FFI?

                                                                          1. 1

                                                                            cgo is basically a separate language. It is a separate implementation.

                                                                            1. 2

                                                                              I can’t see how it’s a separate language. You embed a bit of C code in a special place within a Go file. The C code is compiled by a C compiler, the Go code by a compiler and cgo. And from the C and the Go code, cgo generates some interface code to make C names known to the Go compiler and some Go names known to the C compiler. How is cgo (which, to me, is a program) a separate language?

                                                                              It is a separate implementation.

                                                                              cgo is a separate implementation of what?

                                                                        2. 2

                                                                          Yes 100%, here is my lament from 4 years ago on that topic.

                                                                          https://news.ycombinator.com/item?id=16741043

                                                                          A big part of my pain, and the pain I’ve observed in 15 years of industry, is programming language silos. Too much time is spent on “How do I do X in language Y?” rather than just “How do I do X?”

                                                                          For example, people want a web socket server, or a syntax highlighting library, in pure Python, or Go, or JavaScript, etc. It’s repetitive and drastically increases the amount of code that has to be maintained, and reduces the overall quality of each solution (e.g. think e-mail parsers, video codecs, spam filters, information retrieval libraries, etc.).

                                                                          There’s this tendency of languages to want to be the be-all end-all, i.e. to pretend that they are at the center of the universe. Instead, they should focus on interoperating with other languages (as in the Unix philosophy).

                                                                          One reason I left Google over 6 years ago was the constant code churn without user visible progress. Somebody wrote a Google+ rant about how Python services should be rewritten in Go so that IDEs would work better. I posted something like <troll> … Meanwhile other companies are shipping features that users care about </troll>. Google+ itself is probably another example of that inward looking, out of touch view. (which was of course not universal at Google, but definitely there)

                                                                          This is one reason I’m working on https://www.oilshell.org – with a focus on INTEROPERABILITY and stable “narrow waists” (as discussed on the blog https://www.oilshell.org/blog/2022/02/diagrams.html )

                                                                          (copy of HN comment in response to almost the same observation!)

                                                                          I’m also excited about Zig for this reason. e.g. “maintain it with Zig” https://kristoff.it/blog/maintain-it-with-zig/

                                                                          1. 1

                                                                            On the other hand, oilshell is not(?) compatible with the piles of bash (and sh, and…) scripts out in the world, so folks have to rewrite it to be compatible with your shell. Is this not contradicting what you said earlier?

                                                                            1. 3

                                                                              Hm honest question: Why do you think it’s not compatible?

                                                                              It’s actually the opposite – it’s the ONLY alternative shell that’s compatible with POSIX sh and bash. It’s the most bash compatible shell by a mile.

                                                                              Try running osh myscript.sh on your shell scripts and tell me what happens!


                                                                              The slogan on the front page is supposed to emphasize that, but maybe it’s not crystal clear:

                                                                              It’s our upgrade path from bash to a better language and runtime.

                                                                              Also pretty high up on the FAQ is the statement:

                                                                              http://www.oilshell.org/blog/2021/01/why-a-new-shell.html#introduction

                                                                              OSH is a shell implementation that’s part of the Oil project. It’s compatible with both POSIX and bash. The goal is to run existing shell scripts. It’s done so since January 2018, and has matured in many regular releases since then.


                                                                              Nonetheless I think it could be clearer, so I filed a bug to write a project tour and put it prominently on the home page:

                                                                              https://github.com/oilshell/oil/issues/1127

                                                                              It is disappointing to me that this hasn’t been communicated after so many years … I suspect that some people actually think the project is impossible. It’s both compatible AND it’s a new language.

                                                                              I described in the latest blog post how that works:

                                                                              http://www.oilshell.org/blog/2022/05/release-0.10.0.html#backward-compatibility-the-eternal-puzzle

                                                                              Here are all your other alternative shell choices, NONE of which have the compatibility of OSH.

                                                                              https://github.com/oilshell/oil/wiki/Alternative-Shells

                                                                              (That is why the project is so large and long)

                                                                              1. 1

                                                                                Ah, I’m sorry. I skimmed the FAQ but missed that sentence. For some reason, the impression I got from your FAQ is that it’s basically yet another shell that doesn’t offer backwards compatibility. Obviously I was terribly wrong. I’m not sure how to suggest changes that may have prevented that (other than it’s completely my fault for misreading/skimming and getting the wrong impression.) So, sorry for the noise.

                                                                                1. 1

                                                                                  OK no worries … I think it actually did point out that this crucial fact about the project is somewhat buried. Not entirely buried but “somewhat”.

                                                                          2. 2

                                                                            It is simply not true that “you can’t use any existing code” in Go. There’s cgo and it allows you to to call into C code and provides way for C code to call into go code - that’s pretty much the definition of using existing code. I think a big reason people are complaining about JNI is the the same for people complaining about cgo: Because you are dealing with garbage collected language, there are rules about what you can do with memory and pointers. The same applies to .NET as well.

                                                                            The fact that someone would recomend “reimplementing a large piece of C code in Go” instead of just binding to it is exposing a huge downside of the language.

                                                                            As the article points out, in the very first sentence, most people use mattn/go-sqlite3 which is in fact a wrapper around the canonical C SQLite implementation. A “decent reason” (your words) to not use that library is because “it ruins your portability” because “you add native code”. This reason is at play here.

                                                                            This being said, the shown port to Go is a respectable effort. While being impressive, I’d probably use one of the bindings to the canonical C code if possible as it uses a highly tested implementation. If not possible the cznic provides an interesting alternative.

                                                                            1. 1

                                                                              Yes and no. I mean there is CGO. Which you can use. While it’s worse in Go, also because of threading, especially on Linux you’ll still find “pure” implementations of things that would usually use a C library, sometimes they are faster because calling the FFI might still be slow. Database interfaces are one such an example, where people sometimes find the bottleneck to be the FFI.

                                                                              You also get certain benefits from not using C. I already mentioned the threading part which sometimes bites people in Go, but also you can be sure about memory safety, debugging will be easier, all the people using the project can contribute even when they are not fluent in C, etc.

                                                                              And if you still want/need to use C, there is CGO.

                                                                              There certainly have been cases in other languages where I wished a library wasn’t just a wrapper around C, be it Python, Java, Objective-C/Swift or in node.js-projects. Given circumstances they can be a source for headaches.

                                                                            1. 5

                                                                              To make a tag suggestion stick, you’ll have to find more examples than just one. What other recent or not so recent submissions would benefit from a humou?r tag?

                                                                              1. 5

                                                                                Sure. I looked at the list of recently submitted posts tagged ‘satire’. I found 2x satire; 2x unclear; 1x about satire (but not itself satirical); 4x humorous (including one by a site admin, and one with a comment that the tag may have been unsuitable).

                                                                                1. 1

                                                                                  Maybe we can just rename satire to humour? It seems like a fairly slow tag anyways and I imagine that there is a lot of similar content and preferences between these two.

                                                                                  That being said renaming breaks urls and stuff so many we can document that the “satire” tag includes all humour?

                                                                              1. 2

                                                                                Wow! This is a great introduction to how futures actuatwork below the API! I had a rough idea here before but the code walkthrough really nails it down.

                                                                                1. 5

                                                                                  Yes it matters.

                                                                                  At least with C++ developers can slowly learn the more arcane part of the language language while they use it. A bit more difficult with Rust.

                                                                                  Furthermore, it might be possible to implement some form of borrow checking for existing languages.

                                                                                  Any language should be easy to learn. That’s true for C and python. Language popularity is highly correlated with ease of learning. And this is true for all the new languages out there that try to do fancy things: most developers do not care.

                                                                                  Personally, all I would ever want, is something mostly like C/C++, with pythonic features, easier to read and use, faster to compile, without a GC, statically compiled, without sophisticated things.

                                                                                  1. 16

                                                                                    I wouldn’t call C easy to learn. It probably has less essential complexity than Rust has, but there’s still a lot of fiddly details to learn that wouldn’t come up in languages created decades later with garbage collection and better tooling and syntactic defaults.

                                                                                    1. 8

                                                                                      A couple issues I found when wanting to learn C is all of the variation because of its history. What tooling should I use? Which conventions should I follow? Which version is the current version?

                                                                                      The various C standards are not conveniently discoverable and even when you map them out, they’re written in reference to past standards. So to get the set of rules you have to mentally diff K&R C with a handful of other standards published over 40 years, etc. Starting completely from no knowledge and trying to figure out “What are the complete set of rules for the most modern version of C?” is nearly impossible. At least that has been my experience and biggest struggle when trying to get started with C multiple times over the years.

                                                                                      Then I constantly see veteran C programmers arguing with each other about correct form, there seems to be far less consensus than with modern languages.

                                                                                      1. 4

                                                                                        I’d say C is easy to learn but hard to master. But that could be said about a lot of languages.

                                                                                        1. 2

                                                                                          I think there is a big difference to what is the absolute minimum you can learn.

                                                                                          You can “learn” C with programs that compile and run the happy path mostly correctly. The probably have tons of bugs and security issues but you are using the language.

                                                                                          Rust forces you to handle these issues up front. This does make the minimal learning longer but the total learning to be a “production ready coder” is probably actually shorter.

                                                                                        2. 14

                                                                                          Man, I was terrified when I was learning C++. I would stick to the parts I was “comfortable” with, but when I would call someone else’s code (or a library) I couldn’t reliably know how the features they used would intersect with mine. And the consequences very often were debugging core dumps for hours. I’m no Rust fanboy, but if you’re going to have a language as complicated as Rust or C++, I’d rather learn with one that slaps my hand when doing something I probably oughtn’t do.

                                                                                          1. 11

                                                                                            So, Nim once ARC lands?

                                                                                            1. 3

                                                                                              Is that not the case already ? I unfortunately do not use Nim often these days so I might be out of touch, but if I recall correctly arc/orc are available but not the default.

                                                                                              EDIT: Yeah, It seems to use ref counting by default currently but the doc advise to use orc for newly written code Cf: https://nim-lang.github.io/Nim/mm.html

                                                                                            2. 8

                                                                                              Any language should be easy to learn. That’s true for C and python. Language popularity is highly correlated with ease of learning.

                                                                                              All other things being equal, yes, ease of learning is good. But at some point one may have to sacrifice ease of learning to make the expert’s work easier or more reliable or faster or leaner. Sometimes that’s what has to be done to reach the level of quality we require.

                                                                                              If it means some programmers can’t use it, so be it. It’s okay to keep the incompetents out.

                                                                                              1. 8

                                                                                                I was mostly with you, but “incompetents” is harsh.

                                                                                                1. 4

                                                                                                  Can we at least agree that there is such a thing as incompetent programmers? I’m all for inclusivity, but at some point the job has to get done. Also, people can learn. It’s not always easy, but it’s rarely impossible.

                                                                                                  1. 4

                                                                                                    There are, but generally they’re not going to be successful whether they use Rust or another language. There are inexperienced developers who aren’t incompetent but just haven’t learned yet who will have an easier time learning some languages than Rust, and there are also experienced programmers who simply don’t know Rust who will also have an easier time learning other languages than Rust. Since the incompetent programmers will fail with or without Rust, it seemed like you were referring to the other groups as incompetent.

                                                                                                    1. 5

                                                                                                      Ah, the permanent connotation of “incompetent” eluded me. I was including people who are not competent yet. You only want to keep them out until they become competent.

                                                                                                      My original point was the hypothesis that sometimes, being expert friendly means being beginner hostile to some extent. While it is possible (and desirable) to lower the learning curve as much as we reasonably can, it’s rarely possible to flatten it down to zero, and in some cases, it just has to be steep.

                                                                                                      Take oscilloscopes for instance. The ones I was exposed to in high school were very simple. But the modern stuff I see now is just pouring buttons all over the place like a freaking airliner! That makes them much scarier to me, who have very little skill in electronics. But I also suspect all these buttons are actually valuable to experts, who may have lots of ways to test a wide variety of circuits. And those button give them a more direct access to all that goodness.

                                                                                                      In the end, the question is, are steep learning curve worth it? I believe that in some cases, they are.

                                                                                                      1. 3

                                                                                                        That makes sense. Thanks for clarifying. I don’t know if I have a strong opinion, but I do believe that there are cases that require extreme performance and that often requires expertise. Moreover, having been a C++ programmer for a time, I’m grateful that where C++ would accept a broken program, Rust slaps my hand.

                                                                                                2. 2

                                                                                                  True. Ada Programming language easy to learn but not widely accepted or used

                                                                                                3. 4

                                                                                                  At least with C++ developers can slowly learn the more arcane part of the language language while they use it. A bit more difficult with Rust.

                                                                                                  I’m not sure what parts of Rust you consider “arcane”. The tough parts to learn, borrow checking and lifetimes, aren’t “arcane” parts of Rust; they are basically it’s raison d’être.

                                                                                                  Any language should be easy to learn.

                                                                                                  Ideally languages would be as simple/easy as they can be to meet their goals. But a language might be the easiest-to-learn expression of a particular set of goals and still be tough to learn – it depends on the goals. Some goals might have a lot of inherent complexity.

                                                                                                  1. 3

                                                                                                    If carefully aware of escape analyses as a programmer, you might realize that with Go, and, while programming Go for several years, I’m by no means a Go fanboy.

                                                                                                    In particular, I conjecture that you could write a program in Go that does not use GC, unless the standard library functions you use themselves use GC.

                                                                                                    I need to learn Rust, i realize, having written that prior sentence, and having been originally a C fan.

                                                                                                    1. 6

                                                                                                      Personally, I would strongly recommend the O’Reilly “Programming Rust, 2nd ed.” For me it was a breakthrough that finally allowed me to write Rust and not get stuck. It may not be “perfectly shiny” what I write, but before that, I often stumbled into some situations I just couldn’t get out of. Now I understand enough to be able to at least find some workaround - ugly or not, but it lets me go on writing.

                                                                                                      Also, coming from Go (with a history of C++ long ago beforehand), one thing I had to get over and understand “philosophically” was the apparent lack of simplicity in Rust. For this, my “a ha” moment was realizing, that the two languages make different choices in a priorities triangle of: simplicity vs. performance vs. security. Go does value all 3, but chooses simplicity as the highest among them (thus GC, nulls, etc; but super approachable lang spec and stdlib APIs and docs). Rust does value all 3 too, but chooses performance AND security as the highest. Thus simplicity necessarily is just forced to the back-seat, with a “sorry, man; yes, we do care about you, but now just please stay there for a sec and let us carry out the quarrel we’re having here; we’ll come back to you soon and really try to look into what you’d like us to hear.” And notably the “AND” here is IMO a rather amazing feat, where before I’d assume it just has to often be an “or”. Also this theory rather nicely explains to me the sparking and heated arguments around the use of unsafe in the community - it would appear to happen around the lines where the “AND” is, or looks like it is, kind of stretching/cracking.

                                                                                                    2. 2

                                                                                                      Personally, all I would ever want, is something mostly like C/C++, with pythonic features, easier to read and use, faster to compile, without a GC, statically compiled, without sophisticated things.

                                                                                                      I think you’re looking for Myddin (still WIP) or possibly Hare. Whether they’re “pythonic” is debatable though.

                                                                                                      1. 1

                                                                                                        Any language should be easy to learn.

                                                                                                        Not only easy to run. Easy. Because why would one make it difficult if it clearly can be made easy? The whole point of programming languages is providing simoler alternatives to the targets of their compilers.

                                                                                                      1. 2

                                                                                                        It says the key wkll be backed up by these companies sync services but will I be able to backup my key myself?

                                                                                                        And how do I rotate my key? Likely I need to rotate it for every website? (If supported)

                                                                                                        1. 22

                                                                                                          I think “No Third Party” in the case of magic links is wrong: the user’s e-mail provider is a trusted third party. Pwn the mail provider (or even something on the mail path), pwn the user.

                                                                                                          The same goes for the mail provider being down e.g. for maintenance.

                                                                                                          1. 5

                                                                                                            Oops, thanks for the correction. It should be third-party but decentralized. (Ignoring the case of running your own mail server or telephone network.)

                                                                                                            1. 3

                                                                                                              Even if it is your server, it might not be an encrypted email, or in your DC, or delivered to the right server first… easy enough to say you are ignoring those cases but they may not be that far from the norm.

                                                                                                              1. 1

                                                                                                                That is what I meant. Let me try to rephrase to be more clear:

                                                                                                                I will mark as third-party because the case of running your own tightly controlled mail server is rare and still depends on secure sending from the source server so there are likely third-parties involved.

                                                                                                              2. 2

                                                                                                                I don’t really see why you assume a login link is “leak proof” either - at its core a “login link” is just a one time password delivered out of band (via email).

                                                                                                                The server needs to (a) generate and send you the OTP and (b) have some way to verify that the OTP is correct, so any “leaking” issues that passwords have, can apply to email links just the same, and due to the “if someone can fuck it up they will” law of computer security, I wouldn’t want to assume that people implementing email links are treating them anywhere nearly as “securely” as we expect people to treat regular passwords.

                                                                                                                1. 1

                                                                                                                  Yeah. I thought about this for a while. But I came to the conclusion that these hopefully short-lived OTPs didn’t really have the same impact as leaking a long-lived password.

                                                                                                                  Really everything is a spectrum and I had to pick yes or no. For a well implemented system it seemed more like a yes. Although I agree it is easy to get wrong.

                                                                                                                  1. 2

                                                                                                                    Saying an OTP cannot leak means you’re assuming the the best case scenario for OTP handling, while simultaneously assuming the worst case scenario for password storage.

                                                                                                                    If we consider “might” to mean “has the theoretical possibility, depending on implementation and other factors”, then realistically they both can potentially leak, and I think if you’re going to assume long lived password storage might leak, you have to assume OTP’s might leak too.

                                                                                                                    Yes, an OTP should be short lived, but we don’t know that it’s implemented that way, and that it’s short lived (assuming it’s actually expunged after a short time) is almost certainly going to give people a false sense of security that a less secure implementation is “safe” (i.e. storing them in clear text).

                                                                                                                    Yes an OTP shouldn’t be vulnerable to variants of replay attacks (i.e. a login link explicitly has to be a GET request, which means the OTP is potentially recorded in a lot of intermediate places), but we don’t know that it’s implemented that way.

                                                                                                                    Yes both should be hashed strongly in a manner that nullifies the dangers if the hashes themselves were to leak, but we don’t know that it’s implemented that way.

                                                                                                              3. 4

                                                                                                                The same goes for password recovery emails, FWIW.

                                                                                                                1. 18

                                                                                                                  I think that is the best argument in favour of email login links. “We are going to allow auth resets by email anyways, may as well make that the only way in.”

                                                                                                                  1. 5

                                                                                                                    Password recovery emails aren’t used anywhere nearly as often as a login email may be needed though, so downtime/maintenance issues are much less of a concern, as is non-instant delivery.

                                                                                                                    Additionally, there’s a security aspect here: if your mailbox is compromised and someone uses it to retrieve a login link, you won’t necessarily know, ever.

                                                                                                                    If your mailbox is compromised and someone uses it to reset your password, you’ll know the next time you try to login.

                                                                                                                1. 2

                                                                                                                  I think you made the case for WebAuthn just look better :) Regardless very nice analysis!

                                                                                                                  1. 5

                                                                                                                    Care to elaborate?

                                                                                                                    I think the major downsides of WebAuthn for me are:

                                                                                                                    1. A “regular user” can’t backup or transfer their credentials. Logging with a new device is a disaster.
                                                                                                                    2. IIUC no mutual authentication support. (Although maybe this can be added as I would like to do with passwords).
                                                                                                                    1. 2

                                                                                                                      A “regular user” can’t backup or transfer their credentials. Logging with a new device is a disaster.

                                                                                                                      That’s not currently easy but it’s not a limitation of WebAuthn. Logging in with a new device for most WebAuthn services I’ve used is to log in with one of my existing devices and press the ‘allow this new device’ button. You could make that completely transparent with some QR-code magic: If you’re logged in, show a QR code contains a single-use URL that adds a new device to the logged-in account. If you’re not logged in, show a QR code that contains a single-use URL that requires you to log in with an already-authorised device and signs you in.

                                                                                                                      The flow that you want is not to transfer your credentials because your credentials should always live in a tamper-proof hardware token (TPM, U2F device, enclave on the phone, whatever), it’s to transfer authorisation.

                                                                                                                      The big problem with WebAuthn is in the opposite direction: revocation. If my phone is stolen, hopefully its credential store is protected by a biometrics and something that rate limits (with exponential back-off) attempts to unlock the device with spoofed biometrics, so it’s probably fine, but there’s no good way (yet) of saying ‘revoke my phone’s WebAuthn credentials for every single service that I can sign on with’.

                                                                                                                      IIUC no mutual authentication support. (Although maybe this can be added as I would like to do with passwords).

                                                                                                                      As I understand the WebAuthn flow, this is not quite true. With a password, if I provide my password to a phishing server, it is compromised. With WebAuthn, there is a challenge-response protocol. A phishing site must intercept a valid challenge from the site that it is trying to attack and ask me to sign it. My signature will use a key derived from a local secret and the domain that I am talking to. If I get a challenge that originated from example.com but that was forwarded by evilxample.com that is trying to phish me, then I will sign it with a key that is SomeKDF(evilxample.com, myDeviceSecret). The signature will then not match what example.com expects. The MITM attempt then fails. I don’t get a direct failure indication, but the phishing site can’t then show me anything that I expect to see because it has failed to log in to the site that it is trying to MITM my connection to.

                                                                                                                      Note that I already get the authentication in the opposite direction from TLS, the problem is not at the protocol level, it’s at the UI level: users can’t tell the difference between example.com and example.phish.com. WebAuthn should at least mean that if they log into example.phish.com, the intercepted credentials can’t be used to log into example.com.

                                                                                                                      1. 5

                                                                                                                        The flow that you want is not to transfer your credentials because your credentials should always live in a tamper-proof hardware token (TPM, U2F device, enclave on the phone, whatever), it’s to transfer authorisation.

                                                                                                                        The problem here is that it is a disaster to transfer all of these authorizations when you get a new device. And if you miss one before getting rid of your old devices you are now locked out of your accounts. (Unless there is backup authorization options of course.) This is why I only use 2FA on a tiny number of services. Managing which keys are enrolled in which services is a disaster even for a single-digit number of services. I can’t imagine a word where I had to do this for every login.

                                                                                                                        That is why the password is nice. I can sync my whole list of passwords or even write them down without needing to involve every service.

                                                                                                                        … With WebAuthn, there is a challenge-response protocol …

                                                                                                                        I think you are confusing phishing resistance with mutual authentication.

                                                                                                                        Note that I already get the authentication in the opposite direction from TLS

                                                                                                                        Yes, I agree that for most use cases one direction of auth provided by TLS and one direction of auth provided by whatever else is sufficient. However it is always nice not to need to rely on the PKI. Things are getting better with certificate transparency and pinning but the sad fact is still that for any site you haven’t visited before there are dozens of third-parties who have the authority to validate that connection. Reducing that set to only the site I want to take to is an absolutely huge improvement.

                                                                                                                        1. 1

                                                                                                                          This is why I only use 2FA on a tiny number of services. Managing which keys are enrolled in which services is a disaster even for a single-digit number of services. I can’t imagine a word where I had to do this for every login.

                                                                                                                          Because of this, I wrote my own TOTP app that syncs the 2FA with all my devices. Which is totally wrong if you ask any security expert – the TOTP should be one way.

                                                                                                                          Personally it’s an acceptable risk, because I ensure I don’t sync the passwords via the same mechanisms as the TOTP. If one sync is hacked, it doesn’t affect both parts of the 2FA.

                                                                                                                          1. 4

                                                                                                                            This is why I hate 2FA with TOTP, it loads most the security obligations to the user (while having the server store the shared secret in clear text).

                                                                                                                            1. 2

                                                                                                                              I do the same. All of the OTP secrets go into my password manager.

                                                                                                                              Most sites don’t even support multiple OTP devices and there is no way I’m getting locked out of sites because my phone was broken or stolen.

                                                                                                                        2. 1

                                                                                                                          What do you mean by mutual authentication? Is this not another word for “unphishable”?

                                                                                                                          1. 3

                                                                                                                            They are related and I think mutual authentication probably implies unphishable but not the other way around.

                                                                                                                            For example if I use a password manager I am basically unphisable because it doesn’t fill the password into the wrong site. But mutual authentication provides stronger validation of the site.

                                                                                                                            For example if you are using WebAuthn I think a phishing site can pretend to log you in without knowing anything about you, then ask for sensitive info. For example you go to g00gle.com, tap your authenticator and then it asks you for sensitive information. It seems legit because you just logged in. With mutual authentication unless the site actually knows who you are and has per-exchanged credentials the browser will fail to perform the authentication and let you know. No website can pretend that you have an account.

                                                                                                                            Of course getting UX that makes this obvious to the average user is definitely not easy. Maybe we need a new padlock that shows that you are using a password-authenticated connection. And of course we need to make sure that they can’t make a “signup” form look like a “login” form and get that lock for a new account.

                                                                                                                            1. 1

                                                                                                                              With this explanation I believe The no on mutual authentication for TLS client auth is wrong. TLS client auth sends a signed record of the Key exchange till the auth message. So yes the client don’t directly checks the server cert, but it isn’t a problem because the auth message can only used to auth this exact session. So even if you trick the user to auth against g00gle.com with the key for google.com the attacker can’t do anything with it.

                                                                                                                              1. 1

                                                                                                                                I’m not sure, because the browser prompts you in its sacred area, and also if the domain is new it must go through an enrollment phase which looks distinctly different.

                                                                                                                                1. 3

                                                                                                                                  I tested and this doesn’t appear to be the case. The following code seems to result in a “login” dialog on any site when clicking the button. At lest for my simple stateless security key (Yubikey Neo 4 IIRC). The request does actually fail but at least on Firefox there is no user-visible indication of this. Maybe this UX could be added and at least the site would need to know a valid key for the user.

                                                                                                                                  <button id=b>Click me</button>
                                                                                                                                  <script>
                                                                                                                                  function rand(len) {
                                                                                                                                    let r = new Uint8Array(len);
                                                                                                                                    window.crypto.getRandomValues(r);
                                                                                                                                    return r;
                                                                                                                                  }
                                                                                                                                  
                                                                                                                                  b.onclick = function() {
                                                                                                                                    navigator.credentials.get({
                                                                                                                                      publicKey: {
                                                                                                                                        challenge: rand(32),
                                                                                                                                  	  allowCredentials: [{
                                                                                                                                          type: "public-key",
                                                                                                                                          id: rand(64),
                                                                                                                                        }],
                                                                                                                                      },
                                                                                                                                    })
                                                                                                                                  };
                                                                                                                                  </script>
                                                                                                                                  

                                                                                                                                  Live demo: https://cdpn.io/kevincox-ca/fullpage/rNpvGNy

                                                                                                                                  1. 3

                                                                                                                                    The request does actually fail but at least on Firefox there is no user-visible indication of this

                                                                                                                                    I was able to verify this claim, indeed Firefox doesn’t show any indication of it failing. However if I test this in Chromium it does give a good error! Namely

                                                                                                                                    Try a different security key. You’re using a security key that’s not registered with this website

                                                                                                                                    Tracking this down, I figured out why Firefox doesn’t do the same. It’s caused by Firefox only implementing CTAP version 1 (aka U2F), which can’t detect this. CTAP 2 however does support that, and Chromium implements that. You can see how it works by searching for CTAP2_ERR_NO_CREDENTIALS in the CTAP2 specification. See this bugzilla issue for the status of CTAP2 in Firefox. This also prevents username-less login from working in Firefox at the moment.

                                                                                                                                    Given that, I think your mutual authentication criteria on webauthn can be dismissed. Though we might have to wait a bit longer for Firefox to catch up, but the specification is already stable.

                                                                                                                                    1. 1

                                                                                                                                      It still isn’t clear to me exactly how unguessable these keys are expected to be. I guess these should be basically completely random since they are public keys of a certificate pair but it also depends if they are site-specific (I think they are supposed to be) otherwise you do a 2-part phishing where you get the key ID from one site and use it on a different one.

                                                                                                                                      But it sounds like you are right. I can probably mark WebAuthn as mutual authentication.

                                                                                                                                      1. 2

                                                                                                                                        Guessing public keys isn’t possible, but it also doesn’t matter, because servers don’t have a way to check whether a public key is valid. If you mean credential IDs instead, that’s also unguessable, check the linked specification. And yeah public keys and credential ID’s are site-specific. Webauthn takes great care to make it hard to track users between multiple websites. Check out this section of the specification which goes into detail about this concern.

                                                                                                                                        If you have other questions or concerns about webauthn, I’d love to help. I spent a while with this specification (I worked on a server implementation) so I know it pretty well by now.

                                                                                                                                        1. 2

                                                                                                                                          Thanks for the info! I did find the spec really hard to navigate and maybe skimmed it faster than I should have. I’ve updated the post to say that WebAuthn provides mutual authentication. It maybe isn’t quite as strong as a TLS connection witk PAKE but I think it is close enough that for a simple comparison like in this post saying ✅ is more correct than ❌ and not really worth a footnote for the details.

                                                                                                                          2. 1

                                                                                                                            [removed accidental duplicate]