No matter how many times Go people try to gaslight me, I will not accept this approach to error-handling as anything approaching good. Here’s why:
Go’s philosophy regarding error handling forces developers to incorporate errors as first class citizens of most functions they write.
[…]
Most linters or IDEs will catch that you’re ignoring an error, and it will certaintly be visible to your teammates during code review.
Why must you rely on a linter or IDE to catch this mistake? Because the compiler doesn’t care if you do this.
If you care about correctness, you should want a compiler that considers handling errors part of its purview. This approach is no better than a dynamic language.
The fact that the compiler doesn’t catch it when you ignore an error return has definitely bitten me before. doTheThing() on its own looks like a perfectly innocent line of code, and the compiler won’t even warn on it, but it might be swallowing an error.
I learned that the compiler doesn’t treat unused function results as errors while debugging a bug in production; an operation which failed was treated as if it succeeded and therefore wasn’t re-tried as it should. I had been programming in Go for many years at that point, but it had never occurred to me that silently swallowing an error in Go could possibly be so easy as just calling a function in the normal way. I had always done _ = doTheThing() if I needed to ignore an error, out of the assumption that of course unused error returns is a compile error.
Because errors aren’t special to the Go compiler, and Go doesn’t yell at you if you ignore any return value. It’s probably not the most ideal design decision, but in practice it’s not really a problem. Most functions return something that you have to handle, so when you see a naked function call it stands out like a sore thumb. I obviously don’t have empirical evidence, but in my decade and a half of using Go collaboratively, this has never been a real pain point whether with junior developers or otherwise. It seems like it mostly chafes people who already had strong negative feelings toward Go.
Is there a serious argument behind the sarcasm as to how this is comparable to array bounds checks? Do you have any data about the vulnerabilities that have arisen in Go due to unhandled errors?
Because the programmer made an intentional decision to ignore the error. It won’t let you call a function that returns an error with out assigning it to something, that would be a compile time error. If the programmer decides to ignore it, that’s on the programmer (and so beware 3rd party code).
Now perhaps it might be a good idea for the compiler to insert code when assigned to _ that panics if the result is non-nil. Doesn’t really help at runtime, but at least it would fail loudly so they could be found.
I’ve spent my own share of time tracking down bugs because something appeared to be working but the error/exception was swallowed somewhere without a trace.
huh… til. I always assumed you needed to use the result, probably because of single vs multiple returns needing both being a compile time error. Thanks.
Because the programmer made an intentional decision to ignore the error.
f.Write(s)
is not an intentional decision to ignore the error. Neither is
_, err := f.Write(s)
Yet the go compiler will never flag the first one, and may not flag the second one depending on err being used elsewhere in the scope (e.g. in the unheard of case where you have two different possibly error-ing calls in the same scope and you check the other one).
Yet the go compiler will never flag the first one, and may not flag the second one depending on err being used elsewhere in the scope (e.g. in the unheard of case where you have two different possibly error-ing calls in the same scope and you check the other one).
_, err := f.Write(s) is a compiler error if err already exists (no new variables on left side of :=), and if err doesn’t already exist and you aren’t handling it, you get a different error (declared and not used: err). I think you would have to assign a new variable t, err := f.Write(s) and then take care to handle t in order to silently ignore the err, but yeah, with some work you can get Go to silently swallow it in the variable declaration case.
Because they couldn’t be arsed to add this in v0, and they can’t be arsed to work on it for cmd/vet, and there are third-party linters which do it, so it’s all good. Hopefully you don’t suffer from unknown unknowns and you know you should use one of these linters before you get bit, and they don’t get abandoned.
(TBF you need both that and errcheck, because the unused store one can’t catch ignoring return values entirely).
I don’t really care. Generally speaking, I would expect compilers to either warn or error on an implicitly swallowed error. The Go team could fix this issue by either adding warnings for this case specifically (going back on their decision to avoid warnings), or by making it a compile error, I don’t care which.
This is slightly more nuanced. Go project ships both go build and go vet. go vet is an isomorphic to how Rust handles warnings (that warnings apply to you, not your dependencies).
So there would be nothing wrong per se if this was caught by go vet and not go build.
What is the issue though, is that this isn’t caught by first-party go vet, and requires third party errcheck.
Meh plenty of code bases don’t regularly run go vet. This is a critical enough issue that it should be made apparent as part of any normal build, either as a warning or an error.
If you care about correctness, you should want a compiler that considers handling errors part of its purview. This approach is no better than a dynamic language.
I agree with you that it’s better for this to be a compiler error, but (1) I’ll never understand why this is such a big deal–I’m sure it’s caused bugs, but I don’t think I’ve ever seen one in the dozen or so years of using Go and (2) I don’t think many dynamic languages have tooling that could catch unhandled errors so I don’t really understand the “no better than a dynamic language” claim. I also suspect that the people who say good things about Go’s error handling are making a comparison to exceptions in other languages rather than to Rust’s approach to errors-as-values (which has its own flaws–no one has devised a satisfactory error handling system as far as I’m aware).
The fact that these bugs seem so rare and that the mitigation seems so trivial makes me feel like this is (yet another) big nothingburger.
The most common response to my critique of Go’s error-handling is always some variation on “this never happens”, which I also do not accept because I have seen this happen. In production. So good for you, if you have not; but I know from practice this is an issue of concern.
I don’t think many dynamic languages have tooling that could catch unhandled errors so I don’t really understand the “no better than a dynamic language” claim.
Relying on the programmer to comprehensively test inputs imperatively in a million little checks at runtime is how dynamic languages handle errors. This is how Go approached error-handling, with the added indignity of unnecessary verbosity. At least in Ruby you can write single-line guard clauses.
I don’t really follow your dismissal of Rust since you didn’t actually make an argument, but personally I consider Rust’s Option type the gold standard of error-handling so far. The type system forces you to deal with the possiblity of failure in order to access the inner value. This is objectively better at preventing “trivial” errors than what Go provides.
The most common response to my critique of Go’s error-handling is always some variation on “this never happens”, which I also do not accept because I have seen this happen. In production. So good for you, if you have not; but I know from practice this is an issue of concern.
I’m sure it has happened before, even in production. I think most places run linters in CI which default to checking errors, and I suspect if someone wasn’t doing this and experienced a bug in production, they would just turn on the linter and move on with life. Something so exceedingly rare and so easily mitigated does not meet my threshold for “issue of concern”.
Relying on the programmer to comprehensively test inputs imperatively in a million little checks at runtime is how dynamic languages handle errors
That’s how all languages handle runtime errors. You can’t handle them at compile time. But your original criticism was that Go is no better than a dynamic language with respect to detecting unhandled errors, which seems untrue to me because I’m not aware of any dynamic languages with these kinds of linters. Even if such a linter exists for some dynamic language, I’m skeptical that they’re so widely used that it merits elevating the entire category of dynamic languages.
I don’t really follow your dismissal of Rust since you didn’t actually make an argument, but personally I consider Rust’s Option type the gold standard of error-handling so far. The type system forces you to deal with the possiblity of failure in order to access the inner value. This is objectively better at preventing “trivial” errors than what Go provides.
I didn’t dismiss Rust, I was suggesting that you may have mistaken the article as some sort of criticism of Rust’s error handling. But I will happily register complaints with Rust’s error handling as well–while it does force you to check errors and is strictly better than Go in that regard, this is mostly a theoretical victory insofar as these sorts of bugs are exceedingly rare in Go even without strict enforcement, and Rust makes you choose between the verbosity of managing your own error types, debugging macro expansion errors from crates like thiserror, or punting altogether and doing the bare minimum to provide recoverable error information. I have plenty of criticism for Go’s approach to error handling, but pushing everything into an error interface and switching on the dynamic type gets the job done.
For my money, Rust has the better theoretical approach and Go has the better practical approach, and I think both of them could be significantly improved. They’re both the best I’m aware of, and yet it’s so easy for me to imagine something better (automatic stack trace annotations, capturing and formatting relevant context variables, etc). Neither of them seems so much better in relative or absolute terms that their proponents should express superiority or derision.
Fair enough. It’s a pity things like this are so difficult to answer empirically, and we must rely on our experiences. I am very curious how many orgs are bitten by this and how frequently.
Enabling a linter is different from doing “a million little checks at runtime”. This behaviour is not standard because you can use Go for many reasons other than writing production-grade services, and you don’t want to clutter your terminal with unchecked error warnings.
I admit that it would be better if this behaviour were part of go vet rather than an external linter.
The strange behaviour here is not “Go people are trying to gaslight me”, but people like you coming and complaining about Go’s error handling when you have no interest in the language at all.
Enabling a linter is different from doing “a million little checks at runtime”.
You can’t lint your way out of this problem. The Go type system is simply not good enough to encapsulate your program’s invarients, so even if your inputs pass a type check you still must write lots of imperative checks to ensure correctness.
Needing to do this ad-hoc is strictly less safe than relying on the type system to check this for you. err checks are simply one example of this much larger weakness in the language.
The strange behaviour here is not “Go people are trying to gaslight me”, but people like you coming and complaining about Go’s error handling when you have no interest in the language at all.
I have to work with it professionally, so I absolutely do have an interest in this. And I wouldn’t feel the need to develop this critique of it publicly if there weren’t a constant drip feed of stories telling me how awesome this obviously poor feature is.
Your views about how bad Go’s type system is are obviously not supported by the facts, otherwise Go programs would be full of bugs (or full of minuscule imperative checks) with respect to your_favourite_language.
I understand your point about being forced to use a tool in your $job that you don’t like, that happened to me with Java, my best advice to you is to just change $job instead of complaining under unrelated discussions.
Your views about how bad Go’s type system is are obviously not supported by the facts, otherwise Go programs would be full of bugs (or full of minuscule imperative checks)
They are full of bugs, and they are full of miniscule imperative checks. The verbosity of all the if err != nil checks is one of the first things people notice. Invoke “the facts” without bringing any isn’t meaningfully different than subjective opinion.
Your comments amount to “shut up and go away” and I refuse. To publish a blog post celebrating a language feature, and to surface it on a site of professionals, is to invite comment and critique. I am doing this, and I am being constructive by articulating specific downsides to this language decision and its impacts. This is relevant information that people use to evaluate languages and should be part of the conversation.
If if err != nil checks are the “minuscle imperative checks” you complain about, I have no problem with that.
That you have “facts” about Go programs having worse technical quality (and bug count) than any other language I seriously doubt, at most you have anecdotes.
And the only anecdote you’ve been able to come up with so far is that you’ve found “production bugs” caused by unchecked errors that can be fixed by a linter. Being constructive would mean indicating how the language should change to address your perceived problem, not implying that the entire language should be thrown out the window. If that’s how you feel, just avoid commenting on random Go post.
Yeah, I have seen it happen maybe twice in eight years of using Go professionally, but I have seen it complained about in online comment sections countless times. :-)
If I were making a new language today, I wouldn’t copy Go’s error handling. It would probably look more like Zig. But I also don’t find it to be a source of bugs in practice.
Everyone who has mastered a language builds up muscle memory of how to avoid the Bad Parts. Every language has them. This is not dispositive to the question of whether a particular design is good or not.
Not seeing a problem as a bug in production doesn’t tell you much. It usually just means that the developers spent more writing tests or doing manual testing - and this is just not visible to you. The better the compiler and type-system, the fewer tests you need for the same quality.
Not seeing a problem as a bug in production doesn’t tell you much
Agreed, but I wasn’t talking about just production–I don’t recall seeing a bug like this in any environment, at any stage.
It usually just means that the developers spent more writing tests or doing manual testing - and this is just not visible to you.
In a lot of cases I am the developer, or I’m working closely with junior developers, so it is visible to me.
The better the compiler and type-system, the fewer tests you need for the same quality.
Of course with Go we don’t need to write tests for unhandled errors any more than with Rust, we just use a linter. And even when static analysis isn’t an option, I disagree with the logic that writing tests is always slower. Not all static analysis is equal, and in many cases it’s not cheap from a developer velocity perspective. Checking for errors is very cheap from a developer velocity perspective, but pacifying the borrow checker is not. In many cases, you can write a test or two in the time it would take to satisfy rustc and in some cases I’ve even introduced bugs precisely because my attention was so focused on the borrow checker and not on the domain problem (these were bugs in a rewrite from an existing Go application which didn’t have the bugs to begin with despite not having the hindsight benefit that the Rust rewrite enjoyed). I’m not saying Rust is worse or static analysis is bad, but that the logic that more static analysis necessarily improves quality or velocity is overly simplistic, IMHO.
Of course with Go we don’t need to write tests for unhandled errors any more than with Rust, we just use a linter.
I just want to emphasize that It’s not the same thing - as you also hint to in the next sentence.
I disagree with the logic that writing tests is always slower.
I didn’t say that writing tests is always slower or that using the compiler to catch these things is necessarily always better. I’m not a Rust developer btw. and Rust’s errorhandling is absolutely not the current gold-standard by my own judgement.
I just want to emphasize that It’s not the same thing - as you also hint to in the next sentence.
It kind of is the same thing: static analysis. The only difference is that the static analysis is broken out into two tools instead of one, so slightly more care needs to be taken to ensure the linter is run in CI or locally or wherever appropriate. To be clear, I think Rust is strictly better for having it in the compiler–I mostly just disagree with the implications in this thread that if the compiler isn’t doing the static analysis then the situation is no better than a dynamic language.
I didn’t say that writing tests is always slower or that using the compiler to catch these things is necessarily always better.
What did you mean when you said “It usually just means that the developers spent more writing tests or doing manual testing … The better the compiler and type-system, the fewer tests you need for the same quality.” if not an argument about more rigorous static analysis saving development time? Are we just disagreeing about “always”?
I mostly just disagree with the implications in this thread that if the compiler isn’t doing the static analysis then the situation is no better than a dynamic language.
Ah I see - that is indeed an exaggeration that I don’t share.
Are we just disagreeing about “always”?
First that, but it also in general has other disadvantages. For instance, writing tests or doing manual tests is often easy to do. Learning how to deal with a complex time system is not. Go was specifically created to get people to contribute fast.
Just one example that shows that it’s not so easy to decide which way is more productive.
Swallowing errors is the very worst option there is. Even segfaulting is better, you know at least something is up in that case.
Dynamic languages usually just throw an exception and those have way better behavior (you can’t forget, an empty catch is a deliberate sign to ignore an error, not an implicit one like with go), at least some handler further up will log something and more importantly the local block that experienced the error case won’t just continue executing as if nothing happened.
On a multi-user machine /tmp is shared. For downloads I would prefer some world-unreadable directory. Though I don’t even know what are the recommendations for things like that nowadays. On Linux there’s /run/user/<uid>, maybe this one? Does xdg have any dir for “user’s temp files”?
I know you’re replying to the last question of the parent comment, but, for downloads, if using XDG user directories, might as well use XDG_DOWNLOAD_DIR.
All my home systems are configured with NixOS and have users for everyone that might need to use them, so kind of yes.
But my PoV here is about doing things right w.r.t. security “and just in case”. If you have a system daemon that runs as a separate user, and it’s compromised, you don’t want it to snoop/modify your downloads etc.
I think that’s a bad place for downloads, since the spec says
Applications should use this directory for communication and synchronization purposes and should not place larger files in it, since it might reside in runtime memory and cannot necessarily be swapped out to disk.
Practically speaking, on many distros this directory is a tmpfs a few GBs or so in size, so you do actually run into the size limit, it’s not just a theoretical thing.
On my client machines, I create ~/tmp/ and use that as a location to download, unzip source bundles when I only need a single file, throw a clone of a project for a comparison. I have it set up as a ram disk with a 7 day retention so it works more or less like tmp. Additionally, I set the noexec bit on the directory which makes it a better place for stuff that I just want to dig through.
A huge amount of the stuff that I bring onto my box comes through the ~/tmp/ folder first, and after looking at it and deciding I need it for longer I will move things to a disk some where.
I realized at some point that there’s a lot of stuff I know I’d toss in 2 hours but I also didn’t want to put it in actual tmp because it’s shared and I don’t want to mess with permissions or deleting it later.
I’m so tired of people shitting on curl | sh scripts. An installer is just as dangerous! A deb/rpm package is just as dangerous! The binary you are trying to install is just as dangerous!
Yes, you should be careful what code you run on your computer, but it’s never the stuff you can see that should make you nervous, it’s the stuff you haven’t seen yet or can’t see.
Good post, thank you for acknowledging what so many seem to ignore.
To be fair, in the non-malicious case it is easier to make uninstall work sanely with deb/rpm, and there are like two or three checklist items for curl|sh that are not needed for a downloadable installer.
The #1 problem with curl|sh is the lack of artifacts for post-hoc investigation. The script never hits your disk, so there’s no chance of it being saved somewhere for later analysis.
An installer is just as dangerous! A deb/rpm package is just as dangerous! The binary you are trying to install is just as dangerous!
They can be, but it depends on the actual contents of those artifacts.
Example: Let’s say I download a .deb, and this package does not fetch any additional resources over the network. Then I can now for sure what’s in there, and analyze every part of it.
The same is not true for a resource that I fetch using Curl. The server could give me a different artifact for every new request. Most servers don’t, but they could.
It could try. But if you run apt install gazonk, APT will download the package once. And AFAIK, APT will verify the checksum of the downloaded artifact.
The equivalent of curl foo.bar/baz.sh | sh is wget foo.bar/baz.deb && dpkg -i baz.deb not apt get install baz. It’s a method for installing software not packaged by your distro.
they imported the data reported in the blog post as two numpy/scipy vectors, then ran a student’s t-test to test the hypothesis that the two distributions the two vectors were sampled from are different. (docs on ttest_indhere).
Technically this is bad statistics, because you’d want to specify your confidence significance level ahead of time (typical value is 5%, which roughly corresponds to a difference of two standard deviations between the two distributions). Essentially, this test is saying that if the two variables came from the same underlying distribution, you would expect to see this data no more than 0.07% of the time. I suspect that the default for scipy uses a 5% confidence interval significance level, so statistically you can say that this data comes from the same distribution (i.e. the latency is the same between X11 and Wayland) no more than 5% of the time.
Edit: the above is sorta wrong. You don’t specify a significance level in the t-test itself, but you should specify it before performing the test, otherwise it’s hard to interpret the results based on your null hypothesis. So if we state the null hypothesis as “I expect that if X11 has the same latency as Wayland, and I measure the latency 20 times in both, I expect to get a different result no more than one time out of twenty”, then the above test is coherent and we can reject the null hypothesis.
Reading the docs some more, ~this might also be a bad test choice because itequal_var` says to not assume the underlying distributions have the same variance, which is reasonable I don’t think we have any reason to suspect a priori (i.e. Wayland may have more lag but it may be more consistent than X11, or vice-versa).
It means that if x11 andwayland are actually drawn from identical distributions, then probability of getting two sets of samples with a statistic less than or equal to what we got above is 0.06%.
In other words, the difference would be improbable if x11 and wayland were drawn from identical distributions.
The author has taken some latency measurements under x11 and Wayland. The measurements are all over the place, so eyeballing whether the sets are meaningfully or just noisy is difficult.
The poster above used Welch’s T-Test, a standard statistical test, to determine if the sets are statistically different or not. The low p value indicates that it is likely that the two sets of measurements are drawn from different distributions. Therefore the observed latencies do systematically differ between x11 and Wayland in this experiment.
There are disputes about the answer to that, but according to most theoretical statisticians (including Fisher) and to me, it would indicate that there’s probably something wrong with the statistical model you’re using. [source: closely related to the topic of my PhD and my papers]
Sure. Please DM me and tell me how much detail you want! I know you’re smart (tridactyl) but I don’t know how interested you are in disputes in theoretical statistics.
Or in case you or anyone else doesn’t want to DM me, you can read R.A. Fisher’s original writings on this, and he was enormously influential (even when he was wrong).
That there’s not enough evidence to reject the null hypothesis. That is, we wouldn’t be able to say whether x11 and wayland are drawn from an identical distribution or not.
I’ve found that Puppet can be far more expressive and easier to create constructs that are just plain difficult or ugly to do with the pseudo-DSL YAML used by Ansible and other tools. On the flip-side, unless you have a team with a “proper” developer background, the code-base can quickly spiral into an unmaintainable mess full of convoluted inheritance and dependency anti-patterns. Ansible, for all it’s warts, is much more straightforward in that it’s more or less “do thing x, then do thing y in the order I specify” and is a lot easier for non-developers to pick up.
A more technical way of saying it, is that puppet describes a state, and ansible describes the process used to get to the state.
If you want a machine to be in a specific state, and stay in that state, puppet helps with that. If you just want to get a machine into a state, then destroy it if it deviates, ansible is a great tool. Especially if it’s simple to get it into the state and cheap to recreate.
I think it’s one reason that ansible was more widely used on cattle and puppet was more widely used on pets. Puppet even required certs and approval for nodes by default.
Ah, but don’t you know, they “aren’t” rolling their own crypto, per their FAQ.
Is Session rolling its own cryptography?
No, Session does not roll its own cryptography. Session uses Libsodium, a highly tested, widely used, and highly-regarded crypto library.
Libsodium is completely open-source.
heavily rolls eyes
I like libsodium. It’s a great library of cryptography algorithms.
It doesn’t come, batteries included, with a full protocol for end-to-end encryption built-in. And so anyone who uses libsodium for e2ee is necessarily rolling their own crypto.
I’ve only heard of Session in passing over the years. This result is not surprising.
i initially thought you were being overzealous, until i read the first paragraph of the article
using libsodium to implement the documented signal protocol, i think would be fine. it does have risks, and should have some review before relying on it, but i wouldn’t really call that “rolling your own crypto”. and having a clean-room re-implementation would probably be good for the ecosystem
…but that’s not what they’re doing. they’re implementing their own protocol, and a cursory glance at their reasoning suggests that they want a decentralized messenger and care about security as an afterthought. which would be fine if they branded it that way, and not as an alternative to signal
This may be a little off topic, but I dislike the phrase “don’t roll your own crypto”.
Don’t roll your own crypto is generally a term which in itself is very ambiguous.
I’ve seen this phrase being thrown around when people just use gnutls in C vs people implementing some hash algorithm themselves. One I find very valid while the other one is an area where I would just use libsodium.
There are so many layers in crypto where you can apply the phrase that I find refuting (their claims with) this phrase in itself is meaningless unless you know what the authors intended. In this case it may as well be claims regarding the resistance against certain side channel attacks.
I’ve always asked myself how I can identify the moment I arrive at a skill level where I’m allowed to “roll my own crypto” depending on each possible interpretation people are using.
There are so many layers in crypto where you can apply the phrase that I find refuting this phrase in itself is meaningless unless you know what the authors intended.
Absolutely. And the advice, taken to its logical extreme, would result in zero cryptography ever being developed.
It’s supposed to be along the same lines as advice given from lawyers to their kids that are considering a career in law. They say, “Don’t be a lawyer,” and if their kid isn’t dissuaded and can argue why they’d succeed as a lawyer, then maybe they should be one.
“Don’t roll your own crypto” is along the same lines. I currently do it professionally, but I also have a lot of oversight into the work I do to ensure it’s correct. Detailed specifications, machine-assisted proofs, peer review, you name it. I never start with code; I always start with “here’s why I’m doing this at all” (which includes closely related ideas and why they do not work for the problem I’m solving) and a threat model for my specific solution.
It can take months, or even years, to get a new cryptography design vetted and released with the appropriate amount of safety.
When it comes to cryptography protocol design, the greatest adversary is often your own ego.
I always read the advice as an incomplete sentence, which ends with “unless you know what you’re doing”, which is coincidentally like other safety advice, right? “This thing that you’re about to do is risky and dangerous unless you know how to do it, and in some cases, even if you do. Avoid doing it if you can. Exercise caution and care otherwise.” No?
I always viewed it as “don’t ship your own” - feel free to roll your own to play around with, but be cautious and get some review before putting it into production.
One piece of advice I’ve heard is: Before trying to build crypto, learn how to break crypto. Cryptopals is a good resource for that. It’s mindbending to learn about all the weird ways that crypto can fall apart.
I remember many moons ago that an expert in security and crypto actually published a list of cryptographic choices that should be your default. I wonder if this rings a bell to someone, it would be nice to recover that document, publish it here and see what this community would say in terms of freshen it up.
I might be wrong, but I think in the beginning the meaning of the phrase “don’t roll your own crypto” mean “do not try to come up with cryptographic algorithms on your own; use something tested and done by someone who know what they are doing”. I think the best way to describe what Soatok is putting forward is “don’t skip the default practices of security” or “don’t wave away cryptographic defaults in name of a watered down threat model”.
I think that’s a very reasonable concern. Particularly in light of the very first issue @soatok cites: the removal of PFS from the protocol. I’m on record as being skeptical of the “just use signal” advice that seems frequently deployed as a though-terminating measure in discussions about encrypted communication, but if I wanted to make something that was like signal but really a deniable honeypot, Session makes the same choices I would. It seems like a downgrade from signal in every important way.
Unrelated: the “Ad-blocker not detected” message at the bottom of the post made me laugh quite a bit. I use some tracker-blocking extensions (and browser configs) but I don’t run a real ad blocker in this browser. But many sites give me an “ad blocker detected” message and claim I need to turn off my non-existent ad blocker to support them. This site telling me I’m not running enough of one struck me as very funny.
Sure, its plausible.
But I find basically every time Soatok (or any security researcher) exposes any application that advertises itself as “secure/private” on the box, for their glaring bad practices, people (myself included) immediately go to “this is so stupid it has to be a honeypot”.
Are they all honeypots? (genuinely, maybe yes), or is it just stupidity?
I would posit stupidity. Previous honeypots that weren’t takeovers of server operators have been somewhat targeted: Anom required a buy-in of the phone (as evidence you’re a criminal), Playpen required you be a pedophile (or at least, hanging out online with pedophiles) to be caught in the net, Hansa was a drug market, etc. Creating a general-purpose backdoored app to en masse catch criminals seems to cast quite a wide net when the same arrest portfolio can probably be gathered by doing the same thing to Twitter DMs with a backdoor and a secret court order. I wouldn’t put it past law enforcement but it seems like a mega hassle vs. targeted honeypots and backdoors.
If it were a honeypot (or backdoor), it’s certainly too much hassle for legitimate law enforcement purposes like the ones you described. You’d want this for someone you couldn’t reach through normal court (even a secret rubberstamp like FISA) channels.
This would be more like something you’d use for getting information from a group operating under some legal regime that’s not friendly to you gathering that information. Getting it in place, then convincing the group you were interested in to migrate off, say, Telegram, might be one approach.
The interesting thing in this case (IMO) here is that the fork removes things that were:
Already implemented by an easy-to-reuse library
Not costly in terms of performance or cognitive overhead
Seemingly beneficial to the stated goals of their product
and without articulating the upside to their removal very persuasively. Sure, stupidity is always a possibility. But it feels more like they want to add some features that they don’t want to talk about. On the less-nefarious end of that spectrum, I could imagine that it is as simple as supporting devices that don’t work with the upstream, but that they don’t want to discuss in public. It’s also easy to imagine wanting to support some middle scanner-type box on a corporate network that PFS would break. But it could also be something like being able to read messages from a device where you can maintain passive traffic capture/analysis but can’t (or would prefer not to) maintain an ongoing endpoint compromise without detection. e.g. You have compromised a foreign telco and can pull long term key material off a device when its owner stays in your hotel, but you can’t or won’t leave anything running on there because the risk of detection would then be too high.
That’s just all speculation about when it might serve someone with an effectively unlimited budget to do something like this. Stupidity is certainly more common than such scenarios.
Only the first bit could charitably be attributed to “don’t roll your own crypto”. The rest was just obtuse idiocy or malevolence. Call the library-provided “encrypt this chunk with symmetric encryption using this key” then providing a public key.. that’s not about rolling your own crypto.
I haven’t followed Swift much recently, but the original Swift was simple because it punted on all of the hard problems. It had no mechanism for error handling (there were some small mods towards NSError). It had no support for concurrency and the everything-is-shared-mutable-state model made that really hard to fix (the structured concurrency and actor extensions are trying). It had no support for shared libraries (Swift’s shared library support now is really nicely designed and it’s the only modern language that I think has addressed this problem well, but that inevitably came with complexity).
For macOS development, Objective-C++ is actually a very nice language. C++ for local code that is specialised over strong types, Objective-C for late-bound interfaces that are exposed across module boundaries,
I have never used Objective C++, but from afar it sounds horrifying. Take a monstrously, ineffably complex programming language and another quite complex programming language and mash them together into an amalgamation. Can anybody hope to make sense of it?
Speaking as someone who used it for years, it actually works quite well. Obj-C is not very complex, and its OO stuff is pretty separable from the C part. Obj-C++ extends the C part.
I’ll also point out that modern Objective-C’s ARC (automatic reference counting) composes very well with C++’s object lifetime rules. In pure Objective-C with ARC,
is not allowed, because NSString* is an ARC pointer and the compiler needs to be able to reason about its initialisation and invalidation. The semantics of C structs don’t allow for that, which can make implementing custom data structures tricky: you either have to do it all in the context of Objective-C classes (which has certain overheads), use indirection, or you turn off ARC in the relevant source files and do manual reference counting.
The same code quoted above will compile on Objective-C++ however, because the ARC pointer is treated as having the usual set of C++ constructors, destructor, assignment operator, and so on - it’s not a POD (plain old data) type. This means the struct also gets implicit constructor/destructor/operator implementations when not explicitly implemented.
You can therefore shove pointers to Objective-C objects into all sorts of C++ data structures, that have been implemented without special Objective-C support, including the STL. It all composes rather nicely.
(A significant proportion of my work in recent years has been working on macOS device drivers and related apps, daemons/agents, etc.; I’ve mostly been using Objective-C++ on that, although I’ve recently introduced Rust in one part of such a project. My limited contact with Swift has been exceedingly frustrating, so I’ve avoided it where possible; it never appealed to me in the first place due to the reasons David mentioned, and the practicalities around extremely poor forward- and backwards-compatibility were a nightmare to deal with on a project where it was forced upon me.)
ARC certainly makes this easy, though prior to ARC I implemented a C++ smart pointer class for Objective-C references that did the relevant retain and release operation, which meant that this was already possible in Objective-C++ without ARC, it just became nicer because you didn’t need to use the smart pointer.
Definitely, you weren’t the only one to implement such a smart pointer class. I guess my point was more that modern Objective-C actually composes better with C++ libraries than it does with C libraries and data structures.
I also used it for years and we made Mac apps that spoke to hardware devices through the IOKit kernel driver API (which is in C++). It was indeed quite nice.
There is a non-trivial amount of ObjC++ in Firefox, too, or at least there was last time I checked. For TenFourFox I used ObjC++ as glue code to connect up libraries.
To add to what others have said: a lot of the pain in C++ comes from trying to do things that are easy in Objective-C, and vice versa. With C++, it’s easy to create rich types with no run-time overhead, but that create tight coupling. With Objective-C, you have late-bound interfaces everywhere, but avoiding dynamic dispatch is very hard / hacky. The combination means that you can completely avoid things like raw C pointers. You can used C++ collections inside a module, Objective-C ones across the boundaries.
You should give Obj-C a try, I think! It’s a surprisingly thin layer on top of C, giving a lot of bang for the buck for writing dynamic (but fast!) programs. It’s quite unique in that you have two extremes: C on the one hand, and a fully dynamic OO runtime (you can determine implementation at runtime). Overall syntax is outdated and weird (smalltalk influence), but it is still unmatched in that niche.
It’s not bad in practice. The use case is, you need Objective-C system frameworks but you can’t do without particular C++ libraries. You still spend the bulk of application code in one language.
I had a game that used Bullet physics in this way. I migrated most of the code to Swift after it was introduced, but I kept some Objective-C++ in order to keep Bullet. These days Swift has direct C++ interop, both for library use and for gradual migration of C++ projects to Swift.
I worked with making an iOS app at one point and I found while thinking in Lisp like patterns it seemed to get out my way if I wanted it to. But that is a beginner and greenfield perspective for sure.
I don’t think it sounds too bad, but I haven’t used it myself.
My understanding is it’s just the extra OOP bits from Objective-C overlayed on C++, similar to how it was overlayed on C in the first place. Basically just a second, independent object system. I understand why people wouldn’t like that, but it doesn’t sound too different than C++/CLI or using C++/JNI.
With the caveat that I’ve read their design docs, but not actually used it in anger:
They make a clear distinction between ABI-stable and ABI-unstable shapes of structures. Within a library, there are no ABI guarantees. At a library boundary, you have a choice whether you want to sacrifice some performance for the ability to change a layout later, or sacrifice flexibility for performance. This is a per-structure choice. Depending on the choice that you make, the compiler either lowers to something similar to Objective-C non-fragile ivars, or C struct fields.
They actually have a language notion of a library boundary. This means that generics can be statically reified within a library, but fall back to dynamic dispatch across library boundaries. Contrast this with C++ where templates either have to live in headers (and then end up copied in every compilation unit, including the implementation, and it’s a violation of the one-definition rule to link two libraries that use different versions of the same template) or they are private to a library. The Swift model gracefully falls back. Things may be faster inside a library, but they still work from outside, and the ABI doesn’t leak implementation details of the generics, only their interfaces.
but the original Swift was simple because it punted on all of the hard problems
Hmm…I’d say it was already incredibly complicated despite punting on a lot of hard problems and largely because it tried to codify “solutions” to non-problems into the language. Which never works, because reality, even code reality, is way too messy for that.
As an example, I wrote about the mess that is initialization back in 2014, so right after Swift apepared. There was some pushback from a prominent member of the Swift team saying that my goal of simplicity just wasn’t compatible with some of the goals they had. Hmm….
There was also a great rant by a prominent member of the community about Swift being just a mess of special cases upon special cases. I think he left the Apple ecosystem, and he’s not the only one. Alas I can’t find it and I don’t remember the name.
Anyway, I predicted back then that because they had so many language features for initialization it would never actually work out and there would be a renaissance of builder and factory method patterns and there would be even more language features for initialization. Five years later: hello SwiftUI! :-)
So the failure of Swift now isn’t surprising, the trajectory was already set in stone the day it launched and there wasn’t really much one could have done about it afterward…much less so since the same faulty reasoning that led to the initial problems was still present and guided later evolution.
Anyway, I predicted back then that because they had so many language features for initialization it would never actually work out and there would be a renaissance of builder and factory method patterns and there would be even more language features for initialization. Five years later: hello SwiftUI! :-)
I think this is an instance of correlation not being causation? My understanding is that the actual cause of SwiftUI is the successful design of Flutter (which gave raise to both SwiftUI and Kotlin Compose), and it is relatively orthogonal to language machinery.
Case in point, Kotlin’s initialization story is much more tame than Swit’s one (as it doesn’t try/doesn’t need to prove initialization safety statically), but it also converged on the essentially same design (or rather, vice-verse, IIRC Kotlin work in the area predate’s Swift’s).
Not to disagree with your wider point on constructors, which I agree with, just to point out that SwiftUI is not I think a particularly strong argument here.
I think you might want to have a look at the actual article. Swift introduced yet more special syntax for the part of SwiftUI that creates the view tree. So yet more language features for yet another way of constructing views^Wobjects^Wstructs.
I have read the articles! If I understand your argument correctly, it says that the fact that they needed to add new stuff to support Swift UI means that the original rules were inadequate. My counter-argument is even languages that don’t have Swift-style maze of initialization rule add special cases to support SwiftUI patterns. Ergo, adding stuff for SwiftUI is orthogonal to your normal way to initialize objects. In other words I claim that, in counter-factual where Swift doesn’t have complicated initialization rules and uses Java/Go style “everything is null to start with” or Rust/ML style “everything starts with all the parts specified”, it would have added more or less the same features still for SwiftUI.
The story is even more illustrative with Kotlin — it was specifically designed for DSLs like SwiftUI/Compose. The whole language, with its second-class-lambdas, extensions, and coming out-of-fashion implicit this, works towards that goal. And yet, when the actual UIs started to be implemented, it was quickly apparent that no one wants to write +button(), and a bit more of compiler special sauce is needed for nice surface syntax.
I must be a lousy communicator, because you seem to have misunderstood the article almost completely.
The point was not that Swift has the wrong initialization rules or too many of them. The point is, as it says in the title: “Remove features for greater power”. The many initialization rules are not the source of the problem, they are a symptom of the problem.
First rule of baking programming conventions into the language: Don’t do it!
The second rule of baking programming conventions into the language (experts only): Don’t do it yet!
The problem is trying to bake this stuff into the language. As a consequence, you get 30 pages of initialization rules. As a further consequence, those 30 pages will be forever insufficient.
So for me, the supposed counter-point you bring with Kotlin actually supports my point. You write:
it was specifically designed for DSLs like SwiftUI/Compose. The whole language, with its second-class-lambdas, extensions, and coming out-of-fashion implicit this, works towards that goal
So they baked a whole bunch of features into the language to support the DSL use case. What was the title of the blog post again?
“Remove features for greater power”
So they added a lot of features into the language especially for this use-case and it didn’t even work out for this particular use-case. Surprise surprise!
First rule of baking programming conventions into the language: Don’t do it!
The second rule of baking programming conventions into the language (experts only): Don’t do it yet!
I simply don’t think the static/compiler-oriented mindset is compatible with the sorts of things these languages are trying to do. You put way too much into the language/compiler, and you do it way too early.
Ruby has had a bunch of these kinds of frameworks, and as far as I know they did not require any changes to the language. Because Ruby had fewer but more flexible features to start with.
With Objective-S I seem to be violating that rule, because it certainly does put things into the language. Or at least seems to do so. What I am doing, however, is following the second rule: “don’t do it yet”. (With quite a bit of trepidation, because it is “experts only”).
And I am not actually baking all that much into the language. I am baking a bit of useful surface syntax and the associated metaobject-protocol into the language. What lies behind those metaobject protocols is quite flexible.
So far this appears to strike a good balance between providing some syntactic convenience and compiler support while not making the mistake of baking way too much into the language.
Indeed! I misunderstood your original comment as meaning that SwiftUI is a downstream consequence of initialization rules. I agree that both are rather the result of the lack of expressiveness, which doesn’t allow the user to “do it yourself” in userland code. The Kotlin example was exactly to illustrate that point.
There was also a great rant by a prominent member of the community about Swift being just a mess of special cases upon special cases. I think he left the Apple ecosystem, and he’s not the only one. Alas I can’t find it and I don’t remember the name.
Crucially, the vast majority of this is incidental complexity, not essential complexity. Swift is a crescendo of special cases stopping just short of the general; the result is complexity in the semantics, complexity in the behaviour (i.e. bugs), and complexity in use (i.e. workarounds).
How to fix:
What, then, should be removed? It’s probably too late to remove any of that, …
So I certainly wasn’t the only one who accurately predicted the current sad state of affairs. It was extremely obvious to a lot of people.
I am if the opinion that the only avenue for making a well rounded language that lasts is to put all of the skill points towards a small handful of skills (“fast”, “fast to compile”, “ease of dynamic linking”, “easy to use”, “easy to learn”, whatever) where each “skill” can be thought as a different axis in a graph. Go all in on a few niches. This helps with clarity of purpose for the project itself, a clear pitch for prospective users, and makes it possible to show the benefits for that niche. Without doing this the evolution of the project ends up rudderless, and adoption can stagnate either slowing down or killing the project. Once you’ve reached critical mass, fight for dear life not to abandon those hard earned early design decisions that allowed you to carve a niche, but go all-in on a handful of other axis as well. Rinse and repeat. A language can start as easy to use and once it has a clear base experience it can start trying to produce faster binary. Or it can start as fast to compile and afterwards try to produce optimized code. Or it can start producing super optimized code and later try to become faster to compile or easier to use. All the early design decisions will hamper the later pivot, and that’s ok! No language will ever be all things for everyone (barring some unrealistic investment in monstrous amounts of research, development and maintenance).
Swift had a clear initial view of what it wanted to accomplish, and did so successfully (I believe, don’t have personal experience with using Swift, nor do I keep up with its development). It is now on its expansion era, to allow for niches that previously were either difficult or impossible to cater to. It is a delicate process that can easily go haywire because of subtle interactions between completely unrelated features can make the result worse than the sum of its parts. But I can’t blame them for trying.
I could maybe argue that sync Rust had excellent “early design decisions that allowed you to carve a niche”, but the async related problems are now occupying a vast majority of the brain time of the lang teams. While I don’t think we’re heading to 217 keywords, I certainly feel uneasy seeing “keyword generics” and similar efforts.
The Acorn image editor. Great for people like me who doesn’t need Photoshop just to do some light image editing. To me, this is the rightful successor of Paint Shop Pro.
Obsidian. The UI is not perfect, but I still like it a lot. IMHO, the best note-taking application after Evernote jumped the shark. Everything is stored as plain Markdown files, so lock-in is not a concern.
The Fish shell. Just a pleasure to use. Easy to get started with. IMHO, it’s so much more convenient than Bash.
Preview. Love to have this application included in my OS. Currently using it to read Adam Langley’s A Tour of WebAuthn.
I completely agree. I dual boot a Linux-based system with macOS, and Preview.app is the only reason I ever reboot into macOS.
Usually I can get by with pdf-arranger and okular and sometimes zathura… but each has its own problem. I just want a PDF viewer that allows me to reorganize pages, annotate them, crop them visually (e.g. not specifying the cropbox as coordinates), and yeah.
Re Obsidian has no lock-in: I am using an iPhone and Ubuntu on the desktop. Do you know of anyway to use Dropbox (or equivalent) for syncing and still allow editing via their iOS app?
Since their own sync storage seems to be an USP of their paid tier, I guess I’m out of luck, but it would be very interesting to be proven wrong.
Not sure if I would call that lock-in – IMHO, lock-in is more about locking in the customer’s data without any reasonable export possibilities, which is not the case with Obsidian.
I can warmly recommend an Obsidian subscription, it works really well.
However, what you want to achieve might be possible without a subscription. I would search for ⟬obsidian syncthing iphone⟭ and ⟬obsidian git iphone⟭. Best of luck!
Sometimes one needs to vent, which is fine. But as a Hugo user I can’t relate to this. I was frustrated with Hugo before I understood how it works. After I got over that hurdle, all I need to do to write a blog post is to write some Markdown and then push to my Git host of choice. That workflow suits me very well. YMMV.
Confused. What am I supposed to under from that screenshot? Yes, fish is not zsh. I don’t see a single example of interactivity there. All I see are commands and their output.
Many fish users don’t use ls at all. Myself included. I can navigate and enter the directories interactively with tab completion. Ls becomes unnecessary.
The one thing @soatok does not address at all is the original use case that PGP was designed for, as opposed to all the other ones people started using it for in time between when it was designed and when better options came along: encrypted email.
The closest they get is saying to “use signal” for secure messaging. But I do not trust signal and none of the reasons relate to the need to give them my phone number. And while @soatok correctly points out that my phone number need no longer be revealed to people who want to communicate with me, it still looks like signal-the-org demands that I give it to them.
If you want people to stop using PGP, you need to provide an alternative for its original purpose. And an alternative that requires me to send all my stuff through one org, even if I didn’t need to give them my phone number (and it looks like I do) is not adequate.
I know S/MIME addresses some of the concerns one might have about PGP email, but I wouldn’t say it’s better on balance, and I don’t think S/MIME as implemented in those mailers that still implement it addresses any of the criticisms from this article in any way.
Every time I see an article like this, I get optimistic that there will be a reasonable proposal for encrypted email. There never is.
Aside from Signal, Soatok seems to address this use case (private messaging) from a different direction:
In the near future, I plan on developing end-to-end encryption for direct messages on the Fediverse (including Mastodon). This is what motivated my work on the Public Key Directory to begin with.
I see where they are coming from, but I find “don’t” an unacceptable answer to “how do I encrypt email?”
I am optimistic that work on the fediverse angle could yield an acceptable answer for email style asynchronous encrypted messaging, I’ll certainly be paying attention there.
But for now, I’m more comfortable trying to train a small number of collaborators on s/mime opsec than I am trusting signal. That’s because I know the pitfalls of s/mime better than I know the pitfalls of PGP. But I’d still be more comfortable trying to train a small number of collaborators on PGP opsec than trusting signal.
I see where they are coming from, but I find “don’t” an unacceptable answer to “how do I encrypt email?”
My understanding is that to get to the point of asking about whether you should encrypt email, you first have to make the fundamental mistake of trying to use email for secure messaging. And so the answer really is not “don’t encrypt your email”, it’s “don’t use email for secure messaging”.
All attempts to do encrypted “secure” email are basically just theater. There are so many weak links, technical and social, in the email infrastructure that you can personally set up the most bad-ass hard-ass ultra-mega-NSA-mil-spec XP++++ Pro grade encryption on your end, but any recipient who sees the plaintext can still quite easily quote-reply with the plaintext, or forward it, etc.
So PGP is more the cherry on top of the fecal sundae that is trying to make email into an acceptably secure messaging system. Which is why the answers always try to redirect you to an actual secure messaging system.
So PGP is more the cherry on top of the fecal sundae that is trying to make email into an acceptably secure messaging system. Which is why the answers always try to redirect you to an actual secure messaging system.
Should I say TLS is the cherry on top of the fecal sundae that is trying to make TCP an acceptably secure transmission system, just because there are a whole pile of ways that one party or another can terminate TLS in the wrong place and turn it into security theater? Back when I used to do pen tests, one particularly fun trick was to cause Firefox or IE to log its session keys to a place where I could grab them. It was at least as good as most of the email fun you could pull off, and got you a lot more data.
I opened the bug on the Evolution (GNOME email client) tracker about how it silently turned off encryption on encrypted email messages when you replied, even if you had a public key for everyone else on the thread. I am 99% certain it took over 10 years for that to get fixed. (I believe it was over 15, but I can’t find the notification from when it was finally closed on this laptop.)
I viscerally appreciate what you’re saying about email not being fit for this purpose.
But until we get a modern answer with the properties of email that doesn’t involve me trusting signal or whatsapp, it’s easier for me to work around the deficiencies of email in this regard.
I’ve tried twice now to set up a secure, closed messaging system, which is an easier version of what I’m saying I want here. I tried once using email infrastructure and once using XMPP infrastructure. Email was smoother. I’d like a better alternative.
But until we get a modern answer with the properties of email that doesn’t involve me trusting signal or whatsapp, it’s easier for me to work around the deficiencies of email in this regard.
Leaving your door wide open because you don’t like a particular brand of lock does not strike me as a great security stance.
But then I also don’t know why you have issues with “trust” with Signal, which is open source.
Isn’t the entire point of client-side end-to-end encryption that it shouldn’t matter what the server is doing?
And the whole binary thing is kind of endemic to the mobile ecosystem for regular users (i.e., people who have never heard of F-Droid outside of possibly some spicy fanfics).
Who cares if their server is doing key distribution? Surely you’re not trusting any server’s key distribution, federated or not, if you really care about message security?
but if the server were to provide the wrong public key, it would still be detectable right? can’t you still compare your public key to the public key that your contact has stored for you, if you are with them in person? wouldn’t that relieve you of the need to trust signal’s servers?
short of manually comparing keys, if you trust the client software on both ends, then exchanging messages and verifying in person that they were received should confirm that you have the right public keys on file, right? so that’s another way that you would not have to trust the server.
probably never. I just think we should be careful about saying “you can’t” exchange public keys without trusting Signal’s server. besides, is WhatsApp’s “key transparency” any different?
Yes. The security model of “key transparency’ is that you can build automation that detects when an errant public key has been issued, or if the public key received isn’t in the ledger. You don’t have to rely on squishy humans to do anything.
Additionally, it creates a disincentive for the kind of nation state actors that might perform such an active attack against a target of interest. Whereas we see mechanisms like National Security Letters with a built-in gag order, key transparency prevents stealth. In order to attempt to attack the other person’s communications, you have to first announce yourself to the network.
(This guarantee is much stronger with open source software and reproducible builds, of course.)
Nope. The PGP key server can lie about a lot of things. There is no audit trail (i.e., Merkle Tree proof of inclusion) to guarantee a time period. There are no witness co-signatures. There’s nothing.
@caleb, I’m currently visiting family for a much-needed vacation from technical work. Whether something exists or not is something you’re free to explore on your own. As I have no financial stake in anyone believing anything, I’m going to prioritize spending time with my nephews and drinking enough eggnog to make my physician give me the angry teacher stare when I return in January. I wish you the best.
I’m replying to you here (not the user you’re replying to, but I was interested when I saw you bump it), because it turns out that’s actually the max reply depth and so no-one can reply to you.
I thought the server was open source? (AGPL-3.0 license): https://github.com/signalapp/Signal-Server Seems to get actively published to, at the time of this writing the last commit was 5 hours ago.
I seem to remember some people have verified the client binaries for Android at least once. I can’t seem to find a link for it though, perhaps my memory is faulty.
The server is open source except for the spam filter, which is pretty reasonable. I believe all the other server-side components, particularly SGX enclave implementations, are also open source (though I didn’t double-check).
Leaving your door wide open because you don’t like a particular brand of lock does not strike me as a great security stance.
I’m not leaving the door wide open. Like I said, it’s a closed system that we’re building on email protocols. We use filtering to prevent the kinds of accidental leaks you’re discussing in other messages. We hand out specifically configured clients. And we use servers configured to block plaintext messaging. We tried something similar with XMPP, but it was easier on the people who were responsible for infrastructure using email things, and people using the system liked the email interface better.
But then I also don’t know why you have issues with “trust” with Signal, which is open source.
One of the reasons is because in order to sign up, I have to surrender a phone number, and they are not open enough for me to tell what they might be doing with that. There’s no way for me to see what they’re actually doing on the server side so I can infer it, either.
Email requires you to “surrender” huge amounts of useful information – enough that the secret police might not even need to decrypt the body of the messages to get enough evidence to convict you.
But sure, a phone-based tool that initially used phone numbers (disposable ones are easily available, or so I hear) to avoid the need for centralized storage of things like contact lists and personal data is the real risk, and email is just fine an peachy and dandy. Now let me just go see who’s knocking so loudly at mydoor…
But I can run my own email server. And it’s trivial to give an account on there to anyone whose phone number I have (which is how you found people on Signal, last time I tried it…)
I’m not worrying about surrendering info to myself.
Thanks for the thought exercise. I would probably consider signal a great replacement for email if I could stand up my own server and have my contacts connect their signal apps to it. I suspect that’s a smaller lift now than it used to be, since they’ve started publishing server code again, and might hopefully even simply be a skill issue at this point.
So you can do your own, more complicated and more easily seized thing that still likely leaks plaintext metadata to the secret police, and you think that’s better than something that uses a phone number – which you can easily obtain as a disposable resource – to bootstrap itself without requiring a centralized account system.
which you can easily obtain as a disposable resource
I have admittedly not yet tried very hard, but every place around here that I’ve checked requires a government ID for this.
Looking through the documentation on signal’s site, I don’t see anything explaining why they need it, nor making me expect that it’s disposable. Don’t I need to keep that number as long as I have my signal account?
They do a great job covering the crypto, which is good. Their privacy documentation does not explain, at least in any place that I’ve yet found, what they’re doing with my phone number and how they prevent it from being metadata associated with my encrypted messages.
more easily seized thing that still likely leaks plaintext metadata to the secret police
Signal and their infrastructure is in the same physical jurisdiction I’m in. So police that are interested in me can go after things in Signal’s hosting facility. I can host my own stuff in a different physical jurisdiction, where it’d be less easily seized than Signal’s. That is part of the appeal of self-hosting.
And our police aren’t secret. There are just signs that they’re becoming more worrisome.
Our conversation has persuaded me to continue learning about signal. I know you’re a serious person, and your technical judgement often aligns with mine, so I want to see if it’d work for what I’m putting together.
A thing I’m finding missing from the signal software is the ability to use hardware tokens for crypto. I’d like to have my private keys stored on an external token, and I don’t see a way to do that even if I spend time learning to self-host signal. It looks like that just isn’t a feature they offer, even for their desktop software.
It’s starting to look like, even if my trust issues could be resolved, what I want to do is simply more complicated than what signal supports at the moment.
I’d love to contribute to building something that uses signal’s crypto and protocols with hardware keys at the endpoints, an interface and behavior that’s more like email than SMS (particularly as regards attachments), coordinated on a self-hostable server.
But I need to get this bootstrapped at an in-person meeting that will take place between mid-December and late January. It’ll be a “party” where we distribute tokens that I’ll have sourced, generate keys, configure wireguard and FDE on everyone’s PCs, connect via wireguard to an email server which only accepts traffic over that wireguard connection, doesn’t relay traffic to other email servers, and refuses all plaintext mail.
I don’t think the thing I’d like to contribute to can be ready in that timeframe, and I don’t see where it’s been built already for me. I do think I can get the server I’ve described hosted in an appropriate place and configured.
I don’t think we can have a productive conversation about this because I fundamentally don’t think your apparent preference for spending huge amounts of effort kinda-sorta securing a thing that’s fundamentally un-securable is something that I can engage with or talk you out of. The lengths you’re prepared to go to in order to avoid the possibility of “just use a burner phone” frankly astound me.
We probably can’t. Your insistence on dismissing features like “hardware tokens for private keys”, “richer messaging than SMS (or, we need real attachments)” and “hosted in another jurisdiction” as if it boils down to “not wanting to use a burner phone” means we’re talking past each other.
Your confidence that reasonably anonymous burner phones can be had here is also, curiously, much higher than mine.
But thank you (sincerely!) for discussing it this far.
what makes @hoistbypetard’s proposed system “fundamentally un-securable” and why don’t the same fundamentals apply to Signal? it sounds like his proposal can be more secure because keys and software is set up in-person. aren’t the fundamental limitations more on the Signal side because they are targeting remote key and software distribution, which is much harder to secure? I would be really curious if you could unpack this to any degree.
anyone whose phone number I have (which is how you found people on Signal, last time I tried it…)
This was covered in the article. Adding someone on signal only requires a username these days. A phone number is required to sign up, which is damn easy to buy with cash in most parts of the world.
That’s good. Do you know why signal still needs a phone number from you, though?
Last time I tried to buy a phone number with cash here, the person selling it was pretty insistent on having a copy of a government-issued ID. This was on the east coast of the US. Had I cared more in that case, I think I very likely could’ve got one somewhere else without showing ID, but the ID requirement and the cameras all around the store didn’t make being able to use cash feel like it improved anonymity in any meaningful way.
For my application, I don’t actually mind giving my phone number to the people I’m messaging, and vice versa. We’re all within 2 degrees of each other IRL. I am not yet comfortable with signal having mine and my contacts’ numbers associated with each other.
you can personally set up the most bad-ass hard-ass ultra-mega-NSA-mil-spec XP++++ Pro grade encryption on your end, but any recipient who sees the plaintext can still quite easily quote-reply with the plaintext, or forward it, etc.
Exactly the same is true of signal though. Someone could screenshot your message and share that. If “the recipient could choose to willingly leak the plaintext” is in your threat model, there’s not much you can do.
In the case of Signal, the recipient has to choose to leak it and take active measures to do so.
In the case of email, it’s far too easy for the recipient to accidentally leak it, because they might not have the same super-ultra-opsec config the sender does.
Signal pushes people onto phones, which are more likely to be used in public places where people can see your messages over your shoulder.
Signal also support iOS where the binaries cannot be verified against the published code, so E2EE doesn’t mean much there.
The “just use Signal” mantra also doesn’t help because people are given a false sense of security. With email at least there is a general awareness of the need to be careful and consult a nerd if you actually want to protect your communications.
In the case of Signal, the recipient has to choose to leak it and take active measures to do so.
Smartphone users take screenshots of text all the time without thinking twice, and default smartphone configurations increasingly upload every file to the cloud automatically.
Excuse me, but I find some of these arguments quite weak.
It is true that Signal pushes people onto phones, but if other people glancing at your phone poses an actual threat, the problem is not the smartphone. If you are in that situation you need a well-calibrated threat model and a considerate OPSEC regime. There’s no way around that. And if you need it, it’s not like it’s hard to find a spot where glancing is impossible. Sit alone in a corner or go to a bathroom.
I think Signal has good reasons for the “smartphone first” paradigm.
For most people these days, the smartphone is their primary computer. It’s more common to have a smartphone than a laptop or desktop. One of the primary goals of Signal is to be accessible and that means being available on the most popular devices.
Smartphones are also more locked-down and more secure than general-purpose computers. I personally prefer using a laptop for most tasks, but that’s not for security reasons – I just find it more convenient and ergonomical.
I don’t believe it’s anywhere near reasonable to claim that E2EE is pointless on iOS. I don’t find this argument serious. AFAIK it’s still possible to use IDA Pro or Ghidra to analyze an iPhone app. Reproducible builds would be nice, but it’s not reasonable to claim E2EE isn’t possible just because reproducible builds aren’t available on a certain platform.
Smartphone users take screenshots of text all the time without thinking twice, and default smartphone configurations increasingly upload every file to the cloud automatically.
I think this is a good point! Again, for people where security matters, OPSEC is crucial.
Smartphones are also more locked-down and more secure than general-purpose computers.
Basic security precautions for non-profits and journalists in the United States, early 2019.
[…] - Don’t use an Android phone, use an iPhone instead.
The link doesn’t really explain how or why that’s the case. It also has statements like above which seem counter-intuitive without enough context. Is that to generalize your digital fingerprint or something similar?
It is true that Signal pushes people onto phones, but if other people glancing at your phone poses an actual threat, the problem is not the smartphone. If you are in that situation you need a well-calibrated threat model and a considerate OPSEC regime. There’s no way around that. And if you need it, it’s not like it’s hard to find a spot where glancing is impossible. Sit alone in a corner or go to a bathroom.
@ubernostrum said the user has to take deliberate measures to leak data with Signal, but is liable to leak data by accident with PGP. so we are talking about users who don’t have a considerate OPSEC regime, and I’m pointing out that Signal is not foolproof for those users either. we’re saying the same thing.
I think Signal has good reasons for the “smartphone first” paradigm.
For most people these days, the smartphone is their primary computer. It’s more common to have a smartphone than a laptop or desktop. One of the primary goals of Signal is to be accessible and that means being available on the most popular devices.
it’s not just “smartphone first,” it’s “you absolutely need a smartphone to use Signal.” the organization is not starved for resources, and they have decided to implement useless things like sealed sender when they could have implemented a non-smartphone SMS workflow for account creation and authentication. it certainly doesn’t seem like accessibility is a primary goal.
Smartphones are also more locked-down and more secure than general-purpose computers.
not for the threat models that Signal claims to accommodate.
I don’t believe it’s anywhere near reasonable to claim that E2EE is pointless on iOS. I don’t find this argument serious. AFAIK it’s still possible to use IDA Pro or Ghidra to analyze an iPhone app.
are you saying backdoors are detectable in iOS apps? that is news to me.
Reproducible builds would be nice, but it’s not reasonable to claim E2EE isn’t possible just because reproducible builds aren’t available on a certain platform.
the benefit of E2EE is that you don’t have to trust the service not to log your communications. if the service is not trusted, then what reason is there to believe their claim that an app uses E2EE and is not backdoored? under what threat model would you not trust a service to scan your communications if they passed through their servers in plaintext, but you would trust them to faithfully provide binaries with E2EE that is not broken?
Smartphone users take screenshots of text all the time without thinking twice, and default smartphone configurations increasingly upload every file to the cloud automatically.
I think this is a good point! Again, for people where security matters, OPSEC is crucial.
and yet you still think smartphones are more secure?
Why do you feel Sealed Sender is useless? This is actually one of my primary examples of Signal’s centralized model having advantages - they were able to simply eliminate a swath of metadata unilaterally.
Sealed sender seems like a perfect example of the confusion around Signal’s security model. As far as I can tell it doesn’t hide the sender in practice because Signal can still see sender’s IP address and can build a near-perfect mapping between users and their last used IP address. In theory maybe you can deliberately share IP addresses between a few accounts with the right VPN setup providing a small anonymity set for the sender, but that’s still largely broken if there’s an ongoing conversation where Signal sees the recipient of every message. Nobody seems willing to argue that this mostly useless security enhancement is worth the complexity. It seems like Signal added it mostly because it’s a fairly novel feature that they can write a blog post about and claim that they’re “raising the bar.” In other words they’re willing to add complexity that provides little practical benefit for marketing purposes, which is awfully concerning for an app that claims to be suitable for security-critical applications.
I could easily be missing something; if you think I am I would love to hear it. I think the part about the VPN setup might actually be wrong and overly generous to Signal.
No, it’s not exactly the same. Screenshotting a Signal message and sharing that is a deliberate act. Replying or forwarding a decrypted PGP message in plaintext is done all the time by accident.
Email is insecure. Even with PGP, it’s default-plaintext, which means that even if you do everything right, some totally reasonable person you mail, doing totally reasonable things, will invariably CC the quoted plaintext of your encrypted message to someone else (we don’t know a PGP email user who hasn’t seen this happen).
it’s also perfectly feasible for a smartphone user to screenshot a message for reference and have it automatically uploaded to the cloud by their smartphone.
the evidence for the problem on the PGP side is “we don’t know a PGP email user who hasn’t seen this happen,” but in the smartphone case you wouldn’t see it happen which is even worse. we don’t have the data to say that it’s an issue in one case and not the other.
sure, you can take screenshots on a computer, but you are not encouraged to by a horrible touch based UI and configurations that automatically suck your files into the cloud are much less common.
No. PGP email and S/MIME email clients attempt to solve problems that age does not attempt to solve. The interesting one, IMO, is associating public keys to recipient identities.
To use age for someone other than yourself, you need to get their key out-of-band and encrypt for it:
$ age -o example.jpg.age -r age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p \
-r age1lggyhqrw2nlhcxprm67z43rta597azn8gknawjehu9d9dl0jq3yqqvfafg example.jpg
That’s part of what PGP and S/MIME clients are doing for you.
I don’t know what you mean by “out of band” for PGP — or rather, I don’t know what you would mean by “in band”. Practically speaking, whenever I’ve used PGP mail, I got the public key from a webpage or other “out of band” (?) source. The “web of trust” has never really worked in a general way. So there doesn’t seem like much of a difference, except there’s a local key store, and it would be pretty easy to leverage systemwide keychains now that OSes have them.
“In-band” vs “out-of-band” wasn’t quite the correct term. I mean that I could connect a PGP client or an S/MIME client up to a directory server of some sort, or that I could configure them to trust certain key signers, and they can encrypt messages for a recipient without me having to paste a key for each recipient in.
Whereas what’s being described for age means that I’d need to have each person’s key somewhere, paste it into the command, etc.
The email clients are trying (or did try) to solve this and age clients are not (yet).
Understood. In an ideal world there is an integrated and secure experience between mail client, directory/keyserver, and local keychain. PGP clients do this in theory, but because there’s no trusted directory (soatok’s Public Key Directory project) the practice falls short of the theory.
In the spirit of age (just make the right choices and version the whole thing) perhaps there’s a combination of newer building blocks that would give a smooth experience with better security and practical interoperability. I mean, other than Signal.
It sounds like something they “plan” to work on. It doesn’t sound like something I can tell someone to use now instead of PGP. Which is what the title of the article promised.
Most people, 99% of the time, just care about things like backups, sending files, or signing things. They don’t care about binding the trust relationship between a public key and an “identity” (term left vague due to different social interpretations of the word).
The article delivers on the things people care about, and says “I’m working on it” for the nerd use cases.
I’m not sure I would say that PGP/GnuPG was ever very wide-spread and it certainly hasn’t been gaining traction for the past 15 (20?) years or so. In 2024, I won’t hesitate to say that GnuPG is basically dead.
I have a lot of respect for what Phil Zimmermann did in the 90s, but that was a long time ago – cryptography has made huge leaps since, especially when we talk about applied cryptography. GnuPG is simply not very relevant these days. We have better tools for the problems that GnuPG attempted to solve. The exception being an e-mail-like system with good security properties – which is also discussed in the article.
I’m not an expert in this space, but my interpretation of this fact is not that “it takes nine separate tools to equal what GnuPG can do,” but rather “GnuPG performs nine different functions that—with the benefit of hindsight—did not need to be mashed together into one tool.”
Yes, and I agree with the interpretation. That’s the point that everybody has been making during all this time.
However, the initial question was:
When will the PGP-induced madness end?
And my observation was that we need to try to understand why it hasn’t.
The point being, we need to understand why all these features “wrongly” mashed together into one tool, are still being widely used, when theoretical better alternatives focused on a single problem exist. Even when there has been a considerable effort to promote them and “educate” the public.
If we figure that out, probably we can move forward.
One explanation, covered in the original Latacora article, is that PGP “is also a social network, and a subculture.”
The real world security of PGP has not been publicly tested, because it’s not very widely used, the people using it are mostly people not targeted by law enforcement, and its UX is not amenable to easy use by people who are targeted by law enforcement in the developed world, namely criminals.
Ladybird browser is doing an amazing job making contribution look appealing.
There is a YouTube series showing some debugging and development for it that gave me a programming itch to scratch.
When I’ve tried emulating X86-64 on Apple Silicon using QEMU it’s been incredibly slow, like doing ls took like 1-2 seconds. So if these fine people manage to emulate games then I’m very impressed!
QEMU emulation (TCG) is very slow! Its virtue is that it can run anything on anything, but it’s not useful for productivity or gaming. I used to use it to hack around a FEX RootFS as root, and even just downloading and installing packages with dnf was excruciatingly slow.
Emulators that optimize for performance (such as FEX, box64, and Rosetta, and basically every modern game console emulator too) are in a very different league. Of course, the tradeoff is they only support very specific architecture combinations.
As @lina says, QEMU is general. It works a few instructions at a time, generates an IR (TGIR, which was originally designed for TCC, which was originally an IOCC entry), does a small amount of optimisation, and emits the result.
Rosetta 2 works on much larger units but, more importantly, AArch64 was designed to support x86 emulation and it can avoid the intermediate representation entirely. Most x86-64 instructions are mapped to 1-2 instructions. The x86-64 register file is mapped into 16 of the AArch64 registers, with the rest used for emulator state.
Apple has a few additional features that make it easier:
They use some of the reserved bits in the flags register for x86-compatible flags emulation.
They implement a TSO mode, which automatically sets the fence bits on loads and stores.
FEX doesn’t (I think) take advantage of these (or possible does but only on Apple hardware?), but even without them it’s quite easy (as in, it’s a lot of engineering work, but each bit of it is easy) to translate x86-64 binaries to AArch64. Arm got a few things wrong but both Apple and Microsoft gave a lot of feedback and newer AArch64 revisions have a bunch of extensions that make Rosetta 2-style emulation easy.
RISC-V’s decision to not have a flags register would make this much harder.
There are two more hardware features: SSE denormal handling (FTZ/DAZ) and a change in SIMD vector handling. Those are standardized as FEAT_AFP in newer ARM architectures, but Apple doesn’t implement the standard version yet. The nonstandard Apple version is not usable in FEX due to a technicality in how they implemented it (they made the switch privileged and global, while FEX needs to be able to switch between modes efficiently, unlike Rosetta, and calling into the kernel would be too slow).
FEX does use TSO mode on Apple hardware though, that’s by far the biggest win and something you can’t just emulate performantly if the hardware doesn’t support it. Replacing all the loads/stores with synchronized ones is both slower and also less flexible (fewer addressing modes) so it ends up requiring more instructions too.
them it’s quite easy […] to translate x86-64 binaries to AArch64
[…]
RISC-V’s decision to not have a flags register would make this much harder.
Dumb question: is there a reason not to always ahead-of-time compile to the native arch anyway?
(i believe that is what RPCS3 does, see the LLVM recompiler option).
As I understand it, that’s more or less what Rosetta 2 does: it hooks into mmap calls and binary translates libraries as they’re loaded. The fact that the mapping is simple means that this can be done with very low latency. It has a separate mode for JIT compilers that works more incrementally. I’m impressed by how well the latter works: the Xilinx tools are Linux Java programs (linked to a bunch of native libraries) and they work very well in Rosetta on macOS, in a Linux VM.
The Dynamo Rio work 20 or so years ago showed that JITs can do better by taking advantage of execution patterns. VirtualPC for Mac did this kind of thing to avoid the need to calculate flags (which were more expensive on PowerPC) when they weren’t used. In contrast, Apple Silicon simp,y makes it sufficiently cheap to calculate the flags that this is not needed.
Rosetta does do this, but you have to support runtime code generation (that has to be able to interact with AOT generated code) at minimum because of JITs (though ideally an JIT implementation should check to see if it is being translated and not JIT), but also if you don’t support JIT translating you can get a huge latency spike/pause when a new library is loaded.
So no matter what you always have to support some degree of runtime codegen/translation, so it’s just a question of can you get enough of a win from an AOT as well as the runtime codegen to justify the additional complexity.
Ah, sorry I didn’t bother to actually check what the XML file contained. I expected it to be a standard feed, but it’s probably just some vestigial remnant generated by the CMS they use.
With the understanding that I might just have been lucky, and / or working for a company seen as a more valuable customer … in a previous role I was a director of engineering for a (by Australian standards) large tech company. We had two teams run into difficulties, one with AWS, and another with GCP.
One team reached out to Amazon, and our AWS TAM wound up giving us some great advice around how we’d be better off using different AWS tech, and put us in touch with another AWS person who provided some deeper technical guidance. We wound up saving a tonne of money on our AWS bill with the new implementation, and delighting our internal customers.
The other team reached out to Google, and Google wouldn’t let a human speak to us until we could prove our advertising spend, and as it wasn’t high enough, we never got a straight answer to our question.
Working with Amazon feels like working with a company that actually values its customers; working with Google feels like working with a company that would prefer to deal with its customers solely through APIs and black-box policies.
I’m still confused. What do you have to do in AWS that you don’t have to do in GCP or is significantly easier? I have very little experience with GCP and a lot with AWS, so interested to learn more about how GCP compares.
“Trust me bro” is a difficult sell; but I am here to back up the sentiment. There are simply too many little things that by themselves are not deal breakers at all - but make the experience much more frustrating.
GCP feels like if engineers made a cloud platform for themselves with nearly infinite time and budget (and some idiots tried to bolt crud on the side, like oracle, vmware and various AI stuff).
AWS feels like if engineers were forced to write adjacently workable services without talking to each other and on an obscenely tight deadline - by people who didnt really know what they wanted, and then accidentally made something everyone started to use.
I’m going yo write my own blog post about this, I used to have a list of all my minor frustrations so I could flesh it out.
I’m still working on it, there’s some missing points and I want to bring up a major point about cloud being good for prototyping, but here: https://blog.dijit.sh/gcp-the-only-good-cloud/
The thing you are describing I should have only needed to learn once for each cloud vendor and put in a script: The developer-time should amortise to basically zero for either platform as long as the cloud vendor doesn’t change anything.
The developer-time should amortise to basically zero for either platform as long as the cloud vendor doesn’t change anything.
Yes and no.
Some constructs you don’t even have to worry about on GCP, so, it is best for fast deployments.
However, if you spend 500+ hours a year doing cloud work, then your point stands.
Indeed, if you are doing 5 hours a week of cloud infra work, your application is likely a full-time job or equivalent. I do believe you made the right choice with AWS.
No.
If you are spending 5 hours a week on infrastructure, then your application would be worth spending 40-hours on. Or is infra the only component of your application?
Can you do an actual comparison of the work? I’d be curious. Setting up a web service from a Docker image is a few minutes. Env vars are no extra work. Secrets are trivial. A cert would take maybe 10 minutes?
Altogether, someone new should be able to do this in maybe 30 minutes to an hour. Someone who’s done it before could likely get it done in 10 minutes or less, some of that being downtime while you wait for provisioning.
You’ve posted a lot in this thread to say, roughly ‘they’re different, AWS is better, but I can’t tell you why, trust me’. This doesn’t add much to the discussion. It would help readers (especially people like me, who haven’t done this on either platform) if yo u could slow down a bit and explain the steps in AWS and the steps in GCP and why there’s more friction with the latter.
No, please don’t trust me.
Here’s an exercise for yourself.
Assuming you control a domain domain1.com.
Deploy a dockerized web service (pull one from the Internet if you don’t know how to write one).
Deploy it on gcp1.domain1.com,
Deploy another on aws1.domain1.com.
Compare how many steps it takes and how long it takes.
Here’s how I deploy it on GCP. I never open-sourced my AWS setup but I am happy to see a faster one.
As I said, I have not used AWS or GCP so I have no idea how many steps it requires on either platform. I can’t judge whether your command is best practice for CGP and have no idea what the equivalent is on AWS (I did once do this on Azure and the final deploy step looked similar, from what I recall, but there was some setup to create the Azure Container thingy instance, but I can’t tell from your example if this is not needed on GCP of if you’ve simply done it already). If I tried to do it on AWS, I’d have no idea if I were doing the most efficient thing or some stupid thing that most novices would manage to avoid.
You have, apparently, done it on both. You are in a position to tell me what steps are needed on AWS but not on GCP. Or what abstractions are missing on AWS but are better on GCP. Someone from AWS might even read your message and improve AWS. But at the moment I just see that a thing on GCP is one visible step plus at least zero setup steps, whereas on AWS it is at least one step.
You’ve posted ten times so far in this story to say that GCP is better, but not articulated how or why it’s better. As someone with no experience with either, nothing you have said gives me any information to make that comparison. At least one of the following is true:
GCP has more efficient flows than AWS.
You are more familiar with GCP than AWS and so you are comparing an efficient flow with GCP that you’ve learned to use to an inefficient flow with AWS.
You are paid by GCP to market for them (not very likely).
It sounds as if you believe the first is true and the second two are not but (again) as someone reading your posts who understands the problem but is not familiar with either of the alternatives in any useful level of detail, I cannot judge for myself from your posts.
If you wrote ‘GCP has this set of flows / abstractions that have no equivalent on AWS’ then someone familiar with AWS could say ‘actually, it has this, which is as good’ or ‘you are correct, this is missing and it’s annoying’. But when you write:
I have done it for myself and I’m speaking from experience.
That doesn’t help anyone reading the thread understand what is better or why.
I find AWS to be better organized and documented but much larger that any Google product I’ve seen. There is more to learn because it’s a large, modular system. And it’s trivial to do an easy things the hard way.
I don’t have much direct experience with Google’s hosting but if any of their service products are similar, the only advantage is they do less, which means you can skimp on organization and documentation without too many problems.
In terms of big players I would recommend GCP still, but only because I mostly work with Kubernetes and it’s best there. From smaller players Fly.io is actually works well for me.
I tried using checkpoints for a while but it didn’t do quite what I wanted and it was pretty tedious having to editor configs in a different directory and then running cz apply.
There was a blog post here recently that made the case for writing your own static site generator (that resonated with me). I am starting to think that maybe writing your own purpose-built dot file manager would similarly be a good idea.
I did exactly that for a while, but it was awkward and unwieldy, even after years of adding features and cleaning it up. A more holistic solution (i.e. Nix) ended up being the answer for me.
No matter how many times Go people try to gaslight me, I will not accept this approach to error-handling as anything approaching good. Here’s why:
Why must you rely on a linter or IDE to catch this mistake? Because the compiler doesn’t care if you do this.
If you care about correctness, you should want a compiler that considers handling errors part of its purview. This approach is no better than a dynamic language.
The fact that the compiler doesn’t catch it when you ignore an error return has definitely bitten me before.
doTheThing()on its own looks like a perfectly innocent line of code, and the compiler won’t even warn on it, but it might be swallowing an error.I learned that the compiler doesn’t treat unused function results as errors while debugging a bug in production; an operation which failed was treated as if it succeeded and therefore wasn’t re-tried as it should. I had been programming in Go for many years at that point, but it had never occurred to me that silently swallowing an error in Go could possibly be so easy as just calling a function in the normal way. I had always done
_ = doTheThing()if I needed to ignore an error, out of the assumption that of course unused error returns is a compile error.Does anyone know the reason why the Go compiler allows ignored errors?
Because errors aren’t special to the Go compiler, and Go doesn’t yell at you if you ignore any return value. It’s probably not the most ideal design decision, but in practice it’s not really a problem. Most functions return something that you have to handle, so when you see a naked function call it stands out like a sore thumb. I obviously don’t have empirical evidence, but in my decade and a half of using Go collaboratively, this has never been a real pain point whether with junior developers or otherwise. It seems like it mostly chafes people who already had strong negative feelings toward Go.
It’s similar to array bounds checks in c - not really a problem.
I hope this is sarcasm.
Yes
Is there a serious argument behind the sarcasm as to how this is comparable to array bounds checks? Do you have any data about the vulnerabilities that have arisen in Go due to unhandled errors?
Because the programmer made an intentional decision to ignore the error. It won’t let you call a function that returns an error with out assigning it to something, that would be a compile time error. If the programmer decides to ignore it, that’s on the programmer (and so beware 3rd party code).
Now perhaps it might be a good idea for the compiler to insert code when assigned to _ that panics if the result is non-nil. Doesn’t really help at runtime, but at least it would fail loudly so they could be found.
I’ve spent my own share of time tracking down bugs because something appeared to be working but the error/exception was swallowed somewhere without a trace.
This is incorrect: https://go.dev/play/p/k7ErZU5QYCu
huh… til. I always assumed you needed to use the result, probably because of single vs multiple returns needing both being a compile time error. Thanks.
To be fair I was just as certain as you that of course Go requires using the return values, until I had to debug this production bug. No worries.
is not an intentional decision to ignore the error. Neither is
Yet the go compiler will never flag the first one, and may not flag the second one depending on
errbeing used elsewhere in the scope (e.g. in the unheard of case where you have two different possibly error-ing calls in the same scope and you check the other one).Yeah, I always thought the _ was required, I learned something today!
I do have a few places with err and err2, it does kind of suck - I should probably breakup those functions.
_, err := f.Write(s)is a compiler error iferralready exists (no new variables on left side of :=), and iferrdoesn’t already exist and you aren’t handling it, you get a different error (declared and not used: err). I think you would have to assign a new variablet, err := f.Write(s)and then take care to handletin order to silently ignore theerr, but yeah, with some work you can get Go to silently swallow it in the variable declaration case.Because they couldn’t be arsed to add this in v0, and they can’t be arsed to work on it for cmd/vet, and there are third-party linters which do it, so it’s all good. Hopefully you don’t suffer from unknown unknowns and you know you should use one of these linters before you get bit, and they don’t get abandoned.
(TBF you need both that and errcheck, because the unused store one can’t catch ignoring return values entirely).
Considering how much effort the Go team puts in basically everything, this language makes it very hard to to take you serious.
Yes, except for things that they decide not to be arsed about. I can confirm this as a very real experience of dealing with Go.
Which is fair enough.
Sure, but then it is equally fair to criticize them for it.
The go compiler doesn’t do warnings, only errors. Linters do warnings, and do warn about unchecked errors.
I don’t really care. Generally speaking, I would expect compilers to either warn or error on an implicitly swallowed error. The Go team could fix this issue by either adding warnings for this case specifically (going back on their decision to avoid warnings), or by making it a compile error, I don’t care which.
This is slightly more nuanced. Go project ships both
go buildandgo vet.go vetis an isomorphic to how Rust handles warnings (that warnings apply to you, not your dependencies).So there would be nothing wrong per se if this was caught by
go vetand notgo build.What is the issue though, is that this isn’t caught by first-party
go vet, and requires third partyerrcheck.Meh plenty of code bases don’t regularly run
go vet. This is a critical enough issue that it should be made apparent as part of any normal build, either as a warning or an error.And that’s perfectly fine given that Go is pleasurable even for quick and dirt prototypes, fun side projects, and so on.
I agree with you that it’s better for this to be a compiler error, but (1) I’ll never understand why this is such a big deal–I’m sure it’s caused bugs, but I don’t think I’ve ever seen one in the dozen or so years of using Go and (2) I don’t think many dynamic languages have tooling that could catch unhandled errors so I don’t really understand the “no better than a dynamic language” claim. I also suspect that the people who say good things about Go’s error handling are making a comparison to exceptions in other languages rather than to Rust’s approach to errors-as-values (which has its own flaws–no one has devised a satisfactory error handling system as far as I’m aware).
The fact that these bugs seem so rare and that the mitigation seems so trivial makes me feel like this is (yet another) big nothingburger.
The most common response to my critique of Go’s error-handling is always some variation on “this never happens”, which I also do not accept because I have seen this happen. In production. So good for you, if you have not; but I know from practice this is an issue of concern.
Relying on the programmer to comprehensively test inputs imperatively in a million little checks at runtime is how dynamic languages handle errors. This is how Go approached error-handling, with the added indignity of unnecessary verbosity. At least in Ruby you can write single-line guard clauses.
I don’t really follow your dismissal of Rust since you didn’t actually make an argument, but personally I consider Rust’s
Optiontype the gold standard of error-handling so far. The type system forces you to deal with the possiblity of failure in order to access the inner value. This is objectively better at preventing “trivial” errors than what Go provides.I’m sure it has happened before, even in production. I think most places run linters in CI which default to checking errors, and I suspect if someone wasn’t doing this and experienced a bug in production, they would just turn on the linter and move on with life. Something so exceedingly rare and so easily mitigated does not meet my threshold for “issue of concern”.
That’s how all languages handle runtime errors. You can’t handle them at compile time. But your original criticism was that Go is no better than a dynamic language with respect to detecting unhandled errors, which seems untrue to me because I’m not aware of any dynamic languages with these kinds of linters. Even if such a linter exists for some dynamic language, I’m skeptical that they’re so widely used that it merits elevating the entire category of dynamic languages.
I didn’t dismiss Rust, I was suggesting that you may have mistaken the article as some sort of criticism of Rust’s error handling. But I will happily register complaints with Rust’s error handling as well–while it does force you to check errors and is strictly better than Go in that regard, this is mostly a theoretical victory insofar as these sorts of bugs are exceedingly rare in Go even without strict enforcement, and Rust makes you choose between the verbosity of managing your own error types, debugging macro expansion errors from crates like
thiserror, or punting altogether and doing the bare minimum to provide recoverable error information. I have plenty of criticism for Go’s approach to error handling, but pushing everything into an error interface and switching on the dynamic type gets the job done.For my money, Rust has the better theoretical approach and Go has the better practical approach, and I think both of them could be significantly improved. They’re both the best I’m aware of, and yet it’s so easy for me to imagine something better (automatic stack trace annotations, capturing and formatting relevant context variables, etc). Neither of them seems so much better in relative or absolute terms that their proponents should express superiority or derision.
I don’t accept your unsubstantiated assertion that this is rare, so it seems we are at an impasse.
Fair enough. It’s a pity things like this are so difficult to answer empirically, and we must rely on our experiences. I am very curious how many orgs are bitten by this and how frequently.
Couldn’t agree more (honourable mention to Zig, though).
Enabling a linter is different from doing “a million little checks at runtime”. This behaviour is not standard because you can use Go for many reasons other than writing production-grade services, and you don’t want to clutter your terminal with unchecked error warnings.
I admit that it would be better if this behaviour were part of
go vetrather than an external linter.The strange behaviour here is not “Go people are trying to gaslight me”, but people like you coming and complaining about Go’s error handling when you have no interest in the language at all.
You can’t lint your way out of this problem. The Go type system is simply not good enough to encapsulate your program’s invarients, so even if your inputs pass a type check you still must write lots of imperative checks to ensure correctness.
Needing to do this ad-hoc is strictly less safe than relying on the type system to check this for you.
errchecks are simply one example of this much larger weakness in the language.I have to work with it professionally, so I absolutely do have an interest in this. And I wouldn’t feel the need to develop this critique of it publicly if there weren’t a constant drip feed of stories telling me how awesome this obviously poor feature is.
Your views about how bad Go’s type system is are obviously not supported by the facts, otherwise Go programs would be full of bugs (or full of minuscule imperative checks) with respect to your_favourite_language.
I understand your point about being forced to use a tool in your $job that you don’t like, that happened to me with Java, my best advice to you is to just change $job instead of complaining under unrelated discussions.
They are full of bugs, and they are full of miniscule imperative checks. The verbosity of all the
if err != nilchecks is one of the first things people notice. Invoke “the facts” without bringing any isn’t meaningfully different than subjective opinion.Your comments amount to “shut up and go away” and I refuse. To publish a blog post celebrating a language feature, and to surface it on a site of professionals, is to invite comment and critique. I am doing this, and I am being constructive by articulating specific downsides to this language decision and its impacts. This is relevant information that people use to evaluate languages and should be part of the conversation.
If
if err != nilchecks are the “minuscle imperative checks” you complain about, I have no problem with that.That you have “facts” about Go programs having worse technical quality (and bug count) than any other language I seriously doubt, at most you have anecdotes.
And the only anecdote you’ve been able to come up with so far is that you’ve found “production bugs” caused by unchecked errors that can be fixed by a linter. Being constructive would mean indicating how the language should change to address your perceived problem, not implying that the entire language should be thrown out the window. If that’s how you feel, just avoid commenting on random Go post.
Yeah, I have seen it happen maybe twice in eight years of using Go professionally, but I have seen it complained about in online comment sections countless times. :-)
If I were making a new language today, I wouldn’t copy Go’s error handling. It would probably look more like Zig. But I also don’t find it to be a source of bugs in practice.
Everyone who has mastered a language builds up muscle memory of how to avoid the Bad Parts. Every language has them. This is not dispositive to the question of whether a particular design is good or not.
The happy people are just happily working on solving their real problems. not wasting time complaining.
Not seeing a problem as a bug in production doesn’t tell you much. It usually just means that the developers spent more writing tests or doing manual testing - and this is just not visible to you. The better the compiler and type-system, the fewer tests you need for the same quality.
Agreed, but I wasn’t talking about just production–I don’t recall seeing a bug like this in any environment, at any stage.
In a lot of cases I am the developer, or I’m working closely with junior developers, so it is visible to me.
Of course with Go we don’t need to write tests for unhandled errors any more than with Rust, we just use a linter. And even when static analysis isn’t an option, I disagree with the logic that writing tests is always slower. Not all static analysis is equal, and in many cases it’s not cheap from a developer velocity perspective. Checking for errors is very cheap from a developer velocity perspective, but pacifying the borrow checker is not. In many cases, you can write a test or two in the time it would take to satisfy rustc and in some cases I’ve even introduced bugs precisely because my attention was so focused on the borrow checker and not on the domain problem (these were bugs in a rewrite from an existing Go application which didn’t have the bugs to begin with despite not having the hindsight benefit that the Rust rewrite enjoyed). I’m not saying Rust is worse or static analysis is bad, but that the logic that more static analysis necessarily improves quality or velocity is overly simplistic, IMHO.
I just want to emphasize that It’s not the same thing - as you also hint to in the next sentence.
I didn’t say that writing tests is always slower or that using the compiler to catch these things is necessarily always better. I’m not a Rust developer btw. and Rust’s errorhandling is absolutely not the current gold-standard by my own judgement.
It kind of is the same thing: static analysis. The only difference is that the static analysis is broken out into two tools instead of one, so slightly more care needs to be taken to ensure the linter is run in CI or locally or wherever appropriate. To be clear, I think Rust is strictly better for having it in the compiler–I mostly just disagree with the implications in this thread that if the compiler isn’t doing the static analysis then the situation is no better than a dynamic language.
What did you mean when you said “It usually just means that the developers spent more writing tests or doing manual testing … The better the compiler and type-system, the fewer tests you need for the same quality.” if not an argument about more rigorous static analysis saving development time? Are we just disagreeing about “always”?
Ah I see - that is indeed an exaggeration that I don’t share.
First that, but it also in general has other disadvantages. For instance, writing tests or doing manual tests is often easy to do. Learning how to deal with a complex time system is not. Go was specifically created to get people to contribute fast.
Just one example that shows that it’s not so easy to decide which way is more productive.
Ah, I think we’re agreed then. “always” in particular was probably a poor choice of words on my part.
Swallowing errors is the very worst option there is. Even segfaulting is better, you know at least something is up in that case.
Dynamic languages usually just throw an exception and those have way better behavior (you can’t forget, an empty catch is a deliberate sign to ignore an error, not an implicit one like with go), at least some handler further up will log something and more importantly the local block that experienced the error case won’t just continue executing as if nothing happened.
On a multi-user machine
/tmpis shared. For downloads I would prefer some world-unreadable directory. Though I don’t even know what are the recommendations for things like that nowadays. On Linux there’s/run/user/<uid>, maybe this one? Doesxdghave any dir for “user’s temp files”?XDG_CACHE_HOME would be the closest match.
I know you’re replying to the last question of the parent comment, but, for downloads, if using XDG user directories, might as well use
XDG_DOWNLOAD_DIR.On macOS
$TMPDIRis a per user location unlike/tmp– although I suspect fewer macs are shared than Linux systems.This makes me curious. Do you do a lot of work like described in the article on a multi-user machine?
All my home systems are configured with NixOS and have users for everyone that might need to use them, so kind of yes.
But my PoV here is about doing things right w.r.t. security “and just in case”. If you have a system daemon that runs as a separate user, and it’s compromised, you don’t want it to snoop/modify your downloads etc.
There’s
$XDG_RUNTIME_DIR, which is usually/run/user/<uid>https://specifications.freedesktop.org/basedir-spec/latest/#basics
I think that’s a bad place for downloads, since the spec says
Practically speaking, on many distros this directory is a tmpfs a few GBs or so in size, so you do actually run into the size limit, it’s not just a theoretical thing.
On my client machines, I create ~/tmp/ and use that as a location to download, unzip source bundles when I only need a single file, throw a clone of a project for a comparison. I have it set up as a ram disk with a 7 day retention so it works more or less like tmp. Additionally, I set the noexec bit on the directory which makes it a better place for stuff that I just want to dig through.
A huge amount of the stuff that I bring onto my box comes through the ~/tmp/ folder first, and after looking at it and deciding I need it for longer I will move things to a disk some where.
I realized at some point that there’s a lot of stuff I know I’d toss in 2 hours but I also didn’t want to put it in actual tmp because it’s shared and I don’t want to mess with permissions or deleting it later.
I’m so tired of people shitting on curl | sh scripts. An installer is just as dangerous! A deb/rpm package is just as dangerous! The binary you are trying to install is just as dangerous!
Yes, you should be careful what code you run on your computer, but it’s never the stuff you can see that should make you nervous, it’s the stuff you haven’t seen yet or can’t see.
Good post, thank you for acknowledging what so many seem to ignore.
To be fair, in the non-malicious case it is easier to make uninstall work sanely with deb/rpm, and there are like two or three checklist items for
curl|shthat are not needed for a downloadable installer.The #1 problem with curl|sh is the lack of artifacts for post-hoc investigation. The script never hits your disk, so there’s no chance of it being saved somewhere for later analysis.
One thing I recently learned is that it’s possible for a malicious server to detect curl | sh and serve different content in that case. Not disagreeing with your point, just an aspect that hadn’t occurred to me before.
That link gives me a
SEC_ERROR_EXPIRED_ISSUER_CERTIFICATEthat firefox doesn’t let me bypass.According to Chrome which is a bit more informative the certificate expired 701 days ago.
They can be, but it depends on the actual contents of those artifacts.
Example: Let’s say I download a .deb, and this package does not fetch any additional resources over the network. Then I can now for sure what’s in there, and analyze every part of it.
The same is not true for a resource that I fetch using Curl. The server could give me a different artifact for every new request. Most servers don’t, but they could.
The server could give me a different deb/rpm package for every new request as well.
It could try. But if you run
apt install gazonk, APT will download the package once. And AFAIK, APT will verify the checksum of the downloaded artifact.The equivalent of
curl foo.bar/baz.sh | shiswget foo.bar/baz.deb && dpkg -i baz.debnotapt get install baz. It’s a method for installing software not packaged by your distro.Therefore we reject the null hypothesis that
x11andwaylandare drawn from identical distributions.What does this mean?
they imported the data reported in the blog post as two numpy/scipy vectors, then ran a student’s t-test to test the hypothesis that the two distributions the two vectors were sampled from are different. (docs on
ttest_indhere).Technically this is bad statistics, because you’d want to specify your
confidencesignificance level ahead of time (typical value is 5%, which roughly corresponds to a difference of two standard deviations between the two distributions). Essentially, this test is saying that if the two variables came from the same underlying distribution, you would expect to see this data no more than 0.07% of the time. I suspect that the default for scipy uses a 5%confidence intervalsignificance level, so statistically you can say that this data comes from the same distribution (i.e. the latency is the same between X11 and Wayland) no more than 5% of the time.Edit: the above is sorta wrong. You don’t specify a significance level in the t-test itself, but you should specify it before performing the test, otherwise it’s hard to interpret the results based on your null hypothesis. So if we state the null hypothesis as “I expect that if X11 has the same latency as Wayland, and I measure the latency 20 times in both, I expect to get a different result no more than one time out of twenty”, then the above test is coherent and we can reject the null hypothesis.
Reading the docs some more, ~this might also be a bad test choice because it
I don’t think we have any reasonto suspect a priori (i.e. Wayland may have more lag but it may be more consistent than X11, or vice-versa).Edits: my stats is rusty AF.
Just a correction,
equal_val=Falsemeans we use Welsch’s t-test which drops the equal-variance assumption.I usually use p<0.01. I didn’t put that in since the p-value was below most typical thresholds (except those darn physicists).
from what I remember from statistics class, this means that there is a 0.06% chance that the distribution of both sets of values is the same
in other words the difference is highly unlikely to be due to statistical variance
It means that if
x11andwaylandare actually drawn from identical distributions, then probability of getting two sets of samples with a statistic less than or equal to what we got above is 0.06%.In other words, the difference would be improbable if
x11andwaylandwere drawn from identical distributions.The author has taken some latency measurements under x11 and Wayland. The measurements are all over the place, so eyeballing whether the sets are meaningfully or just noisy is difficult.
The poster above used Welch’s T-Test, a standard statistical test, to determine if the sets are statistically different or not. The low p value indicates that it is likely that the two sets of measurements are drawn from different distributions. Therefore the observed latencies do systematically differ between x11 and Wayland in this experiment.
Interesting! What would a high p value indicate?
There are disputes about the answer to that, but according to most theoretical statisticians (including Fisher) and to me, it would indicate that there’s probably something wrong with the statistical model you’re using. [source: closely related to the topic of my PhD and my papers]
Could you expand on this or share a link?
Sure. Please DM me and tell me how much detail you want! I know you’re smart (tridactyl) but I don’t know how interested you are in disputes in theoretical statistics.
Or in case you or anyone else doesn’t want to DM me, you can read R.A. Fisher’s original writings on this, and he was enormously influential (even when he was wrong).
That there’s not enough evidence to reject the null hypothesis. That is, we wouldn’t be able to say whether
x11andwaylandare drawn from an identical distribution or not.Out of curiosity, anyone who prefers Puppet over Ansible? Ansible has its warts, but I guess that is also true for Puppet.
I’ve found that Puppet can be far more expressive and easier to create constructs that are just plain difficult or ugly to do with the pseudo-DSL YAML used by Ansible and other tools. On the flip-side, unless you have a team with a “proper” developer background, the code-base can quickly spiral into an unmaintainable mess full of convoluted inheritance and dependency anti-patterns. Ansible, for all it’s warts, is much more straightforward in that it’s more or less “do thing x, then do thing y in the order I specify” and is a lot easier for non-developers to pick up.
Yeah, the other comment addresses it pretty well.
A more technical way of saying it, is that puppet describes a state, and ansible describes the process used to get to the state.
If you want a machine to be in a specific state, and stay in that state, puppet helps with that. If you just want to get a machine into a state, then destroy it if it deviates, ansible is a great tool. Especially if it’s simple to get it into the state and cheap to recreate.
I think it’s one reason that ansible was more widely used on cattle and puppet was more widely used on pets. Puppet even required certs and approval for nodes by default.
This list of deficiencies reads like a slightly obscured, writ-large version of “don’t roll your own crypto”.
Ah, but don’t you know, they “aren’t” rolling their own crypto, per their FAQ.
heavily rolls eyes
I like libsodium. It’s a great library of cryptography algorithms.
It doesn’t come, batteries included, with a full protocol for end-to-end encryption built-in. And so anyone who uses libsodium for e2ee is necessarily rolling their own crypto.
I’ve only heard of Session in passing over the years. This result is not surprising.
i initially thought you were being overzealous, until i read the first paragraph of the article
using libsodium to implement the documented signal protocol, i think would be fine. it does have risks, and should have some review before relying on it, but i wouldn’t really call that “rolling your own crypto”. and having a clean-room re-implementation would probably be good for the ecosystem
…but that’s not what they’re doing. they’re implementing their own protocol, and a cursory glance at their reasoning suggests that they want a decentralized messenger and care about security as an afterthought. which would be fine if they branded it that way, and not as an alternative to signal
This may be a little off topic, but I dislike the phrase “don’t roll your own crypto”.
Don’t roll your own crypto is generally a term which in itself is very ambiguous.
I’ve seen this phrase being thrown around when people just use gnutls in C vs people implementing some hash algorithm themselves. One I find very valid while the other one is an area where I would just use libsodium.
There are so many layers in crypto where you can apply the phrase that I find refuting (their claims with) this phrase in itself is meaningless unless you know what the authors intended. In this case it may as well be claims regarding the resistance against certain side channel attacks.
I’ve always asked myself how I can identify the moment I arrive at a skill level where I’m allowed to “roll my own crypto” depending on each possible interpretation people are using.
edit: added intended missing meaning in (…)
Absolutely. And the advice, taken to its logical extreme, would result in zero cryptography ever being developed.
It’s supposed to be along the same lines as advice given from lawyers to their kids that are considering a career in law. They say, “Don’t be a lawyer,” and if their kid isn’t dissuaded and can argue why they’d succeed as a lawyer, then maybe they should be one.
“Don’t roll your own crypto” is along the same lines. I currently do it professionally, but I also have a lot of oversight into the work I do to ensure it’s correct. Detailed specifications, machine-assisted proofs, peer review, you name it. I never start with code; I always start with “here’s why I’m doing this at all” (which includes closely related ideas and why they do not work for the problem I’m solving) and a threat model for my specific solution.
It can take months, or even years, to get a new cryptography design vetted and released with the appropriate amount of safety.
When it comes to cryptography protocol design, the greatest adversary is often your own ego.
I always read the advice as an incomplete sentence, which ends with “unless you know what you’re doing”, which is coincidentally like other safety advice, right? “This thing that you’re about to do is risky and dangerous unless you know how to do it, and in some cases, even if you do. Avoid doing it if you can. Exercise caution and care otherwise.” No?
I always viewed it as “don’t ship your own” - feel free to roll your own to play around with, but be cautious and get some review before putting it into production.
One piece of advice I’ve heard is: Before trying to build crypto, learn how to break crypto. Cryptopals is a good resource for that. It’s mindbending to learn about all the weird ways that crypto can fall apart.
At least one of my online friends agrees.
I think it’s more like, don’t roll your own crypto: don’t do it by yourself, collaborate with other experts, get lots of review from many angles.
I remember many moons ago that an expert in security and crypto actually published a list of cryptographic choices that should be your default. I wonder if this rings a bell to someone, it would be nice to recover that document, publish it here and see what this community would say in terms of freshen it up.
I might be wrong, but I think in the beginning the meaning of the phrase “don’t roll your own crypto” mean “do not try to come up with cryptographic algorithms on your own; use something tested and done by someone who know what they are doing”. I think the best way to describe what Soatok is putting forward is “don’t skip the default practices of security” or “don’t wave away cryptographic defaults in name of a watered down threat model”.
But maybe I am too off?
You might be thinking of “cryptographic right answers” from ’tptacek (2018 version, 2024 post-quantum version)
YES!!! You found it! Thank you @zimpenfish!
There’s also What To Use Instead of PGP from the very blog this Lobsters thread is about.
It was also posted on Lobsters.
Maybe I’m paranoid, but it reads to me like a plausibly deniable honeypot.
I think that’s a very reasonable concern. Particularly in light of the very first issue @soatok cites: the removal of PFS from the protocol. I’m on record as being skeptical of the “just use signal” advice that seems frequently deployed as a though-terminating measure in discussions about encrypted communication, but if I wanted to make something that was like signal but really a deniable honeypot, Session makes the same choices I would. It seems like a downgrade from signal in every important way.
Unrelated: the “Ad-blocker not detected” message at the bottom of the post made me laugh quite a bit. I use some tracker-blocking extensions (and browser configs) but I don’t run a real ad blocker in this browser. But many sites give me an “ad blocker detected” message and claim I need to turn off my non-existent ad blocker to support them. This site telling me I’m not running enough of one struck me as very funny.
Sure, its plausible.
But I find basically every time Soatok (or any security researcher) exposes any application that advertises itself as “secure/private” on the box, for their glaring bad practices, people (myself included) immediately go to “this is so stupid it has to be a honeypot”.
Are they all honeypots? (genuinely, maybe yes), or is it just stupidity?
i used to think that people sending a knockoff paypal payment link from a TLD i’ve never heard of was an obvious scam
then i tried to buy a monitor from hewlett packard via customer support, and i found out who these scammers are imitating
I would posit stupidity. Previous honeypots that weren’t takeovers of server operators have been somewhat targeted: Anom required a buy-in of the phone (as evidence you’re a criminal), Playpen required you be a pedophile (or at least, hanging out online with pedophiles) to be caught in the net, Hansa was a drug market, etc. Creating a general-purpose backdoored app to en masse catch criminals seems to cast quite a wide net when the same arrest portfolio can probably be gathered by doing the same thing to Twitter DMs with a backdoor and a secret court order. I wouldn’t put it past law enforcement but it seems like a mega hassle vs. targeted honeypots and backdoors.
If it were a honeypot (or backdoor), it’s certainly too much hassle for legitimate law enforcement purposes like the ones you described. You’d want this for someone you couldn’t reach through normal court (even a secret rubberstamp like FISA) channels.
This would be more like something you’d use for getting information from a group operating under some legal regime that’s not friendly to you gathering that information. Getting it in place, then convincing the group you were interested in to migrate off, say, Telegram, might be one approach.
The interesting thing in this case (IMO) here is that the fork removes things that were:
and without articulating the upside to their removal very persuasively. Sure, stupidity is always a possibility. But it feels more like they want to add some features that they don’t want to talk about. On the less-nefarious end of that spectrum, I could imagine that it is as simple as supporting devices that don’t work with the upstream, but that they don’t want to discuss in public. It’s also easy to imagine wanting to support some middle scanner-type box on a corporate network that PFS would break. But it could also be something like being able to read messages from a device where you can maintain passive traffic capture/analysis but can’t (or would prefer not to) maintain an ongoing endpoint compromise without detection. e.g. You have compromised a foreign telco and can pull long term key material off a device when its owner stays in your hotel, but you can’t or won’t leave anything running on there because the risk of detection would then be too high.
That’s just all speculation about when it might serve someone with an effectively unlimited budget to do something like this. Stupidity is certainly more common than such scenarios.
Hence “plausibly deniable.”
Only the first bit could charitably be attributed to “don’t roll your own crypto”. The rest was just obtuse idiocy or malevolence. Call the library-provided “encrypt this chunk with symmetric encryption using this key” then providing a public key.. that’s not about rolling your own crypto.
I haven’t followed Swift much recently, but the original Swift was simple because it punted on all of the hard problems. It had no mechanism for error handling (there were some small mods towards NSError). It had no support for concurrency and the everything-is-shared-mutable-state model made that really hard to fix (the structured concurrency and actor extensions are trying). It had no support for shared libraries (Swift’s shared library support now is really nicely designed and it’s the only modern language that I think has addressed this problem well, but that inevitably came with complexity).
For macOS development, Objective-C++ is actually a very nice language. C++ for local code that is specialised over strong types, Objective-C for late-bound interfaces that are exposed across module boundaries,
I have never used Objective C++, but from afar it sounds horrifying. Take a monstrously, ineffably complex programming language and another quite complex programming language and mash them together into an amalgamation. Can anybody hope to make sense of it?
Speaking as someone who used it for years, it actually works quite well. Obj-C is not very complex, and its OO stuff is pretty separable from the C part. Obj-C++ extends the C part.
I’ll also point out that modern Objective-C’s ARC (automatic reference counting) composes very well with C++’s object lifetime rules. In pure Objective-C with ARC,
is not allowed, because
NSString*is an ARC pointer and the compiler needs to be able to reason about its initialisation and invalidation. The semantics of C structs don’t allow for that, which can make implementing custom data structures tricky: you either have to do it all in the context of Objective-C classes (which has certain overheads), use indirection, or you turn off ARC in the relevant source files and do manual reference counting.The same code quoted above will compile on Objective-C++ however, because the ARC pointer is treated as having the usual set of C++ constructors, destructor, assignment operator, and so on - it’s not a POD (plain old data) type. This means the
structalso gets implicit constructor/destructor/operator implementations when not explicitly implemented.You can therefore shove pointers to Objective-C objects into all sorts of C++ data structures, that have been implemented without special Objective-C support, including the STL. It all composes rather nicely.
(A significant proportion of my work in recent years has been working on macOS device drivers and related apps, daemons/agents, etc.; I’ve mostly been using Objective-C++ on that, although I’ve recently introduced Rust in one part of such a project. My limited contact with Swift has been exceedingly frustrating, so I’ve avoided it where possible; it never appealed to me in the first place due to the reasons David mentioned, and the practicalities around extremely poor forward- and backwards-compatibility were a nightmare to deal with on a project where it was forced upon me.)
ARC certainly makes this easy, though prior to ARC I implemented a C++ smart pointer class for Objective-C references that did the relevant retain and release operation, which meant that this was already possible in Objective-C++ without ARC, it just became nicer because you didn’t need to use the smart pointer.
Definitely, you weren’t the only one to implement such a smart pointer class. I guess my point was more that modern Objective-C actually composes better with C++ libraries than it does with C libraries and data structures.
This made me curious. What was your use-case? iOS apps? Mac apps? Something else?
I also used it for years and we made Mac apps that spoke to hardware devices through the IOKit kernel driver API (which is in C++). It was indeed quite nice.
Mac apps, libraries for use in Mac/iOS apps.
There is a non-trivial amount of ObjC++ in Firefox, too, or at least there was last time I checked. For TenFourFox I used ObjC++ as glue code to connect up libraries.
To add to what others have said: a lot of the pain in C++ comes from trying to do things that are easy in Objective-C, and vice versa. With C++, it’s easy to create rich types with no run-time overhead, but that create tight coupling. With Objective-C, you have late-bound interfaces everywhere, but avoiding dynamic dispatch is very hard / hacky. The combination means that you can completely avoid things like raw C pointers. You can used C++ collections inside a module, Objective-C ones across the boundaries.
You should give Obj-C a try, I think! It’s a surprisingly thin layer on top of C, giving a lot of bang for the buck for writing dynamic (but fast!) programs. It’s quite unique in that you have two extremes: C on the one hand, and a fully dynamic OO runtime (you can determine implementation at runtime). Overall syntax is outdated and weird (smalltalk influence), but it is still unmatched in that niche.
It’s not bad in practice. The use case is, you need Objective-C system frameworks but you can’t do without particular C++ libraries. You still spend the bulk of application code in one language.
I had a game that used Bullet physics in this way. I migrated most of the code to Swift after it was introduced, but I kept some Objective-C++ in order to keep Bullet. These days Swift has direct C++ interop, both for library use and for gradual migration of C++ projects to Swift.
Probably people who have actually used it.
I worked with making an iOS app at one point and I found while thinking in Lisp like patterns it seemed to get out my way if I wanted it to. But that is a beginner and greenfield perspective for sure.
I don’t think it sounds too bad, but I haven’t used it myself.
My understanding is it’s just the extra OOP bits from Objective-C overlayed on C++, similar to how it was overlayed on C in the first place. Basically just a second, independent object system. I understand why people wouldn’t like that, but it doesn’t sound too different than C++/CLI or using C++/JNI.
Can you tell us more about why Swift’s shared library support is well-designed?
With the caveat that I’ve read their design docs, but not actually used it in anger:
They make a clear distinction between ABI-stable and ABI-unstable shapes of structures. Within a library, there are no ABI guarantees. At a library boundary, you have a choice whether you want to sacrifice some performance for the ability to change a layout later, or sacrifice flexibility for performance. This is a per-structure choice. Depending on the choice that you make, the compiler either lowers to something similar to Objective-C non-fragile ivars, or C struct fields.
They actually have a language notion of a library boundary. This means that generics can be statically reified within a library, but fall back to dynamic dispatch across library boundaries. Contrast this with C++ where templates either have to live in headers (and then end up copied in every compilation unit, including the implementation, and it’s a violation of the one-definition rule to link two libraries that use different versions of the same template) or they are private to a library. The Swift model gracefully falls back. Things may be faster inside a library, but they still work from outside, and the ABI doesn’t leak implementation details of the generics, only their interfaces.
Wonderful explanation, thank you!
An overview from Rust folks: https://faultlore.com/blah/swift-abi/
Hmm…I’d say it was already incredibly complicated despite punting on a lot of hard problems and largely because it tried to codify “solutions” to non-problems into the language. Which never works, because reality, even code reality, is way too messy for that.
As an example, I wrote about the mess that is initialization back in 2014, so right after Swift apepared. There was some pushback from a prominent member of the Swift team saying that my goal of simplicity just wasn’t compatible with some of the goals they had. Hmm….
There was also a great rant by a prominent member of the community about Swift being just a mess of special cases upon special cases. I think he left the Apple ecosystem, and he’s not the only one. Alas I can’t find it and I don’t remember the name.
Anyway, I predicted back then that because they had so many language features for initialization it would never actually work out and there would be a renaissance of builder and factory method patterns and there would be even more language features for initialization. Five years later: hello SwiftUI! :-)
So the failure of Swift now isn’t surprising, the trajectory was already set in stone the day it launched and there wasn’t really much one could have done about it afterward…much less so since the same faulty reasoning that led to the initial problems was still present and guided later evolution.
I think this is an instance of correlation not being causation? My understanding is that the actual cause of SwiftUI is the successful design of Flutter (which gave raise to both SwiftUI and Kotlin Compose), and it is relatively orthogonal to language machinery.
Case in point, Kotlin’s initialization story is much more tame than Swit’s one (as it doesn’t try/doesn’t need to prove initialization safety statically), but it also converged on the essentially same design (or rather, vice-verse, IIRC Kotlin work in the area predate’s Swift’s).
Not to disagree with your wider point on constructors, which I agree with, just to point out that SwiftUI is not I think a particularly strong argument here.
I think you might want to have a look at the actual article. Swift introduced yet more special syntax for the part of SwiftUI that creates the view tree. So yet more language features for yet another way of constructing views^Wobjects^Wstructs.
The more general problems with SwiftUI (and related) are another issue, which I talk about a little bit here: UIs Are Not Pure Functions of the Model - React.js and Cocoa Side by Side
Last I checked the inspiration for Flutter and SwiftUI etc. was React.
I have read the articles! If I understand your argument correctly, it says that the fact that they needed to add new stuff to support Swift UI means that the original rules were inadequate. My counter-argument is even languages that don’t have Swift-style maze of initialization rule add special cases to support SwiftUI patterns. Ergo, adding stuff for SwiftUI is orthogonal to your normal way to initialize objects. In other words I claim that, in counter-factual where Swift doesn’t have complicated initialization rules and uses Java/Go style “everything is null to start with” or Rust/ML style “everything starts with all the parts specified”, it would have added more or less the same features still for SwiftUI.
The story is even more illustrative with Kotlin — it was specifically designed for DSLs like SwiftUI/Compose. The whole language, with its second-class-lambdas, extensions, and coming out-of-fashion implicit
this, works towards that goal. And yet, when the actual UIs started to be implemented, it was quickly apparent that no one wants to write+button(), and a bit more of compiler special sauce is needed for nice surface syntax.I must be a lousy communicator, because you seem to have misunderstood the article almost completely.
The point was not that Swift has the wrong initialization rules or too many of them. The point is, as it says in the title: “Remove features for greater power”. The many initialization rules are not the source of the problem, they are a symptom of the problem.
The problem is trying to bake this stuff into the language. As a consequence, you get 30 pages of initialization rules. As a further consequence, those 30 pages will be forever insufficient.
So for me, the supposed counter-point you bring with Kotlin actually supports my point. You write:
So they baked a whole bunch of features into the language to support the DSL use case. What was the title of the blog post again?
So they added a lot of features into the language especially for this use-case and it didn’t even work out for this particular use-case. Surprise surprise!
I simply don’t think the static/compiler-oriented mindset is compatible with the sorts of things these languages are trying to do. You put way too much into the language/compiler, and you do it way too early.
Ruby has had a bunch of these kinds of frameworks, and as far as I know they did not require any changes to the language. Because Ruby had fewer but more flexible features to start with.
https://github.com/AndyObtiva/glimmer
With Objective-S I seem to be violating that rule, because it certainly does put things into the language. Or at least seems to do so. What I am doing, however, is following the second rule: “don’t do it yet”. (With quite a bit of trepidation, because it is “experts only”).
And I am not actually baking all that much into the language. I am baking a bit of useful surface syntax and the associated metaobject-protocol into the language. What lies behind those metaobject protocols is quite flexible.
So far this appears to strike a good balance between providing some syntactic convenience and compiler support while not making the mistake of baking way too much into the language.
Indeed! I misunderstood your original comment as meaning that SwiftUI is a downstream consequence of initialization rules. I agree that both are rather the result of the lack of expressiveness, which doesn’t allow the user to “do it yourself” in userland code. The Kotlin example was exactly to illustrate that point.
Found it: Which features overcomplicate Swift? What should be removed?, by Rob Rix.
How to fix:
So I certainly wasn’t the only one who accurately predicted the current sad state of affairs. It was extremely obvious to a lot of people.
And silly me: of course I had referenced it an another one of my posts: The Curious Case of Swift’s Adoption of Smalltalk Keyword Syntax
It’s another example of complexity begetting more complexity, special cases begetting more special cases.
I am if the opinion that the only avenue for making a well rounded language that lasts is to put all of the skill points towards a small handful of skills (“fast”, “fast to compile”, “ease of dynamic linking”, “easy to use”, “easy to learn”, whatever) where each “skill” can be thought as a different axis in a graph. Go all in on a few niches. This helps with clarity of purpose for the project itself, a clear pitch for prospective users, and makes it possible to show the benefits for that niche. Without doing this the evolution of the project ends up rudderless, and adoption can stagnate either slowing down or killing the project. Once you’ve reached critical mass, fight for dear life not to abandon those hard earned early design decisions that allowed you to carve a niche, but go all-in on a handful of other axis as well. Rinse and repeat. A language can start as easy to use and once it has a clear base experience it can start trying to produce faster binary. Or it can start as fast to compile and afterwards try to produce optimized code. Or it can start producing super optimized code and later try to become faster to compile or easier to use. All the early design decisions will hamper the later pivot, and that’s ok! No language will ever be all things for everyone (barring some unrealistic investment in monstrous amounts of research, development and maintenance).
Swift had a clear initial view of what it wanted to accomplish, and did so successfully (I believe, don’t have personal experience with using Swift, nor do I keep up with its development). It is now on its expansion era, to allow for niches that previously were either difficult or impossible to cater to. It is a delicate process that can easily go haywire because of subtle interactions between completely unrelated features can make the result worse than the sum of its parts. But I can’t blame them for trying.
How do you see that wrt to Rust and async?
I could maybe argue that sync Rust had excellent “early design decisions that allowed you to carve a niche”, but the async related problems are now occupying a vast majority of the brain time of the lang teams. While I don’t think we’re heading to 217 keywords, I certainly feel uneasy seeing “keyword generics” and similar efforts.
The Acorn image editor. Great for people like me who doesn’t need Photoshop just to do some light image editing. To me, this is the rightful successor of Paint Shop Pro.
Obsidian. The UI is not perfect, but I still like it a lot. IMHO, the best note-taking application after Evernote jumped the shark. Everything is stored as plain Markdown files, so lock-in is not a concern.
The Fish shell. Just a pleasure to use. Easy to get started with. IMHO, it’s so much more convenient than Bash.
Preview. Love to have this application included in my OS. Currently using it to read Adam Langley’s A Tour of WebAuthn.
Preview is the app I miss the most when I’m on Windows/Linux. I know there are alternatives but they aren’t a patch on Preview.
I completely agree. I dual boot a Linux-based system with macOS, and Preview.app is the only reason I ever reboot into macOS.
Usually I can get by with pdf-arranger and okular and sometimes zathura… but each has its own problem. I just want a PDF viewer that allows me to reorganize pages, annotate them, crop them visually (e.g. not specifying the cropbox as coordinates), and yeah.
Re Obsidian has no lock-in: I am using an iPhone and Ubuntu on the desktop. Do you know of anyway to use Dropbox (or equivalent) for syncing and still allow editing via their iOS app?
Since their own sync storage seems to be an USP of their paid tier, I guess I’m out of luck, but it would be very interesting to be proven wrong.
Not sure if I would call that lock-in – IMHO, lock-in is more about locking in the customer’s data without any reasonable export possibilities, which is not the case with Obsidian.
I can warmly recommend an Obsidian subscription, it works really well.
However, what you want to achieve might be possible without a subscription. I would search for ⟬obsidian syncthing iphone⟭ and ⟬obsidian git iphone⟭. Best of luck!
Emacs, it’s always Emacs for me.
The fact that Emacs has been under active development for so many years is nothing short of amazing, IMHO.
Sometimes one needs to vent, which is fine. But as a Hugo user I can’t relate to this. I was frustrated with Hugo before I understood how it works. After I got over that hurdle, all I need to do to write a blog post is to write some Markdown and then push to my Git host of choice. That workflow suits me very well. YMMV.
With all respect to fish(1) … its not a good interactive shell.
Example: https://pbs.twimg.com/media/GfRKRSGWEAAlRuR?format=jpg&name=medium
Not to mention lack of POSIX syntax for while(1) or for(1) loops …
The zsh(1) shell is way more usable for interactive work:
https://vermaden.wordpress.com/2021/09/19/ghost-in-the-shell-part-7-zsh-setup/
We are lots of people who’ve enjoyed using Fish shell for years. If you have another favourite shell, great for you.
“it’s not a different shell that has different syntax” is a pretty poor argument. It requires learning.
Fish has its own paradigms and workflows. But if you really want, recent versions of fish have a one-liner you can use to enable support for
!!Good to hear that.
Maybe in the future POSIX syntax and
!$will also be supported via various workarounds … or implemented.Confused. What am I supposed to under from that screenshot? Yes, fish is not zsh. I don’t see a single example of interactivity there. All I see are commands and their output.
Many fish users don’t use ls at all. Myself included. I can navigate and enter the directories interactively with tab completion. Ls becomes unnecessary.
TIL, Forgejo is apparently a fork of Gitea. [source]
…which is a fork of Gogs.
And you can configure it to use Gerrit for code review, which is very nice
The one thing @soatok does not address at all is the original use case that PGP was designed for, as opposed to all the other ones people started using it for in time between when it was designed and when better options came along: encrypted email.
The closest they get is saying to “use signal” for secure messaging. But I do not trust signal and none of the reasons relate to the need to give them my phone number. And while @soatok correctly points out that my phone number need no longer be revealed to people who want to communicate with me, it still looks like signal-the-org demands that I give it to them.
If you want people to stop using PGP, you need to provide an alternative for its original purpose. And an alternative that requires me to send all my stuff through one org, even if I didn’t need to give them my phone number (and it looks like I do) is not adequate.
I know S/MIME addresses some of the concerns one might have about PGP email, but I wouldn’t say it’s better on balance, and I don’t think S/MIME as implemented in those mailers that still implement it addresses any of the criticisms from this article in any way.
Every time I see an article like this, I get optimistic that there will be a reasonable proposal for encrypted email. There never is.
The 2019 Latacora post handles this directly.
Aside from Signal, Soatok seems to address this use case (private messaging) from a different direction:
I see where they are coming from, but I find “don’t” an unacceptable answer to “how do I encrypt email?”
I am optimistic that work on the fediverse angle could yield an acceptable answer for email style asynchronous encrypted messaging, I’ll certainly be paying attention there.
But for now, I’m more comfortable trying to train a small number of collaborators on s/mime opsec than I am trusting signal. That’s because I know the pitfalls of s/mime better than I know the pitfalls of PGP. But I’d still be more comfortable trying to train a small number of collaborators on PGP opsec than trusting signal.
My understanding is that to get to the point of asking about whether you should encrypt email, you first have to make the fundamental mistake of trying to use email for secure messaging. And so the answer really is not “don’t encrypt your email”, it’s “don’t use email for secure messaging”.
All attempts to do encrypted “secure” email are basically just theater. There are so many weak links, technical and social, in the email infrastructure that you can personally set up the most bad-ass hard-ass ultra-mega-NSA-mil-spec XP++++ Pro grade encryption on your end, but any recipient who sees the plaintext can still quite easily quote-reply with the plaintext, or forward it, etc.
So PGP is more the cherry on top of the fecal sundae that is trying to make email into an acceptably secure messaging system. Which is why the answers always try to redirect you to an actual secure messaging system.
Should I say TLS is the cherry on top of the fecal sundae that is trying to make TCP an acceptably secure transmission system, just because there are a whole pile of ways that one party or another can terminate TLS in the wrong place and turn it into security theater? Back when I used to do pen tests, one particularly fun trick was to cause Firefox or IE to log its session keys to a place where I could grab them. It was at least as good as most of the email fun you could pull off, and got you a lot more data.
I opened the bug on the Evolution (GNOME email client) tracker about how it silently turned off encryption on encrypted email messages when you replied, even if you had a public key for everyone else on the thread. I am 99% certain it took over 10 years for that to get fixed. (I believe it was over 15, but I can’t find the notification from when it was finally closed on this laptop.)
I viscerally appreciate what you’re saying about email not being fit for this purpose.
But until we get a modern answer with the properties of email that doesn’t involve me trusting signal or whatsapp, it’s easier for me to work around the deficiencies of email in this regard.
I’ve tried twice now to set up a secure, closed messaging system, which is an easier version of what I’m saying I want here. I tried once using email infrastructure and once using XMPP infrastructure. Email was smoother. I’d like a better alternative.
Leaving your door wide open because you don’t like a particular brand of lock does not strike me as a great security stance.
But then I also don’t know why you have issues with “trust” with Signal, which is open source.
Signal is not meaningfully open source. A single entity controls the (not open source) server, and the production of all binaries anyone uses.
Isn’t the entire point of client-side end-to-end encryption that it shouldn’t matter what the server is doing?
And the whole binary thing is kind of endemic to the mobile ecosystem for regular users (i.e., people who have never heard of F-Droid outside of possibly some spicy fanfics).
Sure, but the key distribution is done by their server for example. Lots of important parts under their control despite the e2ee
By “key disytibution”, are you referring to SignedPreKey bundles, which are signed by the IdentityKey stored on the user’s device?
Because this is an X25519 public key and a signature.
Calling it “key distribution” is vague and makes it almost sound sinister.
Who cares if their server is doing key distribution? Surely you’re not trusting any server’s key distribution, federated or not, if you really care about message security?
You can get someone’s signal public key without trusting signal’s server? How?
You can’t.
Safety numbers exist, but that’s a sucky mitigation.
Key transparency is a much better mechanism, but only WhatsApp has it.
but if the server were to provide the wrong public key, it would still be detectable right? can’t you still compare your public key to the public key that your contact has stored for you, if you are with them in person? wouldn’t that relieve you of the need to trust signal’s servers?
short of manually comparing keys, if you trust the client software on both ends, then exchanging messages and verifying in person that they were received should confirm that you have the right public keys on file, right? so that’s another way that you would not have to trust the server.
How often do people actually do this?
probably never. I just think we should be careful about saying “you can’t” exchange public keys without trusting Signal’s server. besides, is WhatsApp’s “key transparency” any different?
Yes. The security model of “key transparency’ is that you can build automation that detects when an errant public key has been issued, or if the public key received isn’t in the ledger. You don’t have to rely on squishy humans to do anything.
Additionally, it creates a disincentive for the kind of nation state actors that might perform such an active attack against a target of interest. Whereas we see mechanisms like National Security Letters with a built-in gag order, key transparency prevents stealth. In order to attempt to attack the other person’s communications, you have to first announce yourself to the network.
(This guarantee is much stronger with open source software and reproducible builds, of course.)
is that basically the same as a PGP key server?
is the WhatsApp key directory / audit record list actually publicly available? I haven’t been able to find it.
Nope. The PGP key server can lie about a lot of things. There is no audit trail (i.e., Merkle Tree proof of inclusion) to guarantee a time period. There are no witness co-signatures. There’s nothing.
For a while, GnuPG wouldn’t even bother to check the fingerprint of the received key versus what was requested.
Interesting. So does the WhatsApp public key directory actually exist or did they just write a blog post about it?
I take it it doesn’t exist?
@caleb, I’m currently visiting family for a much-needed vacation from technical work. Whether something exists or not is something you’re free to explore on your own. As I have no financial stake in anyone believing anything, I’m going to prioritize spending time with my nephews and drinking enough eggnog to make my physician give me the angry teacher stare when I return in January. I wish you the best.
I’m replying to you here (not the user you’re replying to, but I was interested when I saw you bump it), because it turns out that’s actually the max reply depth and so no-one can reply to you.
I googled around, found the key transparency whitepaper, which includes a link (page 11) to the audit proofs.
Fwiw, it appears Cloudflare audits these as a matter of course, now, and have a dashboard showing the status of their auditing.
I was surprised to learn there’s a max depth to replies but not surprised @caleb is the one who plumbed it.
that’s really helpful! you’re a better googler than I am.
I thought the server was open source? (AGPL-3.0 license): https://github.com/signalapp/Signal-Server Seems to get actively published to, at the time of this writing the last commit was 5 hours ago.
I seem to remember some people have verified the client binaries for Android at least once. I can’t seem to find a link for it though, perhaps my memory is faulty.
The server is open source except for the spam filter, which is pretty reasonable. I believe all the other server-side components, particularly SGX enclave implementations, are also open source (though I didn’t double-check).
I’m not leaving the door wide open. Like I said, it’s a closed system that we’re building on email protocols. We use filtering to prevent the kinds of accidental leaks you’re discussing in other messages. We hand out specifically configured clients. And we use servers configured to block plaintext messaging. We tried something similar with XMPP, but it was easier on the people who were responsible for infrastructure using email things, and people using the system liked the email interface better.
One of the reasons is because in order to sign up, I have to surrender a phone number, and they are not open enough for me to tell what they might be doing with that. There’s no way for me to see what they’re actually doing on the server side so I can infer it, either.
Email requires you to “surrender” huge amounts of useful information – enough that the secret police might not even need to decrypt the body of the messages to get enough evidence to convict you.
But sure, a phone-based tool that initially used phone numbers (disposable ones are easily available, or so I hear) to avoid the need for centralized storage of things like contact lists and personal data is the real risk, and email is just fine an peachy and dandy. Now let me just go see who’s knocking so loudly at mydoor…
But I can run my own email server. And it’s trivial to give an account on there to anyone whose phone number I have (which is how you found people on Signal, last time I tried it…)
I’m not worrying about surrendering info to myself.
Thanks for the thought exercise. I would probably consider signal a great replacement for email if I could stand up my own server and have my contacts connect their signal apps to it. I suspect that’s a smaller lift now than it used to be, since they’ve started publishing server code again, and might hopefully even simply be a skill issue at this point.
So you can do your own, more complicated and more easily seized thing that still likely leaks plaintext metadata to the secret police, and you think that’s better than something that uses a phone number – which you can easily obtain as a disposable resource – to bootstrap itself without requiring a centralized account system.
I have admittedly not yet tried very hard, but every place around here that I’ve checked requires a government ID for this.
Looking through the documentation on signal’s site, I don’t see anything explaining why they need it, nor making me expect that it’s disposable. Don’t I need to keep that number as long as I have my signal account?
They do a great job covering the crypto, which is good. Their privacy documentation does not explain, at least in any place that I’ve yet found, what they’re doing with my phone number and how they prevent it from being metadata associated with my encrypted messages.
Signal and their infrastructure is in the same physical jurisdiction I’m in. So police that are interested in me can go after things in Signal’s hosting facility. I can host my own stuff in a different physical jurisdiction, where it’d be less easily seized than Signal’s. That is part of the appeal of self-hosting.
And our police aren’t secret. There are just signs that they’re becoming more worrisome.
Our conversation has persuaded me to continue learning about signal. I know you’re a serious person, and your technical judgement often aligns with mine, so I want to see if it’d work for what I’m putting together.
A thing I’m finding missing from the signal software is the ability to use hardware tokens for crypto. I’d like to have my private keys stored on an external token, and I don’t see a way to do that even if I spend time learning to self-host signal. It looks like that just isn’t a feature they offer, even for their desktop software.
It’s starting to look like, even if my trust issues could be resolved, what I want to do is simply more complicated than what signal supports at the moment.
I’d love to contribute to building something that uses signal’s crypto and protocols with hardware keys at the endpoints, an interface and behavior that’s more like email than SMS (particularly as regards attachments), coordinated on a self-hostable server.
But I need to get this bootstrapped at an in-person meeting that will take place between mid-December and late January. It’ll be a “party” where we distribute tokens that I’ll have sourced, generate keys, configure wireguard and FDE on everyone’s PCs, connect via wireguard to an email server which only accepts traffic over that wireguard connection, doesn’t relay traffic to other email servers, and refuses all plaintext mail.
I don’t think the thing I’d like to contribute to can be ready in that timeframe, and I don’t see where it’s been built already for me. I do think I can get the server I’ve described hosted in an appropriate place and configured.
I don’t think we can have a productive conversation about this because I fundamentally don’t think your apparent preference for spending huge amounts of effort kinda-sorta securing a thing that’s fundamentally un-securable is something that I can engage with or talk you out of. The lengths you’re prepared to go to in order to avoid the possibility of “just use a burner phone” frankly astound me.
We probably can’t. Your insistence on dismissing features like “hardware tokens for private keys”, “richer messaging than SMS (or, we need real attachments)” and “hosted in another jurisdiction” as if it boils down to “not wanting to use a burner phone” means we’re talking past each other.
Your confidence that reasonably anonymous burner phones can be had here is also, curiously, much higher than mine.
But thank you (sincerely!) for discussing it this far.
what makes @hoistbypetard’s proposed system “fundamentally un-securable” and why don’t the same fundamentals apply to Signal? it sounds like his proposal can be more secure because keys and software is set up in-person. aren’t the fundamental limitations more on the Signal side because they are targeting remote key and software distribution, which is much harder to secure? I would be really curious if you could unpack this to any degree.
This was covered in the article. Adding someone on signal only requires a username these days. A phone number is required to sign up, which is damn easy to buy with cash in most parts of the world.
That’s good. Do you know why signal still needs a phone number from you, though?
Last time I tried to buy a phone number with cash here, the person selling it was pretty insistent on having a copy of a government-issued ID. This was on the east coast of the US. Had I cared more in that case, I think I very likely could’ve got one somewhere else without showing ID, but the ID requirement and the cameras all around the store didn’t make being able to use cash feel like it improved anonymity in any meaningful way.
For my application, I don’t actually mind giving my phone number to the people I’m messaging, and vice versa. We’re all within 2 degrees of each other IRL. I am not yet comfortable with signal having mine and my contacts’ numbers associated with each other.
Many places I have been require a copy of your passport to get a SIM card.
Exactly the same is true of signal though. Someone could screenshot your message and share that. If “the recipient could choose to willingly leak the plaintext” is in your threat model, there’s not much you can do.
In the case of Signal, the recipient has to choose to leak it and take active measures to do so.
In the case of email, it’s far too easy for the recipient to accidentally leak it, because they might not have the same super-ultra-opsec config the sender does.
Signal pushes people onto phones, which are more likely to be used in public places where people can see your messages over your shoulder.
Signal also support iOS where the binaries cannot be verified against the published code, so E2EE doesn’t mean much there.
The “just use Signal” mantra also doesn’t help because people are given a false sense of security. With email at least there is a general awareness of the need to be careful and consult a nerd if you actually want to protect your communications.
Smartphone users take screenshots of text all the time without thinking twice, and default smartphone configurations increasingly upload every file to the cloud automatically.
Excuse me, but I find some of these arguments quite weak.
It is true that Signal pushes people onto phones, but if other people glancing at your phone poses an actual threat, the problem is not the smartphone. If you are in that situation you need a well-calibrated threat model and a considerate OPSEC regime. There’s no way around that. And if you need it, it’s not like it’s hard to find a spot where glancing is impossible. Sit alone in a corner or go to a bathroom.
I think Signal has good reasons for the “smartphone first” paradigm.
For most people these days, the smartphone is their primary computer. It’s more common to have a smartphone than a laptop or desktop. One of the primary goals of Signal is to be accessible and that means being available on the most popular devices.
Smartphones are also more locked-down and more secure than general-purpose computers. I personally prefer using a laptop for most tasks, but that’s not for security reasons – I just find it more convenient and ergonomical.
I don’t believe it’s anywhere near reasonable to claim that E2EE is pointless on iOS. I don’t find this argument serious. AFAIK it’s still possible to use IDA Pro or Ghidra to analyze an iPhone app. Reproducible builds would be nice, but it’s not reasonable to claim E2EE isn’t possible just because reproducible builds aren’t available on a certain platform.
I think this is a good point! Again, for people where security matters, OPSEC is crucial.
The link doesn’t really explain how or why that’s the case. It also has statements like above which seem counter-intuitive without enough context. Is that to generalize your digital fingerprint or something similar?
@ubernostrum said the user has to take deliberate measures to leak data with Signal, but is liable to leak data by accident with PGP. so we are talking about users who don’t have a considerate OPSEC regime, and I’m pointing out that Signal is not foolproof for those users either. we’re saying the same thing.
it’s not just “smartphone first,” it’s “you absolutely need a smartphone to use Signal.” the organization is not starved for resources, and they have decided to implement useless things like sealed sender when they could have implemented a non-smartphone SMS workflow for account creation and authentication. it certainly doesn’t seem like accessibility is a primary goal.
not for the threat models that Signal claims to accommodate.
are you saying backdoors are detectable in iOS apps? that is news to me.
the benefit of E2EE is that you don’t have to trust the service not to log your communications. if the service is not trusted, then what reason is there to believe their claim that an app uses E2EE and is not backdoored? under what threat model would you not trust a service to scan your communications if they passed through their servers in plaintext, but you would trust them to faithfully provide binaries with E2EE that is not broken?
and yet you still think smartphones are more secure?
Why do you feel Sealed Sender is useless? This is actually one of my primary examples of Signal’s centralized model having advantages - they were able to simply eliminate a swath of metadata unilaterally.
I wrote a comment about it here.
I could easily be missing something; if you think I am I would love to hear it. I think the part about the VPN setup might actually be wrong and overly generous to Signal.
No, it’s not exactly the same. Screenshotting a Signal message and sharing that is a deliberate act. Replying or forwarding a decrypted PGP message in plaintext is done all the time by accident.
Quoting from the Latacora piece https://latacora.github.io/blog/2019/07/16/the-pgp-problem/#encrypting-email
it’s also perfectly feasible for a smartphone user to screenshot a message for reference and have it automatically uploaded to the cloud by their smartphone.
the evidence for the problem on the PGP side is “we don’t know a PGP email user who hasn’t seen this happen,” but in the smartphone case you wouldn’t see it happen which is even worse. we don’t have the data to say that it’s an issue in one case and not the other.
sure, you can take screenshots on a computer, but you are not encouraged to by a horrible touch based UI and configurations that automatically suck your files into the cloud are much less common.
Isn’t simply sending attachments in age format functionally equivalent to PGP email?
No. PGP email and S/MIME email clients attempt to solve problems that age does not attempt to solve. The interesting one, IMO, is associating public keys to recipient identities.
To use age for someone other than yourself, you need to get their key out-of-band and encrypt for it:
That’s part of what PGP and S/MIME clients are doing for you.
I don’t know what you mean by “out of band” for PGP — or rather, I don’t know what you would mean by “in band”. Practically speaking, whenever I’ve used PGP mail, I got the public key from a webpage or other “out of band” (?) source. The “web of trust” has never really worked in a general way. So there doesn’t seem like much of a difference, except there’s a local key store, and it would be pretty easy to leverage systemwide keychains now that OSes have them.
“In-band” vs “out-of-band” wasn’t quite the correct term. I mean that I could connect a PGP client or an S/MIME client up to a directory server of some sort, or that I could configure them to trust certain key signers, and they can encrypt messages for a recipient without me having to paste a key for each recipient in.
Whereas what’s being described for age means that I’d need to have each person’s key somewhere, paste it into the command, etc.
The email clients are trying (or did try) to solve this and age clients are not (yet).
That’s all I meant.
Understood. In an ideal world there is an integrated and secure experience between mail client, directory/keyserver, and local keychain. PGP clients do this in theory, but because there’s no trusted directory (soatok’s Public Key Directory project) the practice falls short of the theory.
In the spirit of
age(just make the right choices and version the whole thing) perhaps there’s a combination of newer building blocks that would give a smooth experience with better security and practical interoperability. I mean, other than Signal.From the article:
I dunno, seems like this is something the author is working on solving to me?
It sounds like something they “plan” to work on. It doesn’t sound like something I can tell someone to use now instead of PGP. Which is what the title of the article promised.
Right, but this isn’t a common use-case.
Most people, 99% of the time, just care about things like backups, sending files, or signing things. They don’t care about binding the trust relationship between a public key and an “identity” (term left vague due to different social interpretations of the word).
The article delivers on the things people care about, and says “I’m working on it” for the nerd use cases.
Probably it won’t. Now we can try to understand why, before creating yet another tool/system to “kill” PGP.
I’m not sure I would say that PGP/GnuPG was ever very wide-spread and it certainly hasn’t been gaining traction for the past 15 (20?) years or so. In 2024, I won’t hesitate to say that GnuPG is basically dead.
I have a lot of respect for what Phil Zimmermann did in the 90s, but that was a long time ago – cryptography has made huge leaps since, especially when we talk about applied cryptography. GnuPG is simply not very relevant these days. We have better tools for the problems that GnuPG attempted to solve. The exception being an e-mail-like system with good security properties – which is also discussed in the article.
If PGP were effectively dead, I guess we wouldn’t read articles like this every couple of months.
Yes, the article mentions 9 of them.
I’m not an expert in this space, but my interpretation of this fact is not that “it takes nine separate tools to equal what GnuPG can do,” but rather “GnuPG performs nine different functions that—with the benefit of hindsight—did not need to be mashed together into one tool.”
Yours is the correct interpretation.
Yes, and I agree with the interpretation. That’s the point that everybody has been making during all this time.
However, the initial question was:
And my observation was that we need to try to understand why it hasn’t.
The point being, we need to understand why all these features “wrongly”
mashed together into one tool, are still being widely used, when theoretical better alternatives focused on a single problem exist. Even when there has been a considerable effort to promote them and “educate” the public.If we figure that out, probably we can move forward.
One explanation, covered in the original Latacora article, is that PGP “is also a social network, and a subculture.”
The real world security of PGP has not been publicly tested, because it’s not very widely used, the people using it are mostly people not targeted by law enforcement, and its UX is not amenable to easy use by people who are targeted by law enforcement in the developed world, namely criminals.
Ladybird browser is doing an amazing job making contribution look appealing. There is a YouTube series showing some debugging and development for it that gave me a programming itch to scratch.
SerenityOS is another cool project (and for those who aren’t familiar with it, that project is the origin of Ladybird).
Heh. I have the opposite impression.
After seeing the creator’s response to this PR, https://github.com/SerenityOS/serenity/pull/6814, it makes working with him look really UN-appealing.
I have a firm “never work with @$$h0l3s” policy these days.
When I’ve tried emulating X86-64 on Apple Silicon using QEMU it’s been incredibly slow, like doing
lstook like 1-2 seconds. So if these fine people manage to emulate games then I’m very impressed!QEMU emulation (TCG) is very slow! Its virtue is that it can run anything on anything, but it’s not useful for productivity or gaming. I used to use it to hack around a FEX RootFS as root, and even just downloading and installing packages with
dnfwas excruciatingly slow.Emulators that optimize for performance (such as FEX, box64, and Rosetta, and basically every modern game console emulator too) are in a very different league. Of course, the tradeoff is they only support very specific architecture combinations.
As @lina says, QEMU is general. It works a few instructions at a time, generates an IR (TGIR, which was originally designed for TCC, which was originally an IOCC entry), does a small amount of optimisation, and emits the result.
Rosetta 2 works on much larger units but, more importantly, AArch64 was designed to support x86 emulation and it can avoid the intermediate representation entirely. Most x86-64 instructions are mapped to 1-2 instructions. The x86-64 register file is mapped into 16 of the AArch64 registers, with the rest used for emulator state.
Apple has a few additional features that make it easier:
FEX doesn’t (I think) take advantage of these (or possible does but only on Apple hardware?), but even without them it’s quite easy (as in, it’s a lot of engineering work, but each bit of it is easy) to translate x86-64 binaries to AArch64. Arm got a few things wrong but both Apple and Microsoft gave a lot of feedback and newer AArch64 revisions have a bunch of extensions that make Rosetta 2-style emulation easy.
RISC-V’s decision to not have a flags register would make this much harder.
There are two more hardware features: SSE denormal handling (FTZ/DAZ) and a change in SIMD vector handling. Those are standardized as FEAT_AFP in newer ARM architectures, but Apple doesn’t implement the standard version yet. The nonstandard Apple version is not usable in FEX due to a technicality in how they implemented it (they made the switch privileged and global, while FEX needs to be able to switch between modes efficiently, unlike Rosetta, and calling into the kernel would be too slow).
FEX does use TSO mode on Apple hardware though, that’s by far the biggest win and something you can’t just emulate performantly if the hardware doesn’t support it. Replacing all the loads/stores with synchronized ones is both slower and also less flexible (fewer addressing modes) so it ends up requiring more instructions too.
Dumb question: is there a reason not to always ahead-of-time compile to the native arch anyway? (i believe that is what RPCS3 does, see the LLVM recompiler option).
As I understand it, that’s more or less what Rosetta 2 does: it hooks into mmap calls and binary translates libraries as they’re loaded. The fact that the mapping is simple means that this can be done with very low latency. It has a separate mode for JIT compilers that works more incrementally. I’m impressed by how well the latter works: the Xilinx tools are Linux Java programs (linked to a bunch of native libraries) and they work very well in Rosetta on macOS, in a Linux VM.
The Dynamo Rio work 20 or so years ago showed that JITs can do better by taking advantage of execution patterns. VirtualPC for Mac did this kind of thing to avoid the need to calculate flags (which were more expensive on PowerPC) when they weren’t used. In contrast, Apple Silicon simp,y makes it sufficiently cheap to calculate the flags that this is not needed.
Rosetta does do this, but you have to support runtime code generation (that has to be able to interact with AOT generated code) at minimum because of JITs (though ideally an JIT implementation should check to see if it is being translated and not JIT), but also if you don’t support JIT translating you can get a huge latency spike/pause when a new library is loaded.
So no matter what you always have to support some degree of runtime codegen/translation, so it’s just a question of can you get enough of a win from an AOT as well as the runtime codegen to justify the additional complexity.
Hah! That’s cute. Can’t remember the last time I saw Google US making a joke like this.
Is there an RSS feed for Ladybird’s blog/announcements?
AFAIK: No. But rss2email could be an option.
As the name implies, that tool sends RSS via email. Is there an email2rss you had in mind and meant to link to?
Ah, sorry. Yes.
https://kill-the-newsletter.com/
From the headers in the HTML file: https://ladybird.org/posts.xml
Any decent feed reader should be able to extract that if you submit the web page.
that doesn’t include these newsletter posts though, which they seem to treat as a separate thing from the main “news”
Ah, sorry I didn’t bother to actually check what the XML file contained. I expected it to be a standard feed, but it’s probably just some vestigial remnant generated by the CMS they use.
Yeah, unfortunately this article isn’t in the feed.
It is hard to recommend [anything but AWS]
sad but true for large cloud providers
more niche cloud providers like cloudflare and fly.io and the like are making inroads, though
AWS is reliable but too coarse compared to the Google Cloud Platform The engineering productivity cost is much higher with AWS than with GCP.
With the understanding that I might just have been lucky, and / or working for a company seen as a more valuable customer … in a previous role I was a director of engineering for a (by Australian standards) large tech company. We had two teams run into difficulties, one with AWS, and another with GCP.
One team reached out to Amazon, and our AWS TAM wound up giving us some great advice around how we’d be better off using different AWS tech, and put us in touch with another AWS person who provided some deeper technical guidance. We wound up saving a tonne of money on our AWS bill with the new implementation, and delighting our internal customers.
The other team reached out to Google, and Google wouldn’t let a human speak to us until we could prove our advertising spend, and as it wasn’t high enough, we never got a straight answer to our question.
Working with Amazon feels like working with a company that actually values its customers; working with Google feels like working with a company that would prefer to deal with its customers solely through APIs and black-box policies.
Indeed. If you think you will need a human to get guidance, then GCP is an inferior option by a huge margin.
Where in my experience, guidance can include “why is your product behaving in this manner?”.
What does “too coarse” mean?
I’m not sure I believe you that the “engineering productivity cost” is higher with AWS: How exactly did you measure that?
I have a Docker image (Python/Go/static HTML) that I want to deploy as a web server or a web service.
GCP is far superior than AWS on this measure.
I’m still confused. What do you have to do in AWS that you don’t have to do in GCP or is significantly easier? I have very little experience with GCP and a lot with AWS, so interested to learn more about how GCP compares.
I have a good enough experience with both platforms so this is something you have to try and you will see the difference.
GCP primitives are different and much better than AWS.
“Trust me bro” is a difficult sell; but I am here to back up the sentiment. There are simply too many little things that by themselves are not deal breakers at all - but make the experience much more frustrating.
GCP feels like if engineers made a cloud platform for themselves with nearly infinite time and budget (and some idiots tried to bolt crud on the side, like oracle, vmware and various AI stuff).
AWS feels like if engineers were forced to write adjacently workable services without talking to each other and on an obscenely tight deadline - by people who didnt really know what they wanted, and then accidentally made something everyone started to use.
I’m going yo write my own blog post about this, I used to have a list of all my minor frustrations so I could flesh it out.
Thanks, I would love to link it.
I’m still working on it, there’s some missing points and I want to bring up a major point about cloud being good for prototyping, but here: https://blog.dijit.sh/gcp-the-only-good-cloud/
Thanks for sharing.
If it helps I have the same professional experience with AWS and GCP. GCP is easier to work with but an unreliable foundation.
Correct me if I’m wrong but you can do all of the above with Elastic Beanstalk yes? Maybe ECS as well?
The trickiest part would be using AWS Secrets Manager to store/fetch the keys, which has a friendly enough UI through their web, or CLI.
You can definitely do all of this with EKS easily, but that requires k8s knowledge which is a whole other bag of worms.
The thing you are describing I should have only needed to learn once for each cloud vendor and put in a script: The developer-time should amortise to basically zero for either platform as long as the cloud vendor doesn’t change anything.
GCP however, likes to change things, so…
Yes and no. Some constructs you don’t even have to worry about on GCP, so, it is best for fast deployments. However, if you spend 500+ hours a year doing cloud work, then your point stands.
500 hours a year is only about five hours a week.
My application has run for over ten years. It paid for my house.
You’re damn right my point stands.
Indeed, if you are doing 5 hours a week of cloud infra work, your application is likely a full-time job or equivalent. I do believe you made the right choice with AWS.
Five hours a week is a full-time job?
You’ve got some strange measures friend…
read the post you replied to again, it does not say that.
No. If you are spending 5 hours a week on infrastructure, then your application would be worth spending 40-hours on. Or is infra the only component of your application?
Can you do an actual comparison of the work? I’d be curious. Setting up a web service from a Docker image is a few minutes. Env vars are no extra work. Secrets are trivial. A cert would take maybe 10 minutes?
Altogether, someone new should be able to do this in maybe 30 minutes to an hour. Someone who’s done it before could likely get it done in 10 minutes or less, some of that being downtime while you wait for provisioning.
I have done it for myself and I’m speaking from experience.
You’ve posted a lot in this thread to say, roughly ‘they’re different, AWS is better, but I can’t tell you why, trust me’. This doesn’t add much to the discussion. It would help readers (especially people like me, who haven’t done this on either platform) if yo u could slow down a bit and explain the steps in AWS and the steps in GCP and why there’s more friction with the latter.
No, please don’t trust me. Here’s an exercise for yourself.
Assuming you control a domain
domain1.com.Deploy a dockerized web service (pull one from the Internet if you don’t know how to write one). Deploy it on
gcp1.domain1.com, Deploy another onaws1.domain1.com. Compare how many steps it takes and how long it takes.Here’s how I deploy it on GCP. I never open-sourced my AWS setup but I am happy to see a faster one.
As I said, I have not used AWS or GCP so I have no idea how many steps it requires on either platform. I can’t judge whether your command is best practice for CGP and have no idea what the equivalent is on AWS (I did once do this on Azure and the final deploy step looked similar, from what I recall, but there was some setup to create the Azure Container thingy instance, but I can’t tell from your example if this is not needed on GCP of if you’ve simply done it already). If I tried to do it on AWS, I’d have no idea if I were doing the most efficient thing or some stupid thing that most novices would manage to avoid.
You have, apparently, done it on both. You are in a position to tell me what steps are needed on AWS but not on GCP. Or what abstractions are missing on AWS but are better on GCP. Someone from AWS might even read your message and improve AWS. But at the moment I just see that a thing on GCP is one visible step plus at least zero setup steps, whereas on AWS it is at least one step.
You’ve posted ten times so far in this story to say that GCP is better, but not articulated how or why it’s better. As someone with no experience with either, nothing you have said gives me any information to make that comparison. At least one of the following is true:
It sounds as if you believe the first is true and the second two are not but (again) as someone reading your posts who understands the problem but is not familiar with either of the alternatives in any useful level of detail, I cannot judge for myself from your posts.
If you wrote ‘GCP has this set of flows / abstractions that have no equivalent on AWS’ then someone familiar with AWS could say ‘actually, it has this, which is as good’ or ‘you are correct, this is missing and it’s annoying’. But when you write:
That doesn’t help anyone reading the thread understand what is better or why.
Given my experience with AWS, my vote is #2.
I find AWS to be better organized and documented but much larger that any Google product I’ve seen. There is more to learn because it’s a large, modular system. And it’s trivial to do an easy things the hard way.
I don’t have much direct experience with Google’s hosting but if any of their service products are similar, the only advantage is they do less, which means you can skimp on organization and documentation without too many problems.
IMO, it’s hard to recommend AWS as well.
So what do you recommend then? 🙂
In terms of big players I would recommend GCP still, but only because I mostly work with Kubernetes and it’s best there. From smaller players Fly.io is actually works well for me.
Why is it “best” there? I use EKS on aws and have had no issues with it…?
in Kubernetes territory, Google is the elder. Also cleaner and more intuitive UI.
not shocking. aren’t they the original contributors?
I don’t touch EKS’s ui much/ever so I honestly don’t really care about that. Usually use aws via terraform/pulumi.
I tried using checkpoints for a while but it didn’t do quite what I wanted and it was pretty tedious having to editor configs in a different directory and then running cz apply.
There was a blog post here recently that made the case for writing your own static site generator (that resonated with me). I am starting to think that maybe writing your own purpose-built dot file manager would similarly be a good idea.
Nothing wrong with that for those so inclined. But one only has so many free hours per week, and chezmoi is very convenient.
I did exactly that for a while, but it was awkward and unwieldy, even after years of adding features and cleaning it up. A more holistic solution (i.e. Nix) ended up being the answer for me.