I have used NixOS as my daily driver for a couple months now and I love it. However, I have a very superficial understanding of its architecture so I struggle to make meaning out of this. I’ve read https://r13y.com/ already, but it left me with more questions:
What is being compared to determine whether two builds are consistent with one another? (diffoscope?) Isn’t the output necessarily different due to hardware optimizations? Are they turned off for the purposes of these tests?
Does reaching the 100% threshold unlock new capabilities or use-cases?
Are there other 100% reproducible (non toy) operating systems? How non-reproducible are other OSes?
Were there any particularly challenging non-reproducible components?
What is being compared to determine whether two builds are consistent with one another? (diffoscope?) Isn’t the output necessarily different due to hardware optimizations? Are they turned off for the purposes of these tests?
In my understanding reproducible builds require that you target the same hardware, so e.g. arm64 without any extended instruction sets or so. Non-deterministic optimizations need to be turned of for that. https://reproducible-builds.org/docs/ is a nice resource, listing things which makes reproducible builds complicated in practice.
Does reaching the 100% threshold unlock new capabilities or use-cases?
Yes, one can assert whether a given ISO image matches upstream sources and hasn’t had any backdoors or so backed into the binary without disassembling it. This ability is lost if you are <100%.
Are there other 100% reproducible (non toy) operating systems? How non-reproducible are other OSes?
Yes, one can assert whether a given ISO image matches upstream sources
The act of verifying removes the need for verification. When you build it yourself to check, you no longer need to check. Just use your build artifacts.
Reproducible builds are nice for other reasons – eg, caching by hash in distributed builds, but they’re security snake oil.
Finally: if you’ve got a trusting trust attack, you can have a backdoor with no evidence in the code, which still builds reproducibly.
I think preventing it is hard because there are so many avenues to exploit, but reproducible builds can help you determine whether a build has been compromised. If you don’t know whether the attacker managed to alter your build artifacts, you can just rebuild them and do a byte-for-byte comparison. If your builds aren’t reproducible, you have to look at what the differences are: are they changed timestamps? optimization levels? reordered files? etc
You need to build it yourself to check, but could notify others if hashes mismatch as that would be much more suspect than it would be for non-reproducible software. Independent third parties could build other peoples ISOs on a regular basis to check.
Also, I forgot the obvious circumvention (beyond the trusting trust attack): being lazy and putting the exploit into the distributed code – since in practice, nobody actually audits the code they run, it seems like in practice this would effectively circumvent any benefits from reproducible builds. Signing the ISO gets you a hell of a lot more bang for the buck.
Reproducible Builds only concerns itself of the distribution network and build server. It can’t solve compromises on the input because that is not the goal. We need other initiatives to solve that part. Reproducible Builds is only part of the puzzle, and I people like you and Tavis really struggle to see that. I don’t know why.
This is very much like claiming memory safety issues are pointless to mitigate since logic bugs are still going to exist. But wouldn’t eliminating memory safety issues remove a good chunk of the attack surface though? Isn’t that a net gain?
I can get the claimed benefits of reproducible builds by taking the exact steps I’d need to verify them – running a compiler and deploying the output.
If you can tell me how to get memory safety by running a compiler once over existing code, with no changes and no runtime costs, I’d also call any existing memory safety efforts snake oil.
Again, if you’re concerned about the security problems that reproducible builds claim to solve, you can solve them today with no code changes. Just run the builds.
Again, if you’re concerned about the security problems that reproducible builds claim to solve, you can solve them today with no code changes. Just run the builds.
I have better things to do then swap out my distribution with Gentoo and pretend it solves the problem.
Yes of course, my claim was that one could check whether the binary matches the sources, not that it magically solves all security issues. You are right that people need to be able to trust their toolchains in the first place (trusting trust), but this is true for all software, reproducible or not.
Nobody! And even fewer people if the ISO build is not reproducible. Which is the point of reproducible builds.
Ensuring we can reproduce a bit-for-bit identical artifact ensures we can validate the work, even with a signing key compromise. Without reproducible builds we are only left to our own device and have no way to even start validating it.
I have used NixOS as my daily driver for a couple months now and I love it. However, I have a very superficial understanding of its architecture so I struggle to make meaning out of this. I’ve read https://r13y.com/ already, but it left me with more questions:
diffoscope?) Isn’t the output necessarily different due to hardware optimizations? Are they turned off for the purposes of these tests?Yocto Project (an embedded Linux system) also has a reproducibility status page:
https://www.yoctoproject.org/reproducible-build-results/
Here is their wiki page about the topic: https://wiki.yoctoproject.org/wiki/Reproducible_Builds
Thank you. The documentation is very precise, which I find reassuring.
In my understanding reproducible builds require that you target the same hardware, so e.g. arm64 without any extended instruction sets or so. Non-deterministic optimizations need to be turned of for that. https://reproducible-builds.org/docs/ is a nice resource, listing things which makes reproducible builds complicated in practice.
Yes, one can assert whether a given ISO image matches upstream sources and hasn’t had any backdoors or so backed into the binary without disassembling it. This ability is lost if you are <100%.
None that I know of, but many are working on it, see https://reproducible-builds.org/projects/
The act of verifying removes the need for verification. When you build it yourself to check, you no longer need to check. Just use your build artifacts.
Reproducible builds are nice for other reasons – eg, caching by hash in distributed builds, but they’re security snake oil.
Finally: if you’ve got a trusting trust attack, you can have a backdoor with no evidence in the code, which still builds reproducibly.
It’s not. It would explicitly have prevented the Linux Mint ISO replacement attack we saw 6 years ago.
https://blog.linuxmint.com/?p=2994
(It belongs to the story that the parent comment is just parroting talking points from Tavis)
Can you explain how anyone would have noticed without building the ISO from scratch?
I think preventing it is hard because there are so many avenues to exploit, but reproducible builds can help you determine whether a build has been compromised. If you don’t know whether the attacker managed to alter your build artifacts, you can just rebuild them and do a byte-for-byte comparison. If your builds aren’t reproducible, you have to look at what the differences are: are they changed timestamps? optimization levels? reordered files? etc
You need to build it yourself to check, but could notify others if hashes mismatch as that would be much more suspect than it would be for non-reproducible software. Independent third parties could build other peoples ISOs on a regular basis to check.
Also, I forgot the obvious circumvention (beyond the trusting trust attack): being lazy and putting the exploit into the distributed code – since in practice, nobody actually audits the code they run, it seems like in practice this would effectively circumvent any benefits from reproducible builds. Signing the ISO gets you a hell of a lot more bang for the buck.
Reproducible Builds only concerns itself of the distribution network and build server. It can’t solve compromises on the input because that is not the goal. We need other initiatives to solve that part. Reproducible Builds is only part of the puzzle, and I people like you and Tavis really struggle to see that. I don’t know why.
This is very much like claiming memory safety issues are pointless to mitigate since logic bugs are still going to exist. But wouldn’t eliminating memory safety issues remove a good chunk of the attack surface though? Isn’t that a net gain?
I can get the claimed benefits of reproducible builds by taking the exact steps I’d need to verify them – running a compiler and deploying the output.
If you can tell me how to get memory safety by running a compiler once over existing code, with no changes and no runtime costs, I’d also call any existing memory safety efforts snake oil.
Again, if you’re concerned about the security problems that reproducible builds claim to solve, you can solve them today with no code changes. Just run the builds.
I have better things to do then swap out my distribution with Gentoo and pretend it solves the problem.
Yes of course, my claim was that one could check whether the binary matches the sources, not that it magically solves all security issues. You are right that people need to be able to trust their toolchains in the first place (trusting trust), but this is true for all software, reproducible or not.
Another initiative in this direction are “bootstrappable builds”, https://www.bootstrappable.org/
Nobody! And even fewer people if the ISO build is not reproducible. Which is the point of reproducible builds.
Ensuring we can reproduce a bit-for-bit identical artifact ensures we can validate the work, even with a signing key compromise. Without reproducible builds we are only left to our own device and have no way to even start validating it.