My biggest fear is that, before Wayland is “ready” and X11 is “deprecated”, there will be the third thing (or there already is?), which is “gonna be better than Wayland”, and “finally fix everything”.
Because all major desktop environments and distributions target Wayland, and Arcan isn’t anywhere near that level of adoption.
As far as I see*, Arcan is available in AUR, Guix, Nixpkgs, FreeBSD Ports, Void Linux, and Slackware.
Considering it’s been around since ~2013-ish, so its only a few years younger than Wayland, it does not show any kind of adoption levels comparable to Wayland, and I don’t see why that would change. Thus, it’s not the future.
The future is not always the thing that’s technically the most sound, but the thing that people end up using, and that’s Wayland.
Not sure about fundamentally, but Wayland can run GNOME and KDE.
Like it or not, that’s the bar for anything that’s going to replace X or Wayland. If regular users have to learn an entirely new DE, it’s not happening.
I appreciate your perspective, but I disagree with your opinions. Specifically:
We don’t need a version resolver. You may want one, but it is clearly not strictly necessary.
flake.nix‘s grammar and semantics are a strict subset of Nix; it’s not inconsistent at all. It is a downgrade in expressive power, and that’s a good thing when we consider POLA and packages-as-capabilities.
system must be explicit for good cross-compilation. Yeah, it’s a little irritating, but the alternative is vaxocentrism, and anybody who survived the 32-to-64-bit transition will agree on the value of “i686” and “amd64” being different strings.
We don’t need a version resolver. You may want one, but it is clearly not strictly necessary.
If FlakeHub is going to bolt on a centralized version resolver, I really want flakes to properly tackle it in a decentralized manner instead. FlakeHub version resolution meshes poorly with input following and machine-scope flake registries. I am generally worried about FlakeHub becoming how the Nix ecosystem deals with flakes; it’s a glaring point of centralization in a feature set designed for decentralization. Flakes currently only have one enforced point of centralization: the global flake registry, managed by the NixOS foundation.
system must be explicit for good cross-compilation. Yeah, it’s a little irritating, but the alternative is vaxocentrism, and anybody who survived the 32-to-64-bit transition will agree on the value of “i686” and “amd64” being different strings.
I agree! System sets also need to be extensible. systems in the centralized flake registry is the only working solution I’ve seen so far, and it’s woefully underused. We need to heavily enforce one convention across the whole flake ecosystem for extensible systems to be usable, or we need to extend Nix with an official flake system registry that’s also extensible.
I’m curious: Don’t you dislike the fact that basically the entire point of flakes is to avoid the Nixpkgs monorepo? The ability to have a monorepo is a key advantage of free software, and the main reason to avoid it is to support proprietary software. I’d rather we keep the monorepo and just educate people better on how to use it, rather than dismantle it with flakes.
I’m not entirely sure that is the point; I have a bunch of github projects that are deployed out of flakes. There is little to no reason for putting them into the nixpkgs monorepo, as they’re basically used by me only. There is no wider interest that would justify putting them into nixpkgs and nixpkgs would only add massive delays to trying to push new updates to my servers.
edit: I would also like to add in a quick edit that nixpkgs contains plenty of non-free software, such as VSCode. Free software does not enable monorepos any more than non-free software does and monorepos can contain both free and non-free software.
Aren’t you just agreeing with me that the point of flakes is to avoid the Nixpkgs monorepo?
Anyway, if your projects don’t have dependencies on each other, then you don’t need flakes. If they do, then maybe the common component should be published as something shareable, which could go into Nixpkgs. Or just put the projects which depend on each other into a single repo.
The project in question is a collection of various web services, they do not make sense in isolation but don’t share much common code (but some do and the flakes in question are pointing the right direction), if they’re even in the same language.
The issue is that again; this is code solely deployed by me that in parts even only makes sense being deployed by me, in other parts simply stuff I deploy to servers for others. There is no reason to pull them into nixpkgs and increase the maintenance burden of everyone there because they have to maintain some random code I’ve barely touched for a few years.
In this matter, I see nix flakes no different to a Dockerfile; it’s a way for a maintainer to bring availability of their code to a wider audience without having to play the maintainer game of thrones. The upstream is in full control of the deployment and determines the defaults and configurations, as it should be (before going for NixOS I used ArchLinux in part for the reason that they have very minimal patches on top of every package).
There is also no reason to put all my projects into a single repo, I do not like the monorepo approach. It’s a repository approach propagated by the likes of Google and Microsoft, it’s hostile to small open source projects due to the increased maintenance burden. Nixpkgs gets away because it’s got plenty of subtree maintainers, I don’t wanna put that on myself just for my own silly bits of code.
And no, flakes are not there to avoid the nixpkgs monorepo, evidenced by the fact that all of them pull it in for common components. That argument would hold up if the majority or a good slice of flakes, didn’t pull in nixpkgs at all purposefully to avoid it. But almost all flakes hit nixpkgs in various ways.
So nixpkgs should be filled with a bunch of “caterns_repos” folder for each developer having a public hobby project? You can surely agree that it doesn’t scale there.
the entire point of flakes is to avoid the Nixpkgs monorepo
It is a point, but I don’t believe “entire point” does the multitude of motivations for flakes justice. I’ll defer to today’s other frontpage Nix post [1.] and provide this list of Flakes features:
- Dependency locking/tracking between Nix expressions (flake.lock)
- Fetching dependencies with a common interface (builtins.fetchTree)
- Harmonized declarative interface
- New CLI interface semantics
- Pure evaluation semantics (not exactly, but currently linked to the Flakes mental model)
I’m not expert enough in Nix/Flakes or the history of Nix to say whether these other are the most salient motivations for flakes, but collectively they constitute a body of work that goes significantly beyond what you describe as avoiding the nixpkgs monorepo.
The ability to have a monorepo is a key advantage of free software, and the main reason to avoid it is to support proprietary software.
Similarly, I believe this statement is too absolute. The main reason to avoid a monorepo depends on a user/project/organization’s motivations. E.g. my main reason to avoid nixpkgs for a silly little di.fm player [2.] that I develop is that I don’t think it belongs in the nixpkgs monorepo. Few Nix users would benefit from it, as it has few users, and never will have a significant number of users. Going through an additional step of review every time I make a release would take away some of the joy I get out of developing the application. This doesn’t mean di-tui has proprietary aspirations. It’s open source and will remain that way.
nixpkgs is massive, and it is absolutely inconceivable to be able to store, say, the whole python library ecosystem, plus the whole of nodejs and whatnot. There is simply a limit on what can sanely fit inside it and it is already bordering on that limit.
I believe the duality of Flakes and a good base monorepo is actually a quite great approach as is: have a rolling release “blob” of mostly working together, common software I can refer to always and let more exotic software be defined and linked from anywhere – you end up with a Flakes file that pins a working nixpkgs.
Disclaimer: I only use nixpkgs on other Distros, I have never installed NixOS on its own, but I did read up a lot over the years and tinkered, and before flakes I could sum it up as “Nice in theory, but completely unusable for the amount of work I am willing to invest.” - with flakes everything seemed easier and made more sense.
No, I don’t care whether a ports tree is confined to a single git repository, especially not at the size of something like nixpkgs. Flakes only act as a federation mechanism; the integration of code still happens in the same fashion either way.
the integration of code still happens in the same fashion either way.
But that’s not true! Federation in this form requires “stable APIs”, a deformity usually induced by the presence of proprietary software which can’t simply be patched and recompiled for new APIs. “Stable APIs” are only needed when implementers can’t just update their users for the new version. See http://catern.com/freetech.html
We might be thinking of different actions. To me, the integration of code is what the Nix daemon is doing when I hand it a build request. That request will involve two git repositories already (nixpkgs and the upstream repo for the target package), so I’m neither bothered by the idea that the second repository will have Nix expressions, nor by the idea that the second repository is fetched prior to the first repository.
At no point here am I advocating for non-free software. I will point out that, by design, a ports tree usually has no problem expressing the integration of free and non-free software; nixpkgs already carries many such expressions, regardless of whether I use them.
Stability of a Flake API is almost entirely irrelevant from this perspective, because the versions of expressions are locked in flake.lock. Lockfiles provide bitemporality, disconnecting the time of integration from the time of locking.
That is a great advantage of flakes though. Because you can make much more than just packages. See home-manager, for example. Installing it as a flake is considerably easier and more reliable than as a channel. It makes developing Nix module sets that wouldn’t really fit in NixOS considerably more practical.
I don’t get the version solver part. Those expressions don’t have versions or version constraints. You refer to some branch or tag and that’s it. An alternative version solver could be built on top, but that would also require flakes themselves to be versioned differently than people do at the moment. (One foo-2.0 doesn’t equal another foo-2.0 when they pull in different refs of the whole system they depend on)
And that’s more of a nixpkgs thing than a flake thing.
Version solvers make a lot of sense in systems like apt or pip, in which you can only have at most one version of a dependency so everyone needs to agree on what that version is.
However, this can have some weird non-local effects. For example, you can have libA depending on libB ^1.2, and in libA’s CI they use libB 1.2.0 to run their tests. Now a downstream project that depends on libA but that also (either directly or via one of its other deps) requires libB ~1.3 would silently use libA with libB 1.3.0, which it was never tested against.
In Nix, everyone can have their own versions of all their deps, and everything is used with the same versions of dependencies when integrated into a larger project as it does as an individual software project.
This makes reasoning nice and local, but it also wastes a lot of disk space and network bandwidth.
I think that’s a decent tradeoff a lot of the time.
Yeah, it also seems to me like a usual case of people recognizing a pattern they’re familiar with and assuming that they can carry over all of their experience and mindset because of the similarity. There are specific reasons why version resolving works well for programming language package managers, but they don’t necessarily apply to nix flakes. In addition to what you said, here are some differences that come to mind:
The public facing interface of a flake is very hard to pinpoint compared to a library in a programming language, especially one with static types. Unless the flake is a Nix-only utility library, pretty much any change would require a major version number bump because of how deeply the consumer might be interacting with your flake and its outputs.
Nix doesn’t have a runtime! It’s very important to think carefully about the backwards compatibility of packages in programming languages, because you’re mostly concerned about what happens at runtime, when you, the programmer don’t have the means to swoop in and fix the issue. Of course it’s important to care about your consumers’ builds not failing unnecessarily, but the real nightmare fuel is what can happen at runtime. In that respect, since Nix only exists at build-time, which is when a developer is at hand to fix the issue, version issues are less critical.
You are much less likely to be forced to upgrade your flake dependencies. With programming language packages, you often need to upgrade a bunch of mutually interacting packages all at once because everything depends intricately on each other. But with Nix flakes, it’s so easy to add a inputs.my-nixpkgs-for-a-very-specific-purpose.url = github:myorg/myrepo/my-ad-hoc-branch and use my-nixpkgs-for-a-very-specific-purpose for that specific purpose without disturbing anything else.
I often see a similar thing happening when people criticize Nix for being Turing-complete or untyped etc. They just carry over reasoning from related but different domains without reevaluating them for the unusual case that Nix is.
Yes and right now it is what channels handle. But with flakes, the whole point is that you only change things when you decide; that is what the lock is about. So now you have to take care of every single version change of your dependencies and how they may not interact well with each other yourself. It kindly becomes unwieldly
I don’t believe that’s a technical problem which can be solved. You always had to do that yourself. The versions constraints in libraries are mostly just indicative anyways. (we’re fixing some random issue from package updates every other week) If you’re depending on two unrelated projects and reference them both in your flake, that means they don’t depend on each other anyway and there would be no constraint anyway, because you can’t realistically do a full compatibility matrix of everything installable.
For example someone writes a Frobnicator which uses /tmp/foo as a lock file and someone else writes a Bloberator which does the same. When they get independent flakes, who would you expect to test all the combinations and where would the constraint of “those two can’t work together” even live? (And how do you express that they can work, but not at the same time)
Not to even mention that no person or script actually would use such numbers on the command line that trigger the bug. This is the bug that never triggered for anyone who did not deliberately try to make it happen.
I tend to disagree with this thinking. Daniel dismisses the issue is a security hole and then directly asserts no one would even use this feature anyways as if to bolster the argument around it not being a problem or security issue.
To say it’s a buggy feature that no one hits in practice anyways is fine, but it is entirely orthogonal to whether or not this is actually a security issue. If this bug is used as part of a security vulnerability (or within an exploit chain) “they” would deliberately use whatever means they can to trigger UB or buggy behavior to further the exploit.
What you’re saying is exactly the philosophy that leads to CVSS scoring everything “critical”. You can always imagine a scenario where someone would set up themselves for a colossal failure when hitting even a smallest bug.
If you don’t include probability of this happening in your evaluation, you get absurdly inflated danger. You have to consider how unlikely it is that someone would allow attackers to specify the timeout with arbitrarily large numbers, while also critically rely on this timeout to be in a specific range.
Yes, CVSS is entirely ignorant of this dimension. If there is any imaginable deployment scenario in which the vulnerability is triggered by input from the network, you get AV:N (“Attack Vector: Network”, the worst one).
If there is any imaginable scenario where the vulnerability causes severe harm to confidentiality or integrity, you get C:H and I:H - as curl did in this case.
Because you can nearly always come up with very bad scenarios, the most common CVSS by far is exactly 9.8 - everything maxed out except “Attack Complexity: Medium”.
Thus, use-after-free in nginx’ HTTP request parser is a 9.8/10, but so is this total nothingburger in curl. Because in theory you might deploy it in a way where attackers get to control specifically --retry-delay over the network and you might compile it using a hypothetical C compiler that turns integer overflow UB into arbitrary code exec.
For us practitioners, this quickly turns it into a “boy cries wolf” situation. If every CVE is severe, none of them are.
This is why, when I’m writing security announcements, I try to be very descriptive about how the vulnerability might be exploited. That way folks can actually understand how much of a nothingburger the vulnerability actually is. Unfortunately that requires some domain knowledge on the part of the reader, so folks that don’t have that kind of domain knowledge are stuck trying to interpret a number. The whole numbering scheme is very problematic IMO and a source of pain for me personally.
I definitely agree with what you’re saying, so it’s possible I didn’t articulate my thoughts correctly. I 100% agree CVSS scores are often inflated and you have to take into account the likelihood of someone hitting said code path to be a vulnerability.
My point was also not talking about this specific case with curl which I agree is probably a non-issue. I was speaking about a hypothetical scenario in which lets say some program had an argument --foo N where some values of N lead to a real vulnerability for example --foo=999, but in normal use of the program no real users ever set N to that vulnerable value, they all use --foo=3 or --foo=1. I was saying it doesn’t matter that normal users don’t use the vulnerable value, an attacker easily could. So I think in my made up scenario I’m weighing the “ease” at which an attacker could use the vulnerability, and entirely disregarding whether or not a normal user uses the vulnerable values.
But I’ll concede my thinking isn’t really applicable to this case, and thus confused the conversation.
Daniel dismisses the issue is a security hole and then directly asserts no one would even use this feature anyways as if to bolster the argument around it not being a problem or security issue.
I agree that he asserts it’s not a security problem. I would not agree that he asserts it’s not an issue, since it was fixed.
It also is apparent that Daniel spent the time to understand how this would behave in practice, and supplied evidence that it was not a risk in practice.
(down-thread)
you might compile it using a hypothetical C compiler that turns integer overflow UB into arbitrary code exec.
Daniel is direct about the idea of hypothetical/theoretical problems not being sufficient. I do think actual an demonstration is a better place to come from than a hypothetical that then needs to be exhaustively disproven - and it can’t be, the hypothetical space is infinite.
I agree with Daniel here. Unless there exists a C compiler that does turn the UB into arbitrary code exec, there seems no reason to even consider that a real possibility.
It’s just a matter of time before someone creates the world’s worst C compiler which is technically compliant with the standard specifically to win an argument about CVE severity ranking.
People have been joking about the Deathstation 9000 C compiler for like 30 years, but no one has (as far as I know) implemented a compiler that actually uses the most perverse possible interpretation of the standard. Perhaps the closest is the Frama C static anslyser?
And, even if it was arbitrary code exec, I’ve still seen zero examples of a realistic scenario where an attacker would be able to control this parameter without having arbitrary code exec anyway.
The problem with relying on UB to escalate severity is that it’s not possible to write a non-trivial C program that’s completely free of UB, at which point everything just escalates automatically to max severity because there’s probably a code path that hits UB.
I’ve personally switched over to Bitwarden, and am in the process of getting my family out. I migrated (easily!) last year and have not regretted it once.
Bonus: you can (should you so desire) self-host the Bitwarden sync server, which was a draw for me, but so far I’m happy to let them handle server maintenance, backups, etc.
I’m self-hosting Vaultwarden and it’s been working flawlessly. NixOS makes it quite easy via services.vaultwarden. And on the clients the only thing you need to do is put in your custom URL at signup.
I’m possibly not the best at server maintenance, but all the apps work even when the server is down or the client can’t connect to the internet at all. They have a local copy of the encrypted database.
I also switched from LastPass to Bitwarden. I’ve wanted to do that since LogMeIn took over, but it was low prio until the breach last year (mentioned in the article).
I picked Bitwarden between all the alternatives for a few reasons:
managed solution. As LuminantJess says, I don’t want to deal with backup and security for such things as my passwords.
self-hosting is possible (should I ever choose to deal with backups and security afterall).
open source (unlike a lot of other options, starting with Google and Apple, via LastPass and the like) means I can set up some things like @muvlon mentions in his comment before relatiely easily.
The switch was relatively simple - export from lastpass, import in Bitwarden, and I kept both for maybe a month before a full switch.
Also running vaultwarden here, installation was easy, and having the official android and firefox/chrome addons from bitwarden that can talk with my own vaultwarden instance is very nice.
I would only ever use something that can be self-hosted and is open-source, so things like lastpass and 1pass wouldn’t be acceptable.
I still have / keep all passwords also in ‘pass’ (passwordstore.org) … as having a git log of everything, and it’s massive flexibility and programmability is a thing I wouldn’t want to lose.
Am I misunderstanding something or does bitwarden gate the generation of TOTP codes behind a paid plan for their hosted service, i.e., preventing a self hosted instance from being useful for TOTP?
The official Bitwarden server does, but the more efficient Vaultwarden server has no such limitations.
Although, it’s a personal policy decision of whether you want the same store to hold both authentication factors.
Nix (the language) comes to mind, especially viewed against NixOS competitor Guix’s use of Scheme. But I don’t know if I’ve ever encountered a tool written in Scheme, so it’d probably feel as much like a DSL to me as Nix does.
There’s some proposals to refactor Nix to make the layers more separable, which would eventually allow languages other than Nix to be compiled down into derivations and realized in the Nix store. I wonder if there’s a general-purpose language that would be good for that, other than Scheme.
I’m also following that Nix proposal. I think it’s a great idea and am glad the RFC was accepted.
Here’s one reason this is no panacea though: Anything that people will actually want to use will have to be able to call into Nix. Because otherwise, you get no access to nixpkgs, losing you one of the biggest benefits of adopting the Nix ecosystem.
That still leaves you with a few options: You can shell out to nix-instantiate, you can compile to nix code or you can use the little-knowm plugin system of nix that allows you to extend the interpreter. However, tooling for existing general-purpose languages is not really there yet.The only thing in this area I’ve seen is https://github.com/purenix-org/purenix
Endianness is critical, yet C has no standard facility to find out what endianness the host uses, nor to convert values to and from a specific endianness. Due to other problems, these are not trivial to handle yourself either!
The whole “binary input and output” section seems like nonsense to me. There are also a lot of complaints about C not guaranteeing specific kinds of integer, but C only doesn’t guarantee them so that it can be ported to platforms which don’t have them. The fixed-width types do always exist if you restrict yourself to targets that can run Javascript.
The stuff about overflow and out-of-bounds array access is fair enough, although I personally think Javascript does about the worst thing a memory-safe language could possibly do in both cases (yield a value instead of indicating an error).
There are also a lot of complaints about C not guaranteeing specific kinds of integer, but C only doesn’t guarantee them so that it can be ported to platforms which don’t have them.
Which is ridiculous! How many users want to be able to rely on fixed-size integers vs how many users want to target a platform that doesn’t hav 8/16/32 bit integers? (How many have even seen one?)
I obviously hid this fact better than I meant to in my last comment: The people who don’t want to target those weird old platforms can rely on fixed-width integers, and have been able to for decades. The weird concessions C made to support weird old architectures sometimes affect programmers who don’t care about those architectures, but this is not one of those times. Nothing in the C standard requires you to write code that will work with 9-bit bytes.
Also, it’s really easy to setup on NixOS: services.smokeping. It’s not too heavy and it can be a lifesaver when things break, so there’s little reason not to run it on servers.
Quick question, what’s the diff between smokeping and uptime kuma? What do you use them for? I have uptime kuma just for monitoring a few sites, I don’t need anything fancy, but I also brought it to work so we can also use the dashboard export thingy. What extras does smokeping have in comparison?
Smokeping is for Network latency. So I think of uptime kuma is more.of a heartbeat to check if the site is either up or down and notifies you if they are down. And smokeping does analysis of packets sent. But very much could be wrong. I stood up both and they are very similar except for UI
The main downside(s) I’ve found with Funnels is a lack of custom domain names (you’re stuck with your Tailscale Net name) and generally slower speeds when running through a relay. I ended up spinning my own using DNS splitting and a free tier VPS.
I misspoke, apologies. This list is more of a “homelab” list, especially with cloud flare tunnels and tail scale. I’m not using headscale. I’m using the non self hosted tailscale
You know, I was going to say that its OK, as it’s a handful of unsafe blocks around FFI, its comparatively easy to just eyeball them, and, unless you go full CHERI, unsafety has to live somewhere.
And I think that’s a bug and technically UB! CString::new allocates, and you can’t allocate in pre-exec.
So kinda no maybe?
But also maybe yes, or at least much better than alternatives? This bug really jumps out at me, it’s trivial to notice for someone unfamiliar with the code. pre_exec is tricky, and unsafe makes it really stick out, much better then if it were hidden in the guts of the standard library or some native extension.
(also, obligatory PSA: the biggest problem with sudo is not memory unsafety per se, but just the vast scope for this security sensitive utility. For personal use, you can replace sudo with doas, and it’ll probably make a bigger dent in insecurity. If you have to be API and feature-compatible with sudo though, then, yes, what Ferrous folks are doing makes most sense to me)
I often see doas recommended as simpler than sudu. When I compare the documentation I see an almost identical feature set. The only difference in security surface are seems to be the configuration. Is there more that sudo does that doas does not do?
That’s also the funny thing — I don’t actually know what sudo does. I know that OpenDoas clocs at under 5k lines of code, while sudo is a lot more (see the nearby comment by andyc). So, that’s a non-constructive proof that it does something extra!
You’re allowed to allocate in pre-exec. The restrictions the documentation mentions mostly stem from other threads potentially holding locks or having been mid-modification of the env variables.
If you guarantee that there is no threads (and other pre-conditions like reentrancy) running at the time of fork(), you can in fact malloc without any big issues.
I think in case of Rust it’s actually quite murky technically:
First, Rust doesn’t have assert_no_thread!() functionality, so, if you assume at pre_exec time that the program is single-threaded, the calling function should be marked as unsafe with the safety precondition of “must be single threaded”. Which mostly boils down to just “don’t allocate” in practice.
Second, allocation in Rust calls #[global_allocator], which is arbitrary user code. As a general pattern, when you call arbitrary code from within unsafe blocks, there usually is some Rube Goldberg contraption which ends with a shotgun aimed at your feet. That is, if the user tags their own static as a global allocator, they can call various API on that static directly, and that should be enough to maneuver the thing into “technically UB” even without threads. In particular, you could smuggle something like https://github.com/rust-lang/rust/issues/39575#issuecomment-437658766 that way.
But yeah, this is quite subtle, there was some debate whether before_exec needs to be unsafe at all, and I personally am not clear as to what’s the safety contract of before_exec actually is, in terms of Rust APIs.
To be fair, pre_exec is an unsafe function for precisely this reason and it should stay that way. The API contract is largely unspecified because fork() does some amazing things to program state. I think the solution here would be to swap in a CStr over a CString, to avoid the allocation.
edit: One way we could avoid it is by introducing a new unsafe trait; Interruptable. It must be explicitly declared on structs. The trait would simply declare the struct and it’s functions reentrancy safe. A function is automatically interruptible if all non-local data it uses is also interruptible. Then PreExec could simply require the function to also be Interruptable.
To be fair, pre_exec is an unsafe function for precisely this reason and it should stay that way. The API contract is largely unspecified because fork() does some amazing things to program state.
It looks as if it’s intended as an abstraction over different process-creation things and fork and vfork do different amazing things to process state.
Fork creates a copy of the address space and file descriptor table, but that copy has only the current thread in it. If other threads are holding locks, can cannot acquire those locks without deadlocking. You need to register pre-fork hooks to drop them (typically, the pre-fork hook should acquire the locks in the prepare stage and then drop them in the child, it should also try to guarantee consistent state in both). It is unsafe to call malloc from a multithreaded program between fork and execve because it may deadlock with a not-copied thread.
Vfork creates a copy of the file descriptor table but does not create a copy of the address space. You can modify the file-descriptor table until you execve and then you effectively longjmp back to the vfork call (I believe this is actually how Cygwin implements vfork). Because the code in the vfork context is just normal code, it is safe to call malloc there, but anything you don’t free will leak.
This is the main reason that I prefer using vfork: I can use RAII in my setup code and, as long as I reach the end of the scope before calling execve, everything is fine.
This is often a very constrained environment where normal operations like malloc, accessing environment variables through std::env or acquiring a mutex are not guaranteed to work
Which doesn’t read like memory allocations are forbidden in that closure to me.
Ninja edit: To me, it reads like if e.g: your program is single-threaded then you’re fine.
Yes, that rule is mostly about multithreaded environments. And violations usually don’t result in memory corruption but in deadlocks.
The reason is that after forking, you only have the current thread, all the other ones are “frozen” in time. So if you forked while some other thread held a lock inside malloc and then call malloc yourself, you can deadlock.
And also, glibc in particular has special code to make this work, so you can indeed malloc after fork there safely.
So IMO this is POSIX UB and therefore Rust UB by fiat, but very likely not a security issue in practice. It should be fixed, but it’s not super alarming to me.
You definitely can replace sudo with doas… unless you run a RHEL or a clone. I have two Rocky machines and my Ansible does not coexist with the RHEL clones as compared to FreeBSD and my myriad of other operating systems.
I have effectively replaced sudo with doas in all situations except for those platforms.
unsafe does not necessarily mean it does not have memory safety, but that that code needs more scrutiny. I am curious why there aren’t SAFETY comment blocks on each instance of unsafe to explain the invariants that makes it safe or necessary.
Most of them look pretty clearly justified, although it’d still be helpful to have an argument why they’re correct. The unsafe blocks I looked at are all used to interact with unsafe APIs, which is sort of a given for this sort of program, but they’re short and at first glance don’t seem to do anything that would be hard to reason about.
This would mean that the whole titular claim, that this is the “first stable release of a memory safe sudo implementation”, is probably wrong. C code can also “have memory safety”, so the first stable release of a memory safe sudo implementation is probably (at least some version of) sudo itself.
It might be my own failing, but I’m having trouble understanding your comment. That the “memory safe version of sudo itself has no bugs leading to memory errors” seems like a tautology.
If you mean we don’t know if there’s any such version, that’s technically true, and why I said “probably”. It seems pretty likely, though, and it’s certainly at least possible, bar any proof otherwise.
The point is that it doesn’t seem likely to me that sudo, as a reasonably large program in a language with few safety features, has no more such bugs. Sudo has had memory safety bugs in the past, and if I had to guess, I’d expect more to emerge at some point. You may be able to prove that there’s no memory-safe version simply by waiting for that to happen.
Of course, if there is such a version, finding it and proving it to be safe may be considerably more challenging. Which is important in itself: its safety wouldn’t have much value to us unless we knew about it.
You may be able to prove that there’s no memory-safe version simply by waiting for that to happen.
That would show there is (was) a memory-error bug in some version(s), not all prior versions. And it’s all a bit hand-wavy; that we know that there were memory safety bugs in the past only shows that such bugs were found and fixed. From reading elsewhere sudo sounds like it’s more complex than I would’ve thought it should be, but it’s still not so big that it’s impossible that it is (or has been at some point) free of such bugs.
If you prefer, I can rephrase my point (making it slightly weaker): the linked article is claiming the “first stable release of a memory safe sudo implementation” has been made, referring to this Rust implementation, but there is no certainty that all prior implementations had memory-error bugs, so if we take “memory safe” to mean “has no memory-error bugs” then the claim is unproven and could be wrong. (It seems we can’t take “memory safe” to mean “written completely in a memory safe language”, since it apparently uses “unsafe”).
(As to how likely the claim is to be wrong, we may disagree, but there’s probably not much in the way of objective argument that we can make about it).
Even if you implement a program in a memory safe language you do not know for sure that it is memory safe.
You have a very strong reason to believe so, but there is always the possibility of a mistake in the type system design or the language implementation. After a few years the rust language has been addressed with formal methods that mathematically prove correctness for rust programs as well as specifically proving the correctness of various parts of the rust standard library that make use of unsafe blocks. This helps give very strong confidence about memory safety, but again there is the possibility of gaps.
Ugh. Firstly, no, C and C++ don’t prioritize performance over correctness, since turning undefined behaviour into a defined trap (for example) doesn’t make a program correct, it just makes the observable behaviour of the (incorrect) program different.
Secondly:
If you compile this with clang++ -O1, it deletes the loop entirely: main contains only the return 0. In effect, Clang has noticed the uninitialized variable and chosen not to report the error to the user but instead to pretend i is always initialized above 10, making the loop disappear.
This is such an awful mischaracterisation of the type that often comes from the “C(++) compilers are evil, why can’t they just do what I want” camp. Clang “notices the uninitialized variable” only in the sense that it notices that the usage in the loop must be invalid (the variable hasn’t been defined at that point) and so, yes, it erases the loop; no, it doesn’t “pretend i is always initialized above 10”, in fact it doesn’t even consider the entry condition to the loop, since the optimisation pass is looking for code paths it can eliminate and the presence of definite UB on a particular path is good enough for that. Yes, it assumes that the loop body will never execute and logically this would mean, in the absence of UB, that i > 10 at entry to the loop but of course UB isn’t absent and so that logic isn’t right (and the compiler isn’t using it, despite the claim that it is). Clang doesn’t report the error because it doesn’t know there is an error; even though it’s apparent to us, it would require additional analysis steps for the compiler. It’s not as smart as the author pretends (or believes) it is.
I’ve said it before, but here we go again: compilers make use of the presence of UB to remove code because compilers are expected to optimise, optimisation is hard, and removing code is an easy win that’s sound in the absence of UB. It’s not about breaking the code; the code was broken to begin with. It might be reasonable to debate the precise allowances that the language does or should give to compilers in these situations, but there’s no excuse for attributing to compilers the kind of malice that this writing effectively does.
Clang doesn’t report the error because it doesn’t know there is an error; even though it’s apparent to us, it would require additional analysis steps for the compiler.
Yes, because it does precisely only the amount of analysis required for translation itself and performance optimization. It could do more analysis in order to warn the user of a correctness problem in their code, but this is not prioritized.
That’s a choice in the design/behaviour of the compiler, though, not the language; anyway the diagnosis is always going to require further analysis than the optimisation, so it’s not a question of prioritisation even on the part of the compiler.
Right, the language just chose to allow a lot of freedom in this on part of the compiler, which the compilers all chose to spend almost exclusively on performance.
But the design of the language is driven for the most part by compiler vendors, who know exactly which carveouts they require for their performance goals. So the distinction between language design and compiler design is quite blurry now.
Another recent example is Nvidia deciding that no, there is no guarantee that all threads eventually get scheduled, changing the entire deal around forward progress guarantees and throwing around their weight in the committee to make them declare in C++17 that your spinlock-based mutex implementation was actually Wrong All Along. Because this was the only way to keep their performance while being able to declare CUDA C++ “fully C++17 compliant”.
Or even more recently, when the major vendors discovered that their compilation scheme of sequentially consistent atomics for POWER CPUs was not compliant with the rules from the standard. They did not fix their compilers to be compliant, because that would’ve cost them performance. Instead they worked in the committee to change the memory model, weaking its guarantees and breaking a bunch of previously correct code.
There are no separate language design and compiler implementation processes. The vendors do what they must to achieve their performance goals, then the standard finds a way to make that legal. The overall system, whether by design or by happenstance, achieves performance at the cost of most other aspects.
Right, the language just chose to allow a lot of freedom in this on part of the compiler, which the compilers all chose to spend almost exclusively on performance
That’s not really true; witness the extensive sanitizers available with current GCC/LLVM, the options such as “-ftrapv”, and a plethora of other options specifically chosen to inhibit optimisation and/or improve diagnostics in the interest of program correctness or at least in restraining the effects of UB. If anything I’m seeing more and more effort being put into the latter and less and less on attempts to improve performance, in compilers, but even in the language spec: in C++ integers are now guaranteed to be 2’s complement, for example (even if overflow is still UB, this is a limitation on the leeway that compilers have, not an increase in it); the “unreachable” annotation introduced in C23 is specifically designed to allow compilers to be able to better distinguish between unintended and intended UB.
As to the other claims you’ve made, I can’t say I know anything about them (this is the first I’ve heard of either case), but they sound more like “prioritising ability of vendors to claim standards compliance” than they do specifically about prioritising optimisation per se. Granted, this is arguably a finer line. In any case, going back to my original point, compilers are_n’t_ (edit) being malevolent when they employ these optimisations; they’re not knowingly making programs misbehave, as the article suggested.
It’s always slightly surreal to me when people complain that their C compiler is generating buggy programs when they are compiling buggy C and passing in -O7 -fno-crapv --dont-help-me-i-am-an-expert.
Your compiler will prioritize performance over helping you if that is what you tell it to do.
I hear these arguments ad nauseam and they invariably arise from user error. That plus the multiple technical errors in the submission make me reluctant to even engage with the point it wants to make.
The C++ committee could take inspiration from Rust: An alternative to defining signed overflow is to define functions for doing arithmetic with wraparound.
Ditto for unsigned, as it expresses intent. Situation: Clang’s UB sanitizer actually complains about unsigned overflow. Which I’m sure catches bugs, but is really an incorrect thing to do, and worse, removes the only way to do intentional wraparound portably – you have to use compiler specific builtins now.
In practice in all the c++ code I’ve worked on, unsigned over / underflow has always, always been a bug outside of hash functions & similar, and this specific sanitization (-fsanitize=integer) has saved my ass a lot of times. It’s very uncommon to want N-1 to be a few billions when N=0 and causes a lot of issues down the line.
This is, unfortunately, largely unavoidable, because C++ doesn’t have anything like flow-sensitive typing. Consider the following:
if (someOptional)
doSomething(*someOptional);
else
doSomethingElse(*someOptional)
If you had flow-sensitive typing, then you could make this a compile-time error: in the first branch, the type of someOptional is a valid optional, in the second branch it is an optional in an known state, and the dereference operator is defined only on the former. Within the C++ type system, you can’t express that.
The other choice is to have the dereference operator (and value() method) throw an exception or abort if called with an invalid object. This can hurt performance on correct code, because you are relying on the compiler being able to inline the dereference method and CSE the check (which may or may not actually be possible depending on other aliasing properties of the code). Note that UB doesn’t mean ‘must explode’, a standard library implementation is free to insert a check and a branch to abort() here.
In Rust, the type system gives enough flow sensitivity (modulo some soundness bugs that were still open a couple of years ago, not sure if they’re fixed now) that you can make this form a compile failure and it provides enough aliasing guarantees that, even if you made it a dynamic check, the optimiser has enough information to know that it’s safe to elide the second check (this is one of the reasons why you can’t circumvent the borrow checker in unsafe code: it will make these optimisations unsound and introduce incorrect behaviour in distant bits of code).
In C++, the work-around is to use value_or or the monadic operations introduced in C++23. The latter tend to perform well because the compiler’s inlining heuristic will always inline a lambda if it is called in a single place (it is never better to outline code that is used only once, at least until code generation where you may want to move cold paths to a different page). I generally consider using the dereference operator or value() to be a sign that the surrounding code needs careful auditing in code review.
Well yeah, the exception way would have had a performance impact. So they decided not to pursue it, because performance is the top priority in their language design process. Then comes palatability for compiler implementers and then maybe it’s safety third.
And they make even more bewildering decisions:
When they added std::span, they not only made the regular index operator unchecked on pain of UB, they discussed and then dismissed the idea of also offering a checked .at() method, even though it would have been easy to add and consistent with other types like std::vector or std::map.
They’re not even giving users the choice between performance or safety. It’s full YOLO or nothing.
The problem with that approach is that it relies on implicit conversion to Boolean. A lot of C++ types (including all primitive types) support Boolean decay and so you can’t just use that directly, because a present value may decay to false. The optional would have to return something that was, effectively, the optional itself: something that wraps the target and a validity thing and uses the validity bit for Boolean conversions and uses the dereference operator to return the value. And now you’re back where you started because it is UB to call the dereference operator on the new thing and there’s nothing other than programming style to ensure that you use it only in code of this structure where you check it first. Worse, this thing now takes a reference to the underlying optional and so makes it easier to introduce memory safety bugs by having it outlive the optional.
This kind of structure is precisely what the dereference operator on optional was designed to support:
if (auto optional = somethingReturningOptional())
{
doSomething(*optional);
}
Your listing tools can (and do) check that you only use the dereference operator in places where it’s dominated by the check, but you can’t express that in the C++ type system.
An ‘optional’ is a special case of a sum type (optional a is a+unit), which can in general be realised in a language like c++ by church-encoding them; I’m assuming this is basically what the ‘monadic operations’ do.
Situation: Clang’s UB sanitizer actually complains about unsigned overflow.
It doesn’t unless you tell it to. The OP in that question had enabled more than just the default “undefined” sanitizer, and it was another sanitizer option that was reporting the unsigned overflow. From the accepted answer:
In -fsanitize=address,undefined,nullability,implicit-integer-truncation,implicit-integer-arithmetic-value-change,implicit-conversion,integer all except address and undefined enable UBsan checks which are not flagging actual undefined behavior, but conditions that may be unintentional in many cases
i.e. “-fsanitize=undefined” won’t report unsigned overflow.
… and buggy. Last time I had to use Jira there were half a dozen bugs that really got in the way. Like the formatting language being completely different depending on the screen at which you started editing a ticket, and the automatic conversion from one formatting language to the other being broken so that occasionally, even if you did everything right, sometimes your ticket would end up with piles of doubly escaped garbage instead of formatting. This wasn’t an extension, this was core Jira. Though I suspect that particular issue is fixed now.
Are you on a cloud instance or self-hosted server? I’ve seen both be slow, but self-hosted is usually the worst IMO (under-specced or poorly-configured hardware, I would guess).
Yeah, I’ve experienced slowness with both. Even with self-hosted and throwing oversized hardware at it, it still tends to be quite slow (albeit a little faster than cloud hosted).
If the feature set of the processor was driven by the sales team trying to make every last sale and meet every requirement no matter how weird, and the engineering team didn’t have a say in how it was designed, yeah I probably would!
I mean, if you put it like that, yeah I do kinda hate modern x86_64 CPUs for those same reasons.
They’ve been trying, for sales reasons, to meet the increasingly ridiculous requirement “more single-thread performance for the same old instructions”. And this has driven them to make increasingly dangerous engineering decisions, resulting in the slew of CPU vulnerabilities we’ve seen, along with mitigations for them that undo most of the performance wins.
I’d say that ties into my reasons for disliking it too. I think many of the “features” added were just to get more sales, no matter how it crippled other things that were working just fine. Meanwhile, highly requested features go unimplemented for years because Atlassian doesn’t think it’ll make them more money.
It’s pretty hard for IPv6 to succeed on a personal level when Fucking Verizon doesn’t support it for home users. They were claiming they would deploy it to all their customers in like… 2015 or something, and then just shrugged and never mentioned it again. AFAICT Comcast is similar. So something absurd like 70% of the US population just aren’t gonna have an IPv6 address.
Yes I can use a HE tunnel like it’s fucking 2003 again but all in all I’m pretty sick of it.
That said, it’s pretty shameful that github and datadog don’t even support it. Every VPS I’ve paid for since like 2010 has come with IPv6 addresses by default. …that said, now that I look at it, Sourcehut doesn’t have an IPv6 address either. Maybe time to consider Codeberg, hm?
EC2 VMs don’t get public IPv6 by default. And thus, a lot of stuff running on AWS just never gets around to add IPv6. Even Twitch, now owned by Amazon, still doesn’t have it.
That’s interesting. I’m both a Verizon wireless and AT&T business customer, and Verizon is the only provider that doles out both ipv6 and ipv4 addresses to my LTE router upon connection. AT&T business never provides ipv6.
So it’s clearly not a technical problem for Verizon. They’re choosing not to give ipv6 addresses to home users.
It’s a slow rollout. My understanding is that Verizon is going region-by-region to reduce the support burden. My residential Verizon Fios connection near NYC magically started receiving an IPv6 address about a year ago, and it was seamless enough that I didn’t even realize for a while that most of my traffic was going over IPv6. It has been smooth sailing ever since.
I wonder if maybe all of these evil vendors aren’t throwing their customers under the IPv6 bus because it doesn’t matter to the vast majority of the customers. Because 98% of their customer base don’t know or care what an IP address is, and as long as TIkTok streams and games can be played, why would they? There just isn’t a value proposition for the average user that demands IPv6.
If supporting both is more expensive than supporting one, and your choice is between v4 and v6 OR v4, … how would it be easier? I’m not savvy to why v6 has taken so long, but it certainly seems like a network effects kind of thing.
It doesn’t. Which is why we network admins haven’t jumped all over v6. We’re lazy…if it was easier v6 would be the de facto everywhere, and would have been for years.
“x is a disaster” after very little experience. That’s an orange-site sentiment.
FWIW I’ve used IPv6 in production for many years and feel positive. My comparable setup is: The iron has an IPv4 address and provides outgoing IPv4 via NAT to all virtual hosts, spinning up a new virtual host assigns an IPv6 address but no IPv4 so there’s no incoming IPv4, and… that’s all. I don’t even try to avoid using outgoing IPv4, and I don’t need to ever think about whether I have addresses on hand.
Sometimes people need to access one of those servers and can’t. That happened to me only this summer. They accept that it’s their problem, not mine (and while I feel that this sentence should end with a smiley I’m not sure which).
Not in this case. It is a disaster if all you wanted was a normal, working host, and you got a ipv6-only thingy, that can’t even fetch things from github by default. That’s a major no-go if you expect this to work like any other machine. Whether it really makes ipv6 a “disaster”, or more a statement for how slow-moving corporate networks are is up to the reader. IPV6 certainly “just works” if you have a dual stack. But for now only then.
Just a speculation, but if you have a working CDN with load balancing, blacklists and rate limiting, it can be very annoying to adopt ipv6. Even more when you suddenly not only need ipv6 but rather /64 matching (for blacklisting or rate limiting). It’s a giant change in your infrastructure, and we all know what brought AWSfacebook down: a broken corerouter configuration.
Another great example of people leaving lengthy comments about what they think the headline of the article implies without engaging in any part of the contents, probably not even reading past the headline
Resources are limited and time is finite. Rolling out IPv6 to satisfy the aesthetics and “best practices” of people who are unlikely to have a direct stake of real size is a bad idea.
As much as everybody complains about it, the current system works and is a hell of a lot simpler than IPv6. Maybe, if a technology has taken more than two decades (27 years if you ask Wikipedia) to get adopted, there’s some flaw in the technology.
If you are suggesting that Github or Twitch or others lack technical leadership, I think the empirical data disagrees with your proposed metric.
the current system works and is a hell of a lot simpler than IPv6.
Is it? From the blog post there’s a picture of the IPv6 header, and it looked like it has much fewer fields. At a glance, the core of it look simpler. What are the more complicated parts?
One of the transition mechanisms (6to4?) did something like this. It mapped the entire IPv4 address space into the v6 address space. This came with a lot of problems. First, you still need to handle different packet formats, so the boundary involves some translation. Then you have protocols that embed an address in the message, which doesn’t get translated. Then you get the problem that things on the IPv4 side look like they’re on the v6 side, but can’t connect to most hosts there.
Yes, though the bridge can be on one of the endpoints or on an ISP’s network. I believe the idea for several of these was that ISPs offering both IPv4 and v6 would bridge the networks so that their v4 addresses were all visible and servers could move to v6 and still support legacy v4-only clients.
If what you wanted was a normal, working foo, and you got a something the designers of foo considered weird, and it was a pain to use, would you consider that a failure of foo, or would you consider it more of a PEBKAC case?
Dualstack was considered the migration when the various IPv6 designs were compared and the one we have won over the wothers.
If you have dual-stack, you might as well just run ipv4. Because cost-wise, you already needed the ipv4. Access-wise you only need one (ipv4, the v6 people will already need something to get v4 access, but not vice versa). Complexity wise, you’re better off with only one.
So what exactly do I gain by having ipv6 compatibility, apart from internet karma ? Everyone else (relevant) seems to get away not caring about ipv6.
I run dual-stack on my hosts, but I also don’t pay for ipv4 on top, and I only do so since I’m running nftables and have not as much complexity. I also can’t even use the potential of my /64 (for containers etc). Because then I loose ipv4 compatibility. So it’s basically the same as ipv4, except I need to address/debug everything twice.
The cost of complexity is actually why I started doing that. Getting a public IPv4 address onsite required meetings and negotiation and justification, costly complex stuff, so I wrote a script that set up a new VM with a publicly reachable IPv6 address, a DNS entry and NAT for outbound IPv4.
Sounds good, but how do you handle accessing stuff purely from IPv4 ? Because that’s my main problem: making it accessible for both - and thus loosing the freedom of my /64.
It can be a problem. Earlier this summer I held a workshop where I said something like “I made an example site showing [blah blah], it’s at [link]” and some of the participants couldn’t access it because they had no IPv6. But mostly it’s okay. Either because the people who are supposed to use it have IPv6, or because they don’t but perceive it as their problem rather than mine.
This looks interesting, but it’s lacking a bunch of information, so I can’t really make sense of it.
Most importantly: What is this intended to do? What are the properties that it is designed to achieve?
The only thing resembling a “design goal” that I can see would be
To make sure all fork routers can access each other, […]
but it’s unclear to me how the proposed solution
their second route—their reline—points to the router that points to the router that points to them: forming a circular reference.
would ensure that. All it ensures is that routers can reach at least two other routers and themselves.
It is easy to come up with a fork-routed network that satisfies the reline rule but where not all routers can reach all routers (i.e. it is not strongly connected).
For example, you can take any two fork-routed networks and lay them next to each other. The new network has relines, but is not strongly connected.
Or you can take any directed graph with a vertex outdegree of 3 everywhere that is not strongly connected and replace each vertex with 3 fork routers that form a reline loop, turning each of the outgoing edges into a 0-route for one of these fork routers.
It feels like maybe there are further assumptions about the network being made, but these are left unstated here.
oh man. maybe im too stupid to know what details to write lol
bro i rember spending hours thinkin about it til i was confident it would work..
like think about it: if a router can access its previous one through its reline, then it can access its previous previous one cuz its previous has a reline to its previous previous one. and if you can access the previous previous one that means you can access one of the routes of the previous previous one, and so on. so u can literally go anywhere
not necessarily. you can choose the other route of the router you’re pointing to with your reline. and you can also choose not to go through your reline. just look at the diagram: not everything is a loop
The undo keyword reverts the last mutation made by the current thread - mutations made via undo itself don’t count for this, so you can do it several times to travel back in time.
Related: the @ operator when used with a variable lets you specify a particular index in the history of that variable. Positive numbers as the second operand mean steps backwards in that variable’s history.
x = 3
x = 5
x = 2
print(x)
// => 2
print(x@0)
// => 2
print(x@1)
// => 5
print(x@2)
// => 3
Thanks for bringing this up! It’s honestly my favorite part of LOBSTERS-LANG, and doesn’t get used enough. It’s super flexible too. If you use a negative number it’ll use linear regression to look into it’s future.
The last two businesses I have left, I’ve exported the custom emojis from slack (search GitHub for various scripts to achieve this easily) and then added a bunch to new $work’s slack upon joining. Not all of them of course, the in-jokes only work in the original place.
Is that functionally different from allowing inline images? Like can any user send any image as an “emoji” or is there some process by which emojis must be approved before they can be used?
Somewhat, yeah. Having shared, readily available reaction images allows for creating a community language, same as in-jokes or oft-repeated quotes. Using :sickos-yes: has a different flavor than :100: which has a different flavor than :clap-cowboy:, even tho they all mean “I agree with the statement”. But those meanings come from usage and situation, so if they have to be manually curated by each user, they won’t be used nearly as much.
Oh, do you mean “how are they added to slack?” I believe it’s configurable per server, but the servers I’m in allow any user to add a new emoji or create an alias for an existing emoji, and it supports various static and animated image formats.
You say UNIX but then depend on many specifics of GNU/Linux ;) I haven’t actually run the code, but what I’ve spotted so far:
POSIX doesn’t mandate tools to live in any specific location, so you shouldn’t hardcode /usr/bin/. For example, on FreeBSD, echo and date are in /bin, and bash in /usr/local/bin
I’ll change the id and date invocations. As for the full path, there is a security reason behind them: if the user shadows them, she can run stuff under the effective privileged ownership of suc. By using the full path, I’m making sure there are no $PATH shenanigans going on. I should update the article to reflect that.
Yes, or rather, you can setuid it but Linux ignores it. Part of the reason is exactly this sort of $PATH problem. suc works around this by also shipping a C wrapper program that you can setuid and that calls the shell script.
Also, that program appears to have a bug, it fails to null-pointer terminate the argv array passed to execv.
But I think these kinds of nitpicks are holding back shell, e.g. most people don’t want to read id -unr
Agree with just using date, and not /usr/bin/date
The real answer (or at least the future facing one) should be that all Unixes support containers (OS virtualization), so you can build GNU tools or whatever you want inside them, without messing up the rest of the system
FreeBSD, Solaris, and Linux do, and probably a few others. I’d say that any others should get with the times
chroot might be technically sufficient for this case, but we also need common tools. From experience, it’s possible to do it with plain shell scripts, though it’s a lot of work, and some higher level abstractions are useful
Well, that’s rather dismissive, it’s like saying “meh, if you actually need to support other people than me…” :)
The article claimed to be a demonstration of modern UNIX and I only mentioned things that were genuinely broken on 2 of my UNIX systems (macOS & FreeBSD)
Also note that this isn’t portable either on GNU/Linux distributions, e.g. NixOS. I would say relying on $PATH is the best bet, and it’s the user’s job to be wary of the what’s going on in $PATH.
The danger is not to the user ! The user is the danger !
By playing with her $PATH, a user can execute arbitrary code owned by suc, which would let her e.g. edit the chat log (which is supposed to be append-only).
This is a general theme I’ve noticed with Linux: All the stuff I want is basically there, it’s just way more complicated and annoying than I’d like it to be.
Another example: OpenBSD gets praised a lot (rightfully imo) for its “pledge” system. Linux has something to do the same job, seccomp-bpf, but it’s so hard to actually set up that its usage is mostly limited to a very small set of core services.
There is most of a decent capability-based OS buried somewhere in Linux, you can send around file descriptors for almost anything (processes, sockets, memory, even namespaces themselves!) and you have a lot of tools to eliminate ambient authority (which is of course still there by default).
edit: and of course Linux also has a subsystem literally named “capabilities” that has nothing to do with any of this and is just generally awful.
It’s close, but it doesn’t quite do what pledge does. For example, you can’t pledge the shell to avoid doing any network I/O, while allowing programs spawned by the shell to still work.
It feels right that permissions obey a hierarchy, but it turns out to be very useful to break that hierarchy. It’d be even better to do it deeper than just a parent/child relationship, but that gets really complicated, so that one layer is a reasonable tradeoff in practice.
What’s the point of not letting the shell directly access the internet, while still letting it spawn other processes that can access the internet? Or am I misunderstanding?
Imagine that you’ve managed to find some bug in your shell; you want to pivot it to a complete expoit, but you don’t have any syscalls that you can use. If you’re careful about locking things down, you may have unveiled only a few binaries in the path, so you can’t even run arbitrary programs. Now, not only do you have to find an exploit in the shell, you have to pivot it to execute a program that will do what you want, and maybe garble its arguments in a way that it will give you more access.
In a privsep daemon, you may also have a spawner process that primarily has the job of creating processes that do the work; that process needs very little permissions to work, but the workers may need to do more.
It’s not a panacea, but it makes your attacker’s life a lot harder, and makes it possible to ratchet down permissions far more tightly.
Anyone know if any of those “archive software for a million years and have it still usable for whatever species digs it up” efforts have ever considered Wireworld as an aggressively minimal CPU implementation? It might be a bit too minimal, but it’s probably the only thing Turing complete thing I’ve seen simpler than Conway’s Life.
Hmm, I wonder if one could write a compiler to compile arbitrary programs to Wireworld “hardware”…
I was at a computing history museum at the same venue as the 2023 vintage computer fest and showed the people running the museum wireworld, and they asked the same question. :) They also said gosh, maybe we should be using Wireworld as a starting point to teach the basis of composable computing components to people with no prior experience. Which was pretty neat to hear people whose efforts are to teach people exactly those topics get excited about that same idea!
I’m not sure minimalism in the rule set is the right thing to optimize for. The proof that rule 110 is Turing complete is very involved – It’s sort of as if complexity was removed from the automata and moved into the “interpretation” of what a particular state means.
Wireworld seems to be in a nice space where the automata is both simple and “obviously” Turing complete.
Idk about “obviously”, but certainly “more trivially” in that the proof is pretty small and/or slots into other common proofs. More or less just make a Wireworld NAND gate and say “ok you see where I’m going with this”.
Is this just wrapping ancient POSIX APIs, generally regarded as awful, in WASM? If so, that’s kind of sad, i.e. we’ll have to live with these APIs for another few decades.
That was my reaction as well. With WASM, we have a rare opportunity to build a platform that breaks legacy and provides clean abstractions that weren’t designed for single-core minicomputers, so let’s take advantage of this by directly porting the worst ideas from that world. Oh, and let’s make absolutely share that we don’t think about security at all when we do it. No thanks.
Yes, this is the “Worse is Better” to the Bytecode Alliance’s “The Right Thing”.
They are faster to the market, but also I feel like everybody whose problem looks like “I want to run legacy Linux applications in a lightweight sandbox without source modifications” is already well-served by the container ecosystem.
This is not a Bytecode alliance protocol, so it’s just low quality fragmentation on their part. No one else is going to support it. Also, that company is generally malicious.
I disagree. It’s exactly the kind of poorly designed nonsense that solves immediate problems (how do I run this program that assumes a Linux host in a WASM environment without any porting) and creates massive amounts of technical debt along the way that usually succeeds in the market.
As a previous insider, I assure you companies are not very interested in running stuff in wasm. The only interest that existed (back then at least) was from the cryptocurrency hype which has since died. This stuff adds so much tech complexity which no sane person would choose over running containers.
Also, this implementation in particular is written by a guy with no programming/engineering background (look him up) so this technical debt you say may pile up too fast. Company’s a clown show.
My biggest fear is that, before Wayland is “ready” and X11 is “deprecated”, there will be the third thing (or there already is?), which is “gonna be better than Wayland”, and “finally fix everything”.
We’re past that. That was Canonical’s Mir. That’s Arcan. Yet, the future is still Wayland.
Why is the future still Wayland? What does Wayland do fundamentally better than Arcan?
Because all major desktop environments and distributions target Wayland, and Arcan isn’t anywhere near that level of adoption.
As far as I see*, Arcan is available in AUR, Guix, Nixpkgs, FreeBSD Ports, Void Linux, and Slackware.
Considering it’s been around since ~2013-ish, so its only a few years younger than Wayland, it does not show any kind of adoption levels comparable to Wayland, and I don’t see why that would change. Thus, it’s not the future.
The future is not always the thing that’s technically the most sound, but the thing that people end up using, and that’s Wayland.
It most likely will be, but isn’t yet.
That is why it is the future, not the present.
Not sure about fundamentally, but Wayland can run GNOME and KDE.
Like it or not, that’s the bar for anything that’s going to replace X or Wayland. If regular users have to learn an entirely new DE, it’s not happening.
That’s essentially inevitable. 20 years on, The CADT Model remains the best empirical description of the Linux end-user ecosystem in action
I posted an answer to this on Discourse.
I’d also like to link to this very relevant new blog post, “Nix Flakes is an experiment that did too much at once…”
I appreciate your perspective, but I disagree with your opinions. Specifically:
flake.nix‘s grammar and semantics are a strict subset of Nix; it’s not inconsistent at all. It is a downgrade in expressive power, and that’s a good thing when we consider POLA and packages-as-capabilities.systemmust be explicit for good cross-compilation. Yeah, it’s a little irritating, but the alternative is vaxocentrism, and anybody who survived the 32-to-64-bit transition will agree on the value of “i686” and “amd64” being different strings.If FlakeHub is going to bolt on a centralized version resolver, I really want flakes to properly tackle it in a decentralized manner instead. FlakeHub version resolution meshes poorly with input following and machine-scope flake registries. I am generally worried about FlakeHub becoming how the Nix ecosystem deals with flakes; it’s a glaring point of centralization in a feature set designed for decentralization. Flakes currently only have one enforced point of centralization: the global flake registry, managed by the NixOS foundation.
I agree! System sets also need to be extensible.
systemsin the centralized flake registry is the only working solution I’ve seen so far, and it’s woefully underused. We need to heavily enforce one convention across the whole flake ecosystem for extensible systems to be usable, or we need to extend Nix with an official flake system registry that’s also extensible.I’m curious: Don’t you dislike the fact that basically the entire point of flakes is to avoid the Nixpkgs monorepo? The ability to have a monorepo is a key advantage of free software, and the main reason to avoid it is to support proprietary software. I’d rather we keep the monorepo and just educate people better on how to use it, rather than dismantle it with flakes.
I’m not entirely sure that is the point; I have a bunch of github projects that are deployed out of flakes. There is little to no reason for putting them into the nixpkgs monorepo, as they’re basically used by me only. There is no wider interest that would justify putting them into nixpkgs and nixpkgs would only add massive delays to trying to push new updates to my servers.
edit: I would also like to add in a quick edit that nixpkgs contains plenty of non-free software, such as VSCode. Free software does not enable monorepos any more than non-free software does and monorepos can contain both free and non-free software.
Aren’t you just agreeing with me that the point of flakes is to avoid the Nixpkgs monorepo?
Anyway, if your projects don’t have dependencies on each other, then you don’t need flakes. If they do, then maybe the common component should be published as something shareable, which could go into Nixpkgs. Or just put the projects which depend on each other into a single repo.
The project in question is a collection of various web services, they do not make sense in isolation but don’t share much common code (but some do and the flakes in question are pointing the right direction), if they’re even in the same language.
The issue is that again; this is code solely deployed by me that in parts even only makes sense being deployed by me, in other parts simply stuff I deploy to servers for others. There is no reason to pull them into nixpkgs and increase the maintenance burden of everyone there because they have to maintain some random code I’ve barely touched for a few years.
In this matter, I see nix flakes no different to a Dockerfile; it’s a way for a maintainer to bring availability of their code to a wider audience without having to play the maintainer game of thrones. The upstream is in full control of the deployment and determines the defaults and configurations, as it should be (before going for NixOS I used ArchLinux in part for the reason that they have very minimal patches on top of every package).
There is also no reason to put all my projects into a single repo, I do not like the monorepo approach. It’s a repository approach propagated by the likes of Google and Microsoft, it’s hostile to small open source projects due to the increased maintenance burden. Nixpkgs gets away because it’s got plenty of subtree maintainers, I don’t wanna put that on myself just for my own silly bits of code.
And no, flakes are not there to avoid the nixpkgs monorepo, evidenced by the fact that all of them pull it in for common components. That argument would hold up if the majority or a good slice of flakes, didn’t pull in nixpkgs at all purposefully to avoid it. But almost all flakes hit nixpkgs in various ways.
So nixpkgs should be filled with a bunch of “caterns_repos” folder for each developer having a public hobby project? You can surely agree that it doesn’t scale there.
Nikpkgs does not scale as it is. This is an absurd suggestion.
This is not a reasonable suggestion and does not make sense for really any project I think.
It is a point, but I don’t believe “entire point” does the multitude of motivations for flakes justice. I’ll defer to today’s other frontpage Nix post [1.] and provide this list of Flakes features:
I’m not expert enough in Nix/Flakes or the history of Nix to say whether these other are the most salient motivations for flakes, but collectively they constitute a body of work that goes significantly beyond what you describe as avoiding the nixpkgs monorepo.
Similarly, I believe this statement is too absolute. The main reason to avoid a monorepo depends on a user/project/organization’s motivations. E.g. my main reason to avoid nixpkgs for a silly little di.fm player [2.] that I develop is that I don’t think it belongs in the nixpkgs monorepo. Few Nix users would benefit from it, as it has few users, and never will have a significant number of users. Going through an additional step of review every time I make a release would take away some of the joy I get out of developing the application. This doesn’t mean di-tui has proprietary aspirations. It’s open source and will remain that way.
[1] https://samuel.dionne-riel.com/blog/2023/09/06/flakes-is-an-experiment-that-did-too-much-at-once.html
[2] https://github.com/acaloiaro/di-tui/
nixpkgs is massive, and it is absolutely inconceivable to be able to store, say, the whole python library ecosystem, plus the whole of nodejs and whatnot. There is simply a limit on what can sanely fit inside it and it is already bordering on that limit.
I believe the duality of Flakes and a good base monorepo is actually a quite great approach as is: have a rolling release “blob” of mostly working together, common software I can refer to always and let more exotic software be defined and linked from anywhere – you end up with a Flakes file that pins a working nixpkgs.
Why not? I can and do upstream Nix expressions for Python packages I use. I only use what’s packaged in Nixpkgs. Works fine for me.
There is no scaling limit to Nixpkgs, and I’m wondering what is leading you to believe there is one.
Who approves the PRs? It already takes a lot of time to have to have something merged, because the maintainers are spread so thin.
That doesn’t matter, because you can use your fork while waiting for your PRs to be merged.
As an outsider that’s an interesting statement!
Disclaimer: I only use nixpkgs on other Distros, I have never installed NixOS on its own, but I did read up a lot over the years and tinkered, and before flakes I could sum it up as “Nice in theory, but completely unusable for the amount of work I am willing to invest.” - with flakes everything seemed easier and made more sense.
No, I don’t care whether a ports tree is confined to a single git repository, especially not at the size of something like nixpkgs. Flakes only act as a federation mechanism; the integration of code still happens in the same fashion either way.
But that’s not true! Federation in this form requires “stable APIs”, a deformity usually induced by the presence of proprietary software which can’t simply be patched and recompiled for new APIs. “Stable APIs” are only needed when implementers can’t just update their users for the new version. See http://catern.com/freetech.html
We might be thinking of different actions. To me, the integration of code is what the Nix daemon is doing when I hand it a build request. That request will involve two git repositories already (nixpkgs and the upstream repo for the target package), so I’m neither bothered by the idea that the second repository will have Nix expressions, nor by the idea that the second repository is fetched prior to the first repository.
At no point here am I advocating for non-free software. I will point out that, by design, a ports tree usually has no problem expressing the integration of free and non-free software; nixpkgs already carries many such expressions, regardless of whether I use them.
Stability of a Flake API is almost entirely irrelevant from this perspective, because the versions of expressions are locked in
flake.lock. Lockfiles provide bitemporality, disconnecting the time of integration from the time of locking.That is a great advantage of flakes though. Because you can make much more than just packages. See home-manager, for example. Installing it as a flake is considerably easier and more reliable than as a channel. It makes developing Nix module sets that wouldn’t really fit in NixOS considerably more practical.
Yeah I thoroughly agree with you there. Using a lockfile without version solver and a nice UX to upgrade versions is simply calling for so much pain.
I don’t get the version solver part. Those expressions don’t have versions or version constraints. You refer to some branch or tag and that’s it. An alternative version solver could be built on top, but that would also require flakes themselves to be versioned differently than people do at the moment. (One foo-2.0 doesn’t equal another foo-2.0 when they pull in different refs of the whole system they depend on)
And that’s more of a nixpkgs thing than a flake thing.
Version solvers make a lot of sense in systems like apt or pip, in which you can only have at most one version of a dependency so everyone needs to agree on what that version is.
However, this can have some weird non-local effects. For example, you can have
libAdepending onlibB ^1.2, and in libA’s CI they uselibB1.2.0 to run their tests. Now a downstream project that depends onlibAbut that also (either directly or via one of its other deps) requireslibB ~1.3would silently uselibAwithlibB1.3.0, which it was never tested against.In Nix, everyone can have their own versions of all their deps, and everything is used with the same versions of dependencies when integrated into a larger project as it does as an individual software project. This makes reasoning nice and local, but it also wastes a lot of disk space and network bandwidth.
I think that’s a decent tradeoff a lot of the time.
Yeah, it also seems to me like a usual case of people recognizing a pattern they’re familiar with and assuming that they can carry over all of their experience and mindset because of the similarity. There are specific reasons why version resolving works well for programming language package managers, but they don’t necessarily apply to nix flakes. In addition to what you said, here are some differences that come to mind:
inputs.my-nixpkgs-for-a-very-specific-purpose.url = github:myorg/myrepo/my-ad-hoc-branchand usemy-nixpkgs-for-a-very-specific-purposefor that specific purpose without disturbing anything else.I often see a similar thing happening when people criticize Nix for being Turing-complete or untyped etc. They just carry over reasoning from related but different domains without reevaluating them for the unusual case that Nix is.
Yes and right now it is what channels handle. But with flakes, the whole point is that you only change things when you decide; that is what the lock is about. So now you have to take care of every single version change of your dependencies and how they may not interact well with each other yourself. It kindly becomes unwieldly
I don’t believe that’s a technical problem which can be solved. You always had to do that yourself. The versions constraints in libraries are mostly just indicative anyways. (we’re fixing some random issue from package updates every other week) If you’re depending on two unrelated projects and reference them both in your flake, that means they don’t depend on each other anyway and there would be no constraint anyway, because you can’t realistically do a full compatibility matrix of everything installable.
For example someone writes a Frobnicator which uses /tmp/foo as a lock file and someone else writes a Bloberator which does the same. When they get independent flakes, who would you expect to test all the combinations and where would the constraint of “those two can’t work together” even live? (And how do you express that they can work, but not at the same time)
I tend to disagree with this thinking. Daniel dismisses the issue is a security hole and then directly asserts no one would even use this feature anyways as if to bolster the argument around it not being a problem or security issue.
To say it’s a buggy feature that no one hits in practice anyways is fine, but it is entirely orthogonal to whether or not this is actually a security issue. If this bug is used as part of a security vulnerability (or within an exploit chain) “they” would deliberately use whatever means they can to trigger UB or buggy behavior to further the exploit.
What you’re saying is exactly the philosophy that leads to CVSS scoring everything “critical”. You can always imagine a scenario where someone would set up themselves for a colossal failure when hitting even a smallest bug.
If you don’t include probability of this happening in your evaluation, you get absurdly inflated danger. You have to consider how unlikely it is that someone would allow attackers to specify the timeout with arbitrarily large numbers, while also critically rely on this timeout to be in a specific range.
Yes, CVSS is entirely ignorant of this dimension. If there is any imaginable deployment scenario in which the vulnerability is triggered by input from the network, you get AV:N (“Attack Vector: Network”, the worst one). If there is any imaginable scenario where the vulnerability causes severe harm to confidentiality or integrity, you get C:H and I:H - as curl did in this case.
Because you can nearly always come up with very bad scenarios, the most common CVSS by far is exactly 9.8 - everything maxed out except “Attack Complexity: Medium”. Thus, use-after-free in nginx’ HTTP request parser is a 9.8/10, but so is this total nothingburger in curl. Because in theory you might deploy it in a way where attackers get to control specifically
--retry-delayover the network and you might compile it using a hypothetical C compiler that turns integer overflow UB into arbitrary code exec.For us practitioners, this quickly turns it into a “boy cries wolf” situation. If every CVE is severe, none of them are.
This is why, when I’m writing security announcements, I try to be very descriptive about how the vulnerability might be exploited. That way folks can actually understand how much of a nothingburger the vulnerability actually is. Unfortunately that requires some domain knowledge on the part of the reader, so folks that don’t have that kind of domain knowledge are stuck trying to interpret a number. The whole numbering scheme is very problematic IMO and a source of pain for me personally.
I definitely agree with what you’re saying, so it’s possible I didn’t articulate my thoughts correctly. I 100% agree CVSS scores are often inflated and you have to take into account the likelihood of someone hitting said code path to be a vulnerability.
My point was also not talking about this specific case with curl which I agree is probably a non-issue. I was speaking about a hypothetical scenario in which lets say some program had an argument
--foo Nwhere some values ofNlead to a real vulnerability for example--foo=999, but in normal use of the program no real users ever setNto that vulnerable value, they all use--foo=3or--foo=1. I was saying it doesn’t matter that normal users don’t use the vulnerable value, an attacker easily could. So I think in my made up scenario I’m weighing the “ease” at which an attacker could use the vulnerability, and entirely disregarding whether or not a normal user uses the vulnerable values.But I’ll concede my thinking isn’t really applicable to this case, and thus confused the conversation.
I agree that he asserts it’s not a security problem. I would not agree that he asserts it’s not an issue, since it was fixed.
It also is apparent that Daniel spent the time to understand how this would behave in practice, and supplied evidence that it was not a risk in practice.
(down-thread)
Daniel is direct about the idea of hypothetical/theoretical problems not being sufficient. I do think actual an demonstration is a better place to come from than a hypothetical that then needs to be exhaustively disproven - and it can’t be, the hypothetical space is infinite.
I agree with Daniel here. Unless there exists a C compiler that does turn the UB into arbitrary code exec, there seems no reason to even consider that a real possibility.
It’s just a matter of time before someone creates the world’s worst C compiler which is technically compliant with the standard specifically to win an argument about CVE severity ranking.
People have been joking about the Deathstation 9000 C compiler for like 30 years, but no one has (as far as I know) implemented a compiler that actually uses the most perverse possible interpretation of the standard. Perhaps the closest is the Frama C static anslyser?
Haha yes, I am sure someone will be crazy enough to do this. Especially now that our discussion is public! :-)
Shouldn’t be that hard to change ubsan from reporting issues to execve(some_suid_binary) instead…
And, even if it was arbitrary code exec, I’ve still seen zero examples of a realistic scenario where an attacker would be able to control this parameter without having arbitrary code exec anyway.
The problem with relying on UB to escalate severity is that it’s not possible to write a non-trivial C program that’s completely free of UB, at which point everything just escalates automatically to max severity because there’s probably a code path that hits UB.
I’ve personally switched over to Bitwarden, and am in the process of getting my family out. I migrated (easily!) last year and have not regretted it once.
Bonus: you can (should you so desire) self-host the Bitwarden sync server, which was a draw for me, but so far I’m happy to let them handle server maintenance, backups, etc.
I’m self-hosting Vaultwarden and it’s been working flawlessly. NixOS makes it quite easy via
services.vaultwarden. And on the clients the only thing you need to do is put in your custom URL at signup.I’m possibly not the best at server maintenance, but all the apps work even when the server is down or the client can’t connect to the internet at all. They have a local copy of the encrypted database.
I run mine on a raspberry-pi on my home network (using docker-compose). Super straight forward and backups are trivial too
I’m also planning to leave LastPass. What made you choose Bitwarden? Did you look at any other options?
I also switched from LastPass to Bitwarden. I’ve wanted to do that since LogMeIn took over, but it was low prio until the breach last year (mentioned in the article).
I picked Bitwarden between all the alternatives for a few reasons:
The switch was relatively simple - export from lastpass, import in Bitwarden, and I kept both for maybe a month before a full switch.
Also running vaultwarden here, installation was easy, and having the official android and firefox/chrome addons from bitwarden that can talk with my own vaultwarden instance is very nice. I would only ever use something that can be self-hosted and is open-source, so things like lastpass and 1pass wouldn’t be acceptable. I still have / keep all passwords also in ‘pass’ (passwordstore.org) … as having a git log of everything, and it’s massive flexibility and programmability is a thing I wouldn’t want to lose.
Am I misunderstanding something or does bitwarden gate the generation of TOTP codes behind a paid plan for their hosted service, i.e., preventing a self hosted instance from being useful for TOTP?
The official Bitwarden server does, but the more efficient Vaultwarden server has no such limitations. Although, it’s a personal policy decision of whether you want the same store to hold both authentication factors.
Good to know, thanks
Nix (the language) comes to mind, especially viewed against NixOS competitor Guix’s use of Scheme. But I don’t know if I’ve ever encountered a tool written in Scheme, so it’d probably feel as much like a DSL to me as Nix does.
There’s some proposals to refactor Nix to make the layers more separable, which would eventually allow languages other than Nix to be compiled down into derivations and realized in the Nix store. I wonder if there’s a general-purpose language that would be good for that, other than Scheme.
I’m also following that Nix proposal. I think it’s a great idea and am glad the RFC was accepted.
Here’s one reason this is no panacea though: Anything that people will actually want to use will have to be able to call into Nix. Because otherwise, you get no access to nixpkgs, losing you one of the biggest benefits of adopting the Nix ecosystem.
That still leaves you with a few options: You can shell out to nix-instantiate, you can compile to nix code or you can use the little-knowm plugin system of nix that allows you to extend the interpreter. However, tooling for existing general-purpose languages is not really there yet.The only thing in this area I’ve seen is https://github.com/purenix-org/purenix
Typo in your URL, fixed: https://github.com/NixOS/rfcs/pull/134
Tell me you know nothing about C programming without telling me you know nothing about C programming…
The whole “binary input and output” section seems like nonsense to me. There are also a lot of complaints about C not guaranteeing specific kinds of integer, but C only doesn’t guarantee them so that it can be ported to platforms which don’t have them. The fixed-width types do always exist if you restrict yourself to targets that can run Javascript.
The stuff about overflow and out-of-bounds array access is fair enough, although I personally think Javascript does about the worst thing a memory-safe language could possibly do in both cases (yield a value instead of indicating an error).
Which is ridiculous! How many users want to be able to rely on fixed-size integers vs how many users want to target a platform that doesn’t hav 8/16/32 bit integers? (How many have even seen one?)
I obviously hid this fact better than I meant to in my last comment: The people who don’t want to target those weird old platforms can rely on fixed-width integers, and have been able to for decades. The weird concessions C made to support weird old architectures sometimes affect programmers who don’t care about those architectures, but this is not one of those times. Nothing in the C standard requires you to write code that will work with 9-bit bytes.
I thought I was the only person left in the world running smokeping. Cheers!
I also run smokeping. So that’s 3, and by extension infinity.
Also which makes 4 and therefore double infinity.
Make that 5.
Also, it’s really easy to setup on NixOS:
services.smokeping. It’s not too heavy and it can be a lifesaver when things break, so there’s little reason not to run it on servers.Obviously we need to start an irc/mailing list where we can come together and show cool things we do with it!
Quick question, what’s the diff between smokeping and uptime kuma? What do you use them for? I have uptime kuma just for monitoring a few sites, I don’t need anything fancy, but I also brought it to work so we can also use the dashboard export thingy. What extras does smokeping have in comparison?
Smokeping is for Network latency. So I think of uptime kuma is more.of a heartbeat to check if the site is either up or down and notifies you if they are down. And smokeping does analysis of packets sent. But very much could be wrong. I stood up both and they are very similar except for UI
Great list!
How do Cloudflare Tunnels compare to Tailscale Funnels? (Hope I got the feature names right)
The main downside(s) I’ve found with Funnels is a lack of custom domain names (you’re stuck with your Tailscale Net name) and generally slower speeds when running through a relay. I ended up spinning my own using DNS splitting and a free tier VPS.
Wait, how are you self-hosting tailscale? Do you run headscale? https://github.com/juanfont/headscale
I misspoke, apologies. This list is more of a “homelab” list, especially with cloud flare tunnels and tail scale. I’m not using headscale. I’m using the non self hosted tailscale
I see a bunch of instances of
unsafein the codebase. Does it still count as memory safe?You know, I was going to say that its OK, as it’s a handful of unsafe blocks around FFI, its comparatively easy to just eyeball them, and, unless you go full CHERI, unsafety has to live somewhere.
But also, one of those blocks is not FFI:
https://github.com/memorysafety/sudo-rs/blob/9a7f38fbddc59f40a5e0a57555131b36e09811e4/src/exec/mod.rs#L92-L97
And I think that’s a bug and technically UB!
CString::newallocates, and you can’t allocate in pre-exec.So kinda no maybe?
But also maybe yes, or at least much better than alternatives? This bug really jumps out at me, it’s trivial to notice for someone unfamiliar with the code. pre_exec is tricky, and unsafe makes it really stick out, much better then if it were hidden in the guts of the standard library or some native extension.
(also, obligatory PSA: the biggest problem with sudo is not memory unsafety per se, but just the vast scope for this security sensitive utility. For personal use, you can replace
sudowithdoas, and it’ll probably make a bigger dent in insecurity. If you have to be API and feature-compatible with sudo though, then, yes, what Ferrous folks are doing makes most sense to me)I often see doas recommended as simpler than sudu. When I compare the documentation I see an almost identical feature set. The only difference in security surface are seems to be the configuration. Is there more that sudo does that doas does not do?
Generally when I think of sudo’s features where replacing it is somewhat difficult it is usually around plugins and things like storing rules in LDAP.
That’s also the funny thing — I don’t actually know what sudo does. I know that OpenDoas clocs at under 5k lines of code, while sudo is a lot more (see the nearby comment by andyc). So, that’s a non-constructive proof that it does something extra!
You’re allowed to allocate in pre-exec. The restrictions the documentation mentions mostly stem from other threads potentially holding locks or having been mid-modification of the env variables.
If you guarantee that there is no threads (and other pre-conditions like reentrancy) running at the time of fork(), you can in fact malloc without any big issues.
I think in case of Rust it’s actually quite murky technically:
First, Rust doesn’t have
assert_no_thread!()functionality, so, if you assume at pre_exec time that the program is single-threaded, the calling function should be marked asunsafewith the safety precondition of “must be single threaded”. Which mostly boils down to just “don’t allocate” in practice.Second, allocation in Rust calls
#[global_allocator], which is arbitrary user code. As a general pattern, when you call arbitrary code from withinunsafeblocks, there usually is some Rube Goldberg contraption which ends with a shotgun aimed at your feet. That is, if the user tags their own static as a global allocator, they can call various API on that static directly, and that should be enough to maneuver the thing into “technically UB” even without threads. In particular, you could smuggle something like https://github.com/rust-lang/rust/issues/39575#issuecomment-437658766 that way.But yeah, this is quite subtle, there was some debate whether
before_execneeds to beunsafeat all, and I personally am not clear as to what’s the safety contract ofbefore_execactually is, in terms of Rust APIs.To be fair, pre_exec is an unsafe function for precisely this reason and it should stay that way. The API contract is largely unspecified because fork() does some amazing things to program state. I think the solution here would be to swap in a CStr over a CString, to avoid the allocation.
edit: One way we could avoid it is by introducing a new unsafe trait; Interruptable. It must be explicitly declared on structs. The trait would simply declare the struct and it’s functions reentrancy safe. A function is automatically interruptible if all non-local data it uses is also interruptible. Then PreExec could simply require the function to also be Interruptable.
It looks as if it’s intended as an abstraction over different process-creation things and
forkandvforkdo different amazing things to process state.Fork creates a copy of the address space and file descriptor table, but that copy has only the current thread in it. If other threads are holding locks, can cannot acquire those locks without deadlocking. You need to register pre-fork hooks to drop them (typically, the pre-fork hook should acquire the locks in the prepare stage and then drop them in the child, it should also try to guarantee consistent state in both). It is unsafe to call
mallocfrom a multithreaded program betweenforkandexecvebecause it may deadlock with a not-copied thread.Vfork creates a copy of the file descriptor table but does not create a copy of the address space. You can modify the file-descriptor table until you
execveand then you effectivelylongjmpback to thevforkcall (I believe this is actually how Cygwin implementsvfork). Because the code in the vfork context is just normal code, it is safe to callmallocthere, but anything you don’tfreewill leak.This is the main reason that I prefer using
vfork: I can use RAII in my setup code and, as long as I reach the end of the scope before callingexecve, everything is fine.Looking at the documentation, it only mentions the following:
Which doesn’t read like memory allocations are forbidden in that closure to me.
Ninja edit: To me, it reads like if e.g: your program is single-threaded then you’re fine.
Yes, that rule is mostly about multithreaded environments. And violations usually don’t result in memory corruption but in deadlocks. The reason is that after
forking, you only have the current thread, all the other ones are “frozen” in time. So if you forked while some other thread held a lock insidemallocand then callmallocyourself, you can deadlock.And also, glibc in particular has special code to make this work, so you can indeed
mallocafterforkthere safely.So IMO this is POSIX UB and therefore Rust UB by fiat, but very likely not a security issue in practice. It should be fixed, but it’s not super alarming to me.
You definitely can replace sudo with doas… unless you run a RHEL or a clone. I have two Rocky machines and my Ansible does not coexist with the RHEL clones as compared to FreeBSD and my myriad of other operating systems.
I have effectively replaced sudo with doas in all situations except for those platforms.
unsafedoes not necessarily mean it does not have memory safety, but that that code needs more scrutiny. I am curious why there aren’tSAFETYcomment blocks on each instance of unsafe to explain the invariants that makes it safe or necessary.Most of them look pretty clearly justified, although it’d still be helpful to have an argument why they’re correct. The unsafe blocks I looked at are all used to interact with unsafe APIs, which is sort of a given for this sort of program, but they’re short and at first glance don’t seem to do anything that would be hard to reason about.
This would mean that the whole titular claim, that this is the “first stable release of a memory safe sudo implementation”, is probably wrong. C code can also “have memory safety”, so the first stable release of a memory safe sudo implementation is probably (at least some version of) sudo itself.
Yes, but the memory safe version of sudo itself has no bugs leading to memory errors. I don’t know if that version has been written yet.
It might be my own failing, but I’m having trouble understanding your comment. That the “memory safe version of sudo itself has no bugs leading to memory errors” seems like a tautology.
If you mean we don’t know if there’s any such version, that’s technically true, and why I said “probably”. It seems pretty likely, though, and it’s certainly at least possible, bar any proof otherwise.
The point is that it doesn’t seem likely to me that sudo, as a reasonably large program in a language with few safety features, has no more such bugs. Sudo has had memory safety bugs in the past, and if I had to guess, I’d expect more to emerge at some point. You may be able to prove that there’s no memory-safe version simply by waiting for that to happen.
Of course, if there is such a version, finding it and proving it to be safe may be considerably more challenging. Which is important in itself: its safety wouldn’t have much value to us unless we knew about it.
That would show there is (was) a memory-error bug in some version(s), not all prior versions. And it’s all a bit hand-wavy; that we know that there were memory safety bugs in the past only shows that such bugs were found and fixed. From reading elsewhere sudo sounds like it’s more complex than I would’ve thought it should be, but it’s still not so big that it’s impossible that it is (or has been at some point) free of such bugs.
If you prefer, I can rephrase my point (making it slightly weaker): the linked article is claiming the “first stable release of a memory safe sudo implementation” has been made, referring to this Rust implementation, but there is no certainty that all prior implementations had memory-error bugs, so if we take “memory safe” to mean “has no memory-error bugs” then the claim is unproven and could be wrong. (It seems we can’t take “memory safe” to mean “written completely in a memory safe language”, since it apparently uses “unsafe”).
(As to how likely the claim is to be wrong, we may disagree, but there’s probably not much in the way of objective argument that we can make about it).
Even if you implement a program in a memory safe language you do not know for sure that it is memory safe.
You have a very strong reason to believe so, but there is always the possibility of a mistake in the type system design or the language implementation. After a few years the rust language has been addressed with formal methods that mathematically prove correctness for rust programs as well as specifically proving the correctness of various parts of the rust standard library that make use of unsafe blocks. This helps give very strong confidence about memory safety, but again there is the possibility of gaps.
I don’t disagree. (Or at least, I’m not arguing any of those points).
No.
Ugh. Firstly, no, C and C++ don’t prioritize performance over correctness, since turning undefined behaviour into a defined trap (for example) doesn’t make a program correct, it just makes the observable behaviour of the (incorrect) program different.
Secondly:
This is such an awful mischaracterisation of the type that often comes from the “C(++) compilers are evil, why can’t they just do what I want” camp. Clang “notices the uninitialized variable” only in the sense that it notices that the usage in the loop must be invalid (the variable hasn’t been defined at that point) and so, yes, it erases the loop; no, it doesn’t “pretend i is always initialized above 10”, in fact it doesn’t even consider the entry condition to the loop, since the optimisation pass is looking for code paths it can eliminate and the presence of definite UB on a particular path is good enough for that. Yes, it assumes that the loop body will never execute and logically this would mean, in the absence of UB, that i > 10 at entry to the loop but of course UB isn’t absent and so that logic isn’t right (and the compiler isn’t using it, despite the claim that it is). Clang doesn’t report the error because it doesn’t know there is an error; even though it’s apparent to us, it would require additional analysis steps for the compiler. It’s not as smart as the author pretends (or believes) it is.
I’ve said it before, but here we go again: compilers make use of the presence of UB to remove code because compilers are expected to optimise, optimisation is hard, and removing code is an easy win that’s sound in the absence of UB. It’s not about breaking the code; the code was broken to begin with. It might be reasonable to debate the precise allowances that the language does or should give to compilers in these situations, but there’s no excuse for attributing to compilers the kind of malice that this writing effectively does.
Yes, because it does precisely only the amount of analysis required for translation itself and performance optimization. It could do more analysis in order to warn the user of a correctness problem in their code, but this is not prioritized.
That’s a choice in the design/behaviour of the compiler, though, not the language; anyway the diagnosis is always going to require further analysis than the optimisation, so it’s not a question of prioritisation even on the part of the compiler.
Right, the language just chose to allow a lot of freedom in this on part of the compiler, which the compilers all chose to spend almost exclusively on performance. But the design of the language is driven for the most part by compiler vendors, who know exactly which carveouts they require for their performance goals. So the distinction between language design and compiler design is quite blurry now.
Another recent example is Nvidia deciding that no, there is no guarantee that all threads eventually get scheduled, changing the entire deal around forward progress guarantees and throwing around their weight in the committee to make them declare in C++17 that your spinlock-based mutex implementation was actually Wrong All Along. Because this was the only way to keep their performance while being able to declare CUDA C++ “fully C++17 compliant”.
Or even more recently, when the major vendors discovered that their compilation scheme of sequentially consistent atomics for POWER CPUs was not compliant with the rules from the standard. They did not fix their compilers to be compliant, because that would’ve cost them performance. Instead they worked in the committee to change the memory model, weaking its guarantees and breaking a bunch of previously correct code.
There are no separate language design and compiler implementation processes. The vendors do what they must to achieve their performance goals, then the standard finds a way to make that legal. The overall system, whether by design or by happenstance, achieves performance at the cost of most other aspects.
That’s not really true; witness the extensive sanitizers available with current GCC/LLVM, the options such as “-ftrapv”, and a plethora of other options specifically chosen to inhibit optimisation and/or improve diagnostics in the interest of program correctness or at least in restraining the effects of UB. If anything I’m seeing more and more effort being put into the latter and less and less on attempts to improve performance, in compilers, but even in the language spec: in C++ integers are now guaranteed to be 2’s complement, for example (even if overflow is still UB, this is a limitation on the leeway that compilers have, not an increase in it); the “unreachable” annotation introduced in C23 is specifically designed to allow compilers to be able to better distinguish between unintended and intended UB.
As to the other claims you’ve made, I can’t say I know anything about them (this is the first I’ve heard of either case), but they sound more like “prioritising ability of vendors to claim standards compliance” than they do specifically about prioritising optimisation per se. Granted, this is arguably a finer line. In any case, going back to my original point, compilers are_n’t_ (edit) being malevolent when they employ these optimisations; they’re not knowingly making programs misbehave, as the article suggested.
This, this, so much this.
It’s always slightly surreal to me when people complain that their C compiler is generating buggy programs when they are compiling buggy C and passing in
-O7 -fno-crapv --dont-help-me-i-am-an-expert.Your compiler will prioritize performance over helping you if that is what you tell it to do.
I hear these arguments ad nauseam and they invariably arise from user error. That plus the multiple technical errors in the submission make me reluctant to even engage with the point it wants to make.
The C++ committee could take inspiration from Rust: An alternative to defining signed overflow is to define functions for doing arithmetic with wraparound.
Ditto for unsigned, as it expresses intent. Situation: Clang’s UB sanitizer actually complains about unsigned overflow. Which I’m sure catches bugs, but is really an incorrect thing to do, and worse, removes the only way to do intentional wraparound portably – you have to use compiler specific builtins now.
In practice in all the c++ code I’ve worked on, unsigned over / underflow has always, always been a bug outside of hash functions & similar, and this specific sanitization (-fsanitize=integer) has saved my ass a lot of times. It’s very uncommon to want N-1 to be a few billions when N=0 and causes a lot of issues down the line.
They’ve made dereference of
std::optionalunchecked and UB when it’s empty.In C++ there’s no will to change the priorities from safety third.
This is, unfortunately, largely unavoidable, because C++ doesn’t have anything like flow-sensitive typing. Consider the following:
If you had flow-sensitive typing, then you could make this a compile-time error: in the first branch, the type of
someOptionalis a valid optional, in the second branch it is an optional in an known state, and the dereference operator is defined only on the former. Within the C++ type system, you can’t express that.The other choice is to have the dereference operator (and
value()method) throw an exception or abort if called with an invalid object. This can hurt performance on correct code, because you are relying on the compiler being able to inline the dereference method and CSE the check (which may or may not actually be possible depending on other aliasing properties of the code). Note that UB doesn’t mean ‘must explode’, a standard library implementation is free to insert a check and a branch toabort()here.In Rust, the type system gives enough flow sensitivity (modulo some soundness bugs that were still open a couple of years ago, not sure if they’re fixed now) that you can make this form a compile failure and it provides enough aliasing guarantees that, even if you made it a dynamic check, the optimiser has enough information to know that it’s safe to elide the second check (this is one of the reasons why you can’t circumvent the borrow checker in unsafe code: it will make these optimisations unsound and introduce incorrect behaviour in distant bits of code).
In C++, the work-around is to use
value_oror the monadic operations introduced in C++23. The latter tend to perform well because the compiler’s inlining heuristic will always inline a lambda if it is called in a single place (it is never better to outline code that is used only once, at least until code generation where you may want to move cold paths to a different page). I generally consider using the dereference operator orvalue()to be a sign that the surrounding code needs careful auditing in code review.Well yeah, the exception way would have had a performance impact. So they decided not to pursue it, because performance is the top priority in their language design process. Then comes palatability for compiler implementers and then maybe it’s safety third.
And they make even more bewildering decisions: When they added
std::span, they not only made the regular index operator unchecked on pain of UB, they discussed and then dismissed the idea of also offering a checked.at()method, even though it would have been easy to add and consistent with other types likestd::vectororstd::map. They’re not even giving users the choice between performance or safety. It’s full YOLO or nothing.Isn’t there something that could approximate Rusty
if let Some?The problem with that approach is that it relies on implicit conversion to Boolean. A lot of C++ types (including all primitive types) support Boolean decay and so you can’t just use that directly, because a present value may decay to false. The optional would have to return something that was, effectively, the optional itself: something that wraps the target and a validity thing and uses the validity bit for Boolean conversions and uses the dereference operator to return the value. And now you’re back where you started because it is UB to call the dereference operator on the new thing and there’s nothing other than programming style to ensure that you use it only in code of this structure where you check it first. Worse, this thing now takes a reference to the underlying optional and so makes it easier to introduce memory safety bugs by having it outlive the optional.
This kind of structure is precisely what the dereference operator on optional was designed to support:
Your listing tools can (and do) check that you only use the dereference operator in places where it’s dominated by the check, but you can’t express that in the C++ type system.
An ‘optional’ is a special case of a sum type (optional a is a+unit), which can in general be realised in a language like c++ by church-encoding them; I’m assuming this is basically what the ‘monadic operations’ do.
It doesn’t unless you tell it to. The OP in that question had enabled more than just the default “undefined” sanitizer, and it was another sanitizer option that was reporting the unsigned overflow. From the accepted answer:
i.e. “-fsanitize=undefined” won’t report unsigned overflow.
No, I definitely hate JIRA for catering to and enabling exactly these kinds of workflows.
… and also for being slow.
… and buggy. Last time I had to use Jira there were half a dozen bugs that really got in the way. Like the formatting language being completely different depending on the screen at which you started editing a ticket, and the automatic conversion from one formatting language to the other being broken so that occasionally, even if you did everything right, sometimes your ticket would end up with piles of doubly escaped garbage instead of formatting. This wasn’t an extension, this was core Jira. Though I suspect that particular issue is fixed now.
Are you on a cloud instance or self-hosted server? I’ve seen both be slow, but self-hosted is usually the worst IMO (under-specced or poorly-configured hardware, I would guess).
Yeah, I’ve experienced slowness with both. Even with self-hosted and throwing oversized hardware at it, it still tends to be quite slow (albeit a little faster than cloud hosted).
Do you also hate the processors that run the instructions to make it possible?
If the feature set of the processor was driven by the sales team trying to make every last sale and meet every requirement no matter how weird, and the engineering team didn’t have a say in how it was designed, yeah I probably would!
I mean, if you put it like that, yeah I do kinda hate modern x86_64 CPUs for those same reasons.
They’ve been trying, for sales reasons, to meet the increasingly ridiculous requirement “more single-thread performance for the same old instructions”. And this has driven them to make increasingly dangerous engineering decisions, resulting in the slew of CPU vulnerabilities we’ve seen, along with mitigations for them that undo most of the performance wins.
I’d say that ties into my reasons for disliking it too. I think many of the “features” added were just to get more sales, no matter how it crippled other things that were working just fine. Meanwhile, highly requested features go unimplemented for years because Atlassian doesn’t think it’ll make them more money.
It’s pretty hard for IPv6 to succeed on a personal level when Fucking Verizon doesn’t support it for home users. They were claiming they would deploy it to all their customers in like… 2015 or something, and then just shrugged and never mentioned it again. AFAICT Comcast is similar. So something absurd like 70% of the US population just aren’t gonna have an IPv6 address.
Yes I can use a HE tunnel like it’s fucking 2003 again but all in all I’m pretty sick of it.
That said, it’s pretty shameful that github and datadog don’t even support it. Every VPS I’ve paid for since like 2010 has come with IPv6 addresses by default. …that said, now that I look at it, Sourcehut doesn’t have an IPv6 address either. Maybe time to consider Codeberg, hm?
EC2 VMs don’t get public IPv6 by default. And thus, a lot of stuff running on AWS just never gets around to add IPv6. Even Twitch, now owned by Amazon, still doesn’t have it.
Comcast sucks, but they have had IPv6 support across their network since at least 2020.
That’s interesting. I’m both a Verizon wireless and AT&T business customer, and Verizon is the only provider that doles out both ipv6 and ipv4 addresses to my LTE router upon connection. AT&T business never provides ipv6.
So it’s clearly not a technical problem for Verizon. They’re choosing not to give ipv6 addresses to home users.
It’s a slow rollout. My understanding is that Verizon is going region-by-region to reduce the support burden. My residential Verizon Fios connection near NYC magically started receiving an IPv6 address about a year ago, and it was seamless enough that I didn’t even realize for a while that most of my traffic was going over IPv6. It has been smooth sailing ever since.
FWIW - I’m on Verizon Fios in the Boston area, and it appears that I have IPv6 running.
I wonder if maybe all of these evil vendors aren’t throwing their customers under the IPv6 bus because it doesn’t matter to the vast majority of the customers. Because 98% of their customer base don’t know or care what an IP address is, and as long as TIkTok streams and games can be played, why would they? There just isn’t a value proposition for the average user that demands IPv6.
You’d think that it makes life easier for them as network admins, though.
If supporting both is more expensive than supporting one, and your choice is between v4 and v6 OR v4, … how would it be easier? I’m not savvy to why v6 has taken so long, but it certainly seems like a network effects kind of thing.
It doesn’t. Which is why we network admins haven’t jumped all over v6. We’re lazy…if it was easier v6 would be the de facto everywhere, and would have been for years.
“x is a disaster” after very little experience. That’s an orange-site sentiment.
FWIW I’ve used IPv6 in production for many years and feel positive. My comparable setup is: The iron has an IPv4 address and provides outgoing IPv4 via NAT to all virtual hosts, spinning up a new virtual host assigns an IPv6 address but no IPv4 so there’s no incoming IPv4, and… that’s all. I don’t even try to avoid using outgoing IPv4, and I don’t need to ever think about whether I have addresses on hand.
Sometimes people need to access one of those servers and can’t. That happened to me only this summer. They accept that it’s their problem, not mine (and while I feel that this sentence should end with a smiley I’m not sure which).
Not in this case. It is a disaster if all you wanted was a normal, working host, and you got a ipv6-only thingy, that can’t even fetch things from github by default. That’s a major no-go if you expect this to work like any other machine. Whether it really makes ipv6 a “disaster”, or more a statement for how slow-moving corporate networks are is up to the reader. IPV6 certainly “just works” if you have a dual stack. But for now only then.
The point is ipv6 is not the “disaster”. The fact that so few network-oriented businesses are willing to show technical leadership is.
It felt pretty clear to me that “ipv6 is a disaster” isn’t referring to technical flaws; the implication is “ipv6 adoption is a disaster”.
The post I’m actually interested in reading is about why even big-names like github haven’t bothered to fix it.
Just a speculation, but if you have a working CDN with load balancing, blacklists and rate limiting, it can be very annoying to adopt ipv6. Even more when you suddenly not only need ipv6 but rather /64 matching (for blacklisting or rate limiting). It’s a giant change in your infrastructure, and we all know what brought
AWSfacebook down: a broken core router configuration.Another great example of people leaving lengthy comments about what they think the headline of the article implies without engaging in any part of the contents, probably not even reading past the headline
Counter-argument:
Resources are limited and time is finite. Rolling out IPv6 to satisfy the aesthetics and “best practices” of people who are unlikely to have a direct stake of real size is a bad idea.
As much as everybody complains about it, the current system works and is a hell of a lot simpler than IPv6. Maybe, if a technology has taken more than two decades (27 years if you ask Wikipedia) to get adopted, there’s some flaw in the technology.
If you are suggesting that Github or Twitch or others lack technical leadership, I think the empirical data disagrees with your proposed metric.
Is it? From the blog post there’s a picture of the IPv6 header, and it looked like it has much fewer fields. At a glance, the core of it look simpler. What are the more complicated parts?
IPv6 is quite complex. Much of the complexity is in the required supporting protocols: NDP, MLD, ICMPv6, and so on.
Skimming the table of contents of RFC 4861 may give some sense of this.
Edit: And about headers specifically - the post didn’t mention IPv6 extension headers.
So what about an ipv8 that doubles ipv4 space, and reserves the last half for ipv4 compatibility?
One of the transition mechanisms (6to4?) did something like this. It mapped the entire IPv4 address space into the v6 address space. This came with a lot of problems. First, you still need to handle different packet formats, so the boundary involves some translation. Then you have protocols that embed an address in the message, which doesn’t get translated. Then you get the problem that things on the IPv4 side look like they’re on the v6 side, but can’t connect to most hosts there.
The entirety of IPv4 is mapped into IPv6 not just once, but at least five times, for different transition mechanisms: https://en.m.wikipedia.org/wiki/IPv6_address#Transition_from_IPv4
These require your connection hit a bridge along the way, right? Instead of being able to be switched and routed in the same hardware.
Yes, though the bridge can be on one of the endpoints or on an ISP’s network. I believe the idea for several of these was that ISPs offering both IPv4 and v6 would bridge the networks so that their v4 addresses were all visible and servers could move to v6 and still support legacy v4-only clients.
If what you wanted was a normal, working foo, and you got a something the designers of foo considered weird, and it was a pain to use, would you consider that a failure of foo, or would you consider it more of a PEBKAC case?
Dualstack was considered the migration when the various IPv6 designs were compared and the one we have won over the wothers.
If you have dual-stack, you might as well just run ipv4. Because cost-wise, you already needed the ipv4. Access-wise you only need one (ipv4, the v6 people will already need something to get v4 access, but not vice versa). Complexity wise, you’re better off with only one.
So what exactly do I gain by having ipv6 compatibility, apart from internet karma ? Everyone else (relevant) seems to get away not caring about ipv6.
I run dual-stack on my hosts, but I also don’t pay for ipv4 on top, and I only do so since I’m running nftables and have not as much complexity. I also can’t even use the potential of my /64 (for containers etc). Because then I loose ipv4 compatibility. So it’s basically the same as ipv4, except I need to address/debug everything twice.
If cost is what you care about: Having outboung IPv4 costs my nothing, having inbound IPv4 would incur a per-VM cost.
it’s always about inbound - and costs for complexity
The cost of complexity is actually why I started doing that. Getting a public IPv4 address onsite required meetings and negotiation and justification, costly complex stuff, so I wrote a script that set up a new VM with a publicly reachable IPv6 address, a DNS entry and NAT for outbound IPv4.
Sounds good, but how do you handle accessing stuff purely from IPv4 ? Because that’s my main problem: making it accessible for both - and thus loosing the freedom of my /64.
I don’t bother with that.
It can be a problem. Earlier this summer I held a workshop where I said something like “I made an example site showing [blah blah], it’s at [link]” and some of the participants couldn’t access it because they had no IPv6. But mostly it’s okay. Either because the people who are supposed to use it have IPv6, or because they don’t but perceive it as their problem rather than mine.
I can’t imagine any metric for an internet protocol by which IPv6 would be anything other than a complete disaster.
I mean I guess the packet header is mostly well designed?
This looks interesting, but it’s lacking a bunch of information, so I can’t really make sense of it. Most importantly: What is this intended to do? What are the properties that it is designed to achieve?
The only thing resembling a “design goal” that I can see would be
but it’s unclear to me how the proposed solution
would ensure that. All it ensures is that routers can reach at least two other routers and themselves. It is easy to come up with a fork-routed network that satisfies the reline rule but where not all routers can reach all routers (i.e. it is not strongly connected).
For example, you can take any two fork-routed networks and lay them next to each other. The new network has relines, but is not strongly connected. Or you can take any directed graph with a vertex outdegree of 3 everywhere that is not strongly connected and replace each vertex with 3 fork routers that form a reline loop, turning each of the outgoing edges into a 0-route for one of these fork routers.
It feels like maybe there are further assumptions about the network being made, but these are left unstated here.
oh man. maybe im too stupid to know what details to write lol
bro i rember spending hours thinkin about it til i was confident it would work..
like think about it: if a router can access its previous one through its reline, then it can access its previous previous one cuz its previous has a reline to its previous previous one. and if you can access the previous previous one that means you can access one of the routes of the previous previous one, and so on. so u can literally go anywhere
Wait, the reline loops after 3 routes, doesn’t it? So you’ll just be going in circles.
not necessarily. you can choose the other route of the router you’re pointing to with your reline. and you can also choose not to go through your reline. just look at the diagram: not everything is a loop
The
undokeyword reverts the last mutation made by the current thread - mutations made viaundoitself don’t count for this, so you can do it several times to travel back in time.Related: the
@operator when used with a variable lets you specify a particular index in the history of that variable. Positive numbers as the second operand mean steps backwards in that variable’s history.Thanks for bringing this up! It’s honestly my favorite part of LOBSTERS-LANG, and doesn’t get used enough. It’s super flexible too. If you use a negative number it’ll use linear regression to look into it’s future.
Good article, but I laughed at this line
as on my slack servers, the most important feature is allowing custom emoji. On one server, we’re up to roughly 4k, and show no signs of stopping.
The last two businesses I have left, I’ve exported the custom emojis from slack (search GitHub for various scripts to achieve this easily) and then added a bunch to new $work’s slack upon joining. Not all of them of course, the in-jokes only work in the original place.
Maybe you could send emoji as sixel or kitty terminal graphics protocol pics.
Is that functionally different from allowing inline images? Like can any user send any image as an “emoji” or is there some process by which emojis must be approved before they can be used?
Somewhat, yeah. Having shared, readily available reaction images allows for creating a community language, same as in-jokes or oft-repeated quotes. Using
:sickos-yes:has a different flavor than:100:which has a different flavor than:clap-cowboy:, even tho they all mean “I agree with the statement”. But those meanings come from usage and situation, so if they have to be manually curated by each user, they won’t be used nearly as much.Oh, do you mean “how are they added to slack?” I believe it’s configurable per server, but the servers I’m in allow any user to add a new emoji or create an alias for an existing emoji, and it supports various static and animated image formats.
Damn, :sickos-yes: is a must have, I got to add this to the company chat.
You say UNIX but then depend on many specifics of GNU/Linux ;) I haven’t actually run the code, but what I’ve spotted so far:
/usr/bin/. For example, on FreeBSD,echoanddateare in/bin, andbashin/usr/local/bindatedoesn’t need to support--iso-8601, some systems have-Ibut a portable way would be to use a format string:date +"%Y-%m-%dT%H:%M:%S %Z"(https://pubs.opengroup.org/onlinepubs/9699919799/utilities/date.html)iddoesn’t need to support--user --name --real, a portable way would beid -unr(https://pubs.opengroup.org/onlinepubs/9699919799/utilities/id.html)Fun idea though, I wouldn’t complain about replacing our company’s Slack with it. ;)
Thank you, that is really helpful.
I’ll change the id and date invocations. As for the full path, there is a security reason behind them: if the user shadows them, she can run stuff under the effective privileged ownership of suc. By using the full path, I’m making sure there are no $PATH shenanigans going on. I should update the article to reflect that.
This is a moot point on Guix, as the full path to the exact binary I need (e.g. /gnu/store/SOMEHASH-coreutils/bin/echo) is put into suc.sh when the package is installed (see https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/packages/plan9.scm#L99).
Maybe I could use a configure-like step for other *nixes that would fix those paths at install time.
What do you mean by this? You can’t/shouldn’t setuid a bash script, right?
Yes, or rather, you can setuid it but Linux ignores it. Part of the reason is exactly this sort of $PATH problem.
sucworks around this by also shipping a C wrapper program that you can setuid and that calls the shell script.Also, that program appears to have a bug, it fails to null-pointer terminate the argv array passed to execv.
I have not tested this, but you could probably set PATH at the top of the script, such that any changes made by the untrusted user are overwritten.
Meh if you actually need portability, sure
But I think these kinds of nitpicks are holding back shell, e.g. most people don’t want to read
id -unrAgree with just using
date, and not/usr/bin/dateThe real answer (or at least the future facing one) should be that all Unixes support containers (OS virtualization), so you can build GNU tools or whatever you want inside them, without messing up the rest of the system
FreeBSD, Solaris, and Linux do, and probably a few others. I’d say that any others should get with the times
chroot might be technically sufficient for this case, but we also need common tools. From experience, it’s possible to do it with plain shell scripts, though it’s a lot of work, and some higher level abstractions are useful
Well, that’s rather dismissive, it’s like saying “meh, if you actually need to support other people than me…” :)
The article claimed to be a demonstration of modern UNIX and I only mentioned things that were genuinely broken on 2 of my UNIX systems (macOS & FreeBSD)
Also note that this isn’t portable either on GNU/Linux distributions, e.g. NixOS. I would say relying on
$PATHis the best bet, and it’s the user’s job to be wary of the what’s going on in$PATH.The danger is not to the user ! The user is the danger !
By playing with her $PATH, a user can execute arbitrary code owned by suc, which would let her e.g. edit the chat log (which is supposed to be append-only).
As for nix, one can patch the hardcoded path on install, as I did for guix: https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/packages/plan9.scm#L99
This is a general theme I’ve noticed with Linux: All the stuff I want is basically there, it’s just way more complicated and annoying than I’d like it to be.
Another example: OpenBSD gets praised a lot (rightfully imo) for its “pledge” system. Linux has something to do the same job, seccomp-bpf, but it’s so hard to actually set up that its usage is mostly limited to a very small set of core services.
There is most of a decent capability-based OS buried somewhere in Linux, you can send around file descriptors for almost anything (processes, sockets, memory, even namespaces themselves!) and you have a lot of tools to eliminate ambient authority (which is of course still there by default).
edit: and of course Linux also has a subsystem literally named “capabilities” that has nothing to do with any of this and is just generally awful.
pledge for linux: https://justine.lol/pledge/
It’s close, but it doesn’t quite do what pledge does. For example, you can’t pledge the shell to avoid doing any network I/O, while allowing programs spawned by the shell to still work.
It feels right that permissions obey a hierarchy, but it turns out to be very useful to break that hierarchy. It’d be even better to do it deeper than just a parent/child relationship, but that gets really complicated, so that one layer is a reasonable tradeoff in practice.
What’s the point of not letting the shell directly access the internet, while still letting it spawn other processes that can access the internet? Or am I misunderstanding?
Imagine that you’ve managed to find some bug in your shell; you want to pivot it to a complete expoit, but you don’t have any syscalls that you can use. If you’re careful about locking things down, you may have unveiled only a few binaries in the path, so you can’t even run arbitrary programs. Now, not only do you have to find an exploit in the shell, you have to pivot it to execute a program that will do what you want, and maybe garble its arguments in a way that it will give you more access.
In a privsep daemon, you may also have a spawner process that primarily has the job of creating processes that do the work; that process needs very little permissions to work, but the workers may need to do more.
It’s not a panacea, but it makes your attacker’s life a lot harder, and makes it possible to ratchet down permissions far more tightly.
Anyone know if any of those “archive software for a million years and have it still usable for whatever species digs it up” efforts have ever considered Wireworld as an aggressively minimal CPU implementation? It might be a bit too minimal, but it’s probably the only thing Turing complete thing I’ve seen simpler than Conway’s Life.
Hmm, I wonder if one could write a compiler to compile arbitrary programs to Wireworld “hardware”…
I was at a computing history museum at the same venue as the 2023 vintage computer fest and showed the people running the museum wireworld, and they asked the same question. :) They also said gosh, maybe we should be using Wireworld as a starting point to teach the basis of composable computing components to people with no prior experience. Which was pretty neat to hear people whose efforts are to teach people exactly those topics get excited about that same idea!
Is it really simpler than game of life? Both are istropic 2D cellular automata, but wireworld has 4 states whereas life has only 2.
If you’re going for absolute minimalism, maybe Rule 110 is an idea. It’s a 1D, 2-state cellular automaton and makes some pretty pictures.
I’m not sure minimalism in the rule set is the right thing to optimize for. The proof that rule 110 is Turing complete is very involved – It’s sort of as if complexity was removed from the automata and moved into the “interpretation” of what a particular state means.
Wireworld seems to be in a nice space where the automata is both simple and “obviously” Turing complete.
Idk about “obviously”, but certainly “more trivially” in that the proof is pretty small and/or slots into other common proofs. More or less just make a Wireworld NAND gate and say “ok you see where I’m going with this”.
Is this just wrapping ancient POSIX APIs, generally regarded as awful, in WASM? If so, that’s kind of sad, i.e. we’ll have to live with these APIs for another few decades.
That was my reaction as well. With WASM, we have a rare opportunity to build a platform that breaks legacy and provides clean abstractions that weren’t designed for single-core minicomputers, so let’s take advantage of this by directly porting the worst ideas from that world. Oh, and let’s make absolutely share that we don’t think about security at all when we do it. No thanks.
Yes, this is the “Worse is Better” to the Bytecode Alliance’s “The Right Thing”.
They are faster to the market, but also I feel like everybody whose problem looks like “I want to run legacy Linux applications in a lightweight sandbox without source modifications” is already well-served by the container ecosystem.
Yeah it’s a misguided attempt to make containers from scratch.
This is not a Bytecode alliance protocol, so it’s just low quality fragmentation on their part. No one else is going to support it. Also, that company is generally malicious.
I disagree. It’s exactly the kind of poorly designed nonsense that solves immediate problems (how do I run this program that assumes a Linux host in a WASM environment without any porting) and creates massive amounts of technical debt along the way that usually succeeds in the market.
As a previous insider, I assure you companies are not very interested in running stuff in wasm. The only interest that existed (back then at least) was from the cryptocurrency hype which has since died. This stuff adds so much tech complexity which no sane person would choose over running containers.
Also, this implementation in particular is written by a guy with no programming/engineering background (look him up) so this technical debt you say may pile up too fast. Company’s a clown show.
I am especially saddened by inclusion of proc_fork. Have we learned nothing?