Threads for hauleth

1. 6

   capitano 8 days ago | link | on: ko: Easy Go Containers

   Why do you need containers for running a single binary? I suppose it would make sense if everything else in the stack is running in containers and you want to create isolated networks, but part from that, idk. Perhaps someone could enlighten me?

   1. 5

      david_chisnall 8 days ago | link

      I can think of two reasons. The first, as you say, is that you have some infrastructure for deploying containers, making things look like containers is useful. The second is that there’s often more to a system than just the program. You probably have configuration and data. The configuration is easy to add as a new container layer. The data may be stored in a separate service (e.g. accessed via the network) but if it’s filesystem based the container orchestration tools provide a good way of attaching filesystem trees to containers.

      There’s also a bad reason: security. People often conflate containers (a distribution and orchestration model) with a specific isolation mechanism. On Linux this is a mess of cgroups, namespaces, and seccomp-bpf that keeps having security holes. On most cloud platforms, it’s VM isolation because the shared-kernel model doesn’t give sufficiently strong guarantee for tenant isolation.

      There’s also a silly but real argument: cost. A lot of cloud providers have container billing that does finer accounting of memory and CPU usage than their IaaS (VM) services and so running a program on their container infrastructure is cheaper than running it on a rented VM.

      1. 4

         insanitybit 8 days ago | link

         Security is a really good reason. The “security holes” you’re talking about are kernel exploits - not enough for tenancy, certainly, but definitely nice given that putting something in a container is virtually free.

         That said, it’s worth noting that this is a build tool.

         1. 3

            hauleth 8 days ago | link

            For “security” using containers is not needed as you can have that on Linux without all that fuss. Just deploy binary and systemd’s unit for your service (which in case of single binary can be even within your binary with just single command away) and you are good to go. Much less stuff needed, and also this can give you some additional features that not all container runtimes provide.

            1. 1

               insanitybit 8 days ago | link

               If you want to use systemd, go for it. Obviously there’s nothing you can do with a container that can’t be done natively - but if you’re already using containers there’s some good stuff that you get “for free”.
            2. 1

               weberc2 8 days ago | link

               Putting something in a container and checking the ‘isolation’ box on a cloud provider (gvisor on GCP and IIRC firecracker on AWS) is a lot easier than managing Linux hosts and configuring all of the security/isolation stuff yourself.
         2. 2

            david_chisnall 8 days ago | link

            Security is a really good reason. The “security holes” you’re talking about are kernel exploits

            They are sometimes kernel exploits, they are sometimes misconfigurations in policy. For example, there was one that I recall that was caused by the fact that the Docker seccomp-bpf policy didn’t default to deny and so a kernel upgrade added a system call that allowed a container escape. Sometimes they’re exploits but, importantly, they’re exploits relative to a threat model that 99% of the kernel code was never written to consider. The Linux kernel still doesn’t really have a notion of a jailed process (unlike the FreeBSD or Solaris kernels) and so you are reliant on the kernel enforcing isolation built out of a set of distinct subsystems that were not designed together and where any kernel subsystem may forget a check.

            but definitely nice given that putting something in a container is virtually free.

            You might want to run some benchmarks before deciding that it’s free. Depending on the workload, it can be as much as a 20% perf hit to run via runc Linux versus running in the root namespace with no seccomp-bpf policy or custom cgroup. For others, the overhead is close to zero. The overhead can be even worse depending on the filesystem layering mechanism that your container platform is using (some of the union-based drivers can have a huge impact on anything with a moderately large disk I/O component).

            1. 3

               insanitybit 8 days ago | link

               they are sometimes misconfigurations in policy.

               Sure. These are increasingly rare though.

               Sometimes they’re exploits but, importantly, they’re exploits relative to a threat model that 99% of the kernel code was never written to consider

               I don’t really agree. The kernel has long had a security model of “unprivileged users should not be able to escalate privileges”. It has not had “privileged users should not be able to escalate to kernel” until much more recently.

               I don’t know what notion of jailed you want but namespaces certainly seem to fit the bill. They’re a security boundary from the kernel that applies to a namespaced process.

               Depending on the workload, it can be as much as a 20% perf hit to run via runc Linux versus running in the root namespace with no seccomp-bpf policy or custom cgroup.

               Source?

               I think the point here is that, yes, the Linux kernel is a security trashfire, but I think you are underestimating the effort to escape a sandbox. Building a reliable kernel exploit, even for an nday, can be weeks or months of work.
      2. 2

         reezer edited 8 days ago | link

         For a lot of stuff you could use a wide range of tools. For example Nomad’s (isolated) exec driver.

         Regarding security. Running Go binaries with pledge and unveil is really easy and great.

         Usually run it with this simple rc-script then, just replacing my_binary:

         #!/bin/ksh
         
         daemon="/usr/local/bin/my_binary"
         
         . /etc/rc.d/rc.subr
         
         rc_start() {
                 ${rcexec} "${daemon} ${daemon_flags} 2>&1 | logger -t my_binary &"
         }
         
         rc_cmd $1
         

         There’s also a silly but real argument: cost. A lot of cloud providers have container billing that does finer accounting of memory and CPU usage than their IaaS (VM) services and so running a program on their container infrastructure is cheaper than running it on a rented VM.

         This is not always true though, because often these more “finer accounting” solutions have a higher price on their own, so it really depends on utilization.
   2. 3

      caboteria 8 days ago | link

      Likely it’s for people who are running Kubernetes so everything has to be a container.
   3. 3

      mperham 8 days ago | link

      Binaries are not a deployable unit. Containers are.

      1. 6

         weberc2 8 days ago | link

         I sort of see your point, but I’m inclined to argue the contrary. Statically linked binaries essentially are a deployable unit. Maybe you’d argue that containers can bundle configuration, but so can binaries. Maybe you’d make some distinction about “not needing to recompile the binary to change configuration” but you still need to rebuild the container which is the more expensive part by far (for a Go app, anyway), even with a decent hit rate on the build cache–there’s no fundamental difference between compiling a binary and running a Docker build except that the latter is wayyyyy more complex and expensive (in most cases, you need a daemon running installed rather than just a compiler/toolchain).

         Containers are great for apps that can’t easily be distributed as a static binary (particularly when it would be very large, since container layers are individually addressable/cacheable) or for cases where you’re deploying a static binary in a containerized environment (e.g., your org uses k8s), but a single binary absolutely is a unit of deployment.

         1. 0

            mperham 7 days ago | link

            It isn’t though. Show me the cloud service that allows me to deploy a single Linux binary.

            1. 2

               weberc2 7 days ago | link

               What cloud providers support isn’t useful for answering the question. One can easily imagine a cloud service that takes a static elf file and drops it into a firecracker VM—the reason this doesn’t exist (to my knowledge) is that their customers are typically deploying apps in languages that don’t have a good story for static ELF binaries (and the ones that are can just wrap their binaries in a container), not because ELF isn’t a deployment format.
      2. 2

         nathants 7 days ago | link

         updating code in a lambda with a zipped binary is significantly faster than with a binary in a container.
   4. 1

      southclaws 8 days ago | link

      Typically containers run one app each anyway, but in my experience it’s just generally nice to have one unified format to build, distribute and run server side software. I can build an image on my windows computer, throw it onto my mac and it works with zero changes then I can push that same image up to our hosting provider and it runs there too, and we can switch hosting provider (which I have done a few times) with minimal effort. Sure, under the hood you’ve got EXE, ELF, DMG, PKG, etc on all the various operating systems but when it comes to getting production code live, containers really do make life easier!
   5. 1

      xyproto 8 days ago | link

      • Containers have become like universal server executables.
      • It requires less work to run a container as a Google Cloud Run instance than an executable, source tarball or a repository.
      1. 2

         weberc2 8 days ago | link

         Agreed, but I still think it would be cool if we orchestrators had decent support for ELF files. I’m pretty tired of creating all of these single-binary containers just because there’s no way to run binaries without taking on the full burden of managing one’s own hosts.

         1. 2

            dimitarvp 7 days ago | link

            That’s a sensible requirement. How hard could it be for the hosting providers to detect an EXE / ELF file and just wrap it inside a stereotypical container? I’d think it’s something close to a five-line shell script.

1. 2

   hauleth 9 days ago | link | on: Programming with one or fewer hands

   Try modal editing to reduce the need to move hand to mouse. Maybe try also something like roller mouse, trackball, or keyboard with track point. Everything to reduce the horizontal movements of the hand.

   1. 1

      bios-marcel 9 days ago | link

      roller mouse looks interesting. But I guess its basically just a weird external touchpad?

      Maybe also a nice excuse to try out something like the https://gaming.tobii.com/product/eye-tracker-5/

      I remember seeing there was an open source linux driver for either this one or an older version.

1. 3

   hauleth 9 days ago | link | on: On the RustConf keynote

   Honestly, after all that fuss again I do not want to hear apologies once again, what I want to see is heads on the pikes for people who are responsible for that stuff. From all what I have seen so far, there was one individual or small group of such that imposed themselves as a BDFLs without involving others in the Foundation. And sweeping it under the rug will not kill the smell of the crap, that need to be thrown out, and in this case, as loudly as possible to show people “hey, we had some bad actors, but we actively fight against them and we prosecute them with all possible power”. Otherwise it will be hard to gain any trust back.

1. 3

   tux0r 9 days ago | link | on: On the RustConf keynote

   Although I appreciate that the team responsible publicly admits that they made a huge mistake, I wonder when and why all reputable open source projects have shifted their focus from actual technical merits to “conferences” and codes of conduct. I am not surprised that large software corporations still lead the market. They don’t have conferences and codes of conduct, they actually write some code.

   1. 4

      hauleth 9 days ago | link

      They don’t have conferences and codes of conduct

      Of course they have, it is called Human Resources division.

1. 1

   hauleth 17 days ago | link | on: How Much Memory Do You Need to Run 1 Million Concurrent Tasks?

   Pasting my comment that was update response to this comment

   -module(ccc).
   
   -export([worker/0]).
   -export([start/1]).
   
   worker() ->
       timer:sleep(10000),
       parent ! done.
   
   spawn_many(0) -> done;
   spawn_many(N) ->
       spawn(?MODULE, worker, []),
       spawn_many(N - 1).
   
   start(Args) ->
       {Count, []} = string:to_integer(hd(Args)),
       register(parent, self()),
       spawn_many(Count),
       await(Count),
       io:format("Done"),
       halt().
   
   await(0) -> done;
   await(Count) ->
       receive
           _ -> await(Count-1)
       end.
   

   some updates to the codebase and bunch of flags:

   erl -run ccc start 1000000 -noshell -pa -nonstick -noinput +hms 1 +a 16 +d +L -start_epmd false +ec +P 2000000
   

   Results in:

   1006190592 peak memory footprint
   

   on my machine. So it is a little bit less than 1 GiB, which tops the Go code by factor of 2 using the Go code from the example (2660564992, so almost 2.5 GiB) on my machine, and it seems that it would put in on par with Java Virtual Threads.

   I was surprised as well, but all of that with almost negligible performance impact and on my machine (with bunch of other stuff going in the background, so take that with grain of salt) Erlang (OTP 25.3.2 without JIT enabled) code was consistently faster by about 2s than Go code (Go 1.20.4).

   1. 9

      matklad 17 days ago | link

      Changing Go from

              defer wg.Done()
              time.Sleep(10 * time.Second)
      

      to

              time.Sleep(10 * time.Second)
              wg.Done()
      

      Improves Go CPU time by about 2X. Go’s defer is a runtime thing, and would have some impact in a micro bench.

      1. 1

         cmcaine 17 days ago | link

         For anyone else wandering why this is:

         How’s defer pushes the function call onto a stack at runtime and then calls all the functions in the stack at function exit.

         Some other languages (like zig) have a defer statement that is executed at scope exit, and these are easily implemented without runtime support.

         Presumably Go uses a runtime stack because it’s a simple way to properly order defers inside loops and conditional defers.

         1. 3

            masklinn 17 days ago | link

            Presumably Go uses a runtime stack because it’s a simple way to properly order defers inside loops and conditional defers.

            Also because defers act as panic (exception) handlers. That go’s defers are function scoped regularly takes people by surprise, especially because it’s easy to accumulate defers in a loop and get what amounts to a resource leak.
      2. 1

         hauleth 17 days ago | link

         Nice, I knew that something must be off.
   2. 2

      vhf 16 days ago | link

      I translated your code to Elixir, my 3yo macbook reports:

      Physical footprint (peak):  972.3M
      completed in 11.637862s
      

      using

      elixir 1.14.3-otp-25
      erlang 25.2.3
      

      1. 1

         hauleth 16 days ago | link

         I quite intentionally written it in Erlang to omit starting up and launching all the additional applications that Elixir uses (for example logger) to reduce the memory footprint.

1. 6

   Corbin 1 month ago | link | on: In Erlang/OTP 27, +0.0 will no longer be exactly equal to -0.0

   Previously, on Lobsters, we had a closely-related conversation. It seems like Erlang is catching up to the paradigm of two equality operators: one which is precise enough to verify identities, and one which is looser and respects IEEE 754.

   1. 3

      justinpombrio 1 month ago | link

      It looks like I can’t reply to your comment on that earlier post, so let me reply here if you don’t mind.

      I like that design!

      I wonder if you could get away with using two relations like that as a variation on how Rust handles equality. Right now Rust has four traits:

      • PartialEq for equality that is symmetric and transitive but not necessarily reflexive, with operations written ==, !=
      • Eq (extends PartialEq) for things that are PartialEq and also reflexive
      • PartialOrd (extends PartialEq) for partial orders, with operations written <, <=, >, >=, ==, !=. (If you write == for something that implements both PartialEq and PartialOrd, I’m not sure which is called, but they’re required by spec to behave the same.)
      • Ord (extends Eq + PartialOrd) for full orders

      As an example, floats implement PartialOrd, and ints implement Ord.

      However, this gives more buckets that I think tend to show up in practice. The kinds of relations that actually show up, from what I’ve seen, are (i) Ord, (ii) Eq, and (iii) floats which are nonsense. So it seems excessive that you have to write #[derive(PartialEq, Eq, PartialOrd, Ord)] all the time.

      What I’m wondering now is whether in practice you could get away with just two traits:

      • Ord for partial orders, with operations written <, <=, >, >=, <>, <=>. Floats implement this.
      • Eq for symmetric, transitive, reflexive equality, with operations written “==” and “!=”.

      With the requirement that if something implements both Eq and Ord then it’s a full order, and additionally == and != must coincide with <=> and <>, and you get a warning if you use <=> or <> because it would be more clear to use == or !=.

      Though as I’m writing this it’s seeming questionable because it doesn’t cleave things along as clear mathematical boundaries. And the starting point was thinking that if you wrote == you knew you’d be getting reflexive equality and if you wrote <=> you knew you weren’t, but that doesn’t extend to < which could be either a partial or full order.

      1. 2

         hauleth 1 month ago | link

         There are structures that are PartialEq/Eq but aren’t ordered - most obvious example are complex numbers where we can compare them, but not order them.

1. 7

   minus 1 month ago | link | on: 1Password is adding telemetry

   I wonder what they hope to achieve with that. I always do when someone adds telemetry. So far I haven’t seen a convincing answer.

   1. 18

      matklad edited 1 month ago | link

      Stuff I, as a developer, used telemetry for in IntelliJ Rust:

      • Crash reports. They are sort-of-different from the telemetry, but also are the same.
      • Figuring out what to focus on. Eg, if a lot of people use IntelliJ Rust with PyCharm, it makes sense to spend some time integrating into PyCharm-specific project view.
      • Figuring out when to drop support for old platforms. If only 5% of users are on a particularly old IJ, plug-in can stop supporting it.
      • Getting personal moral boost from seeing the number of installs going up
      • Justifying continuing investment into the project by the company

      I didn’t use fine-grained telemetry for specific features, as, at that time, barely anything was working at all.

      1. 1

         reezer 1 month ago | link

         It doesn’t really say they do that. Ending the experience feels different than finding hugs. It could mean both though but without a precise description of how data will be used all bets are off.

         For what makes sense and what people want interacting with your users, etc. tends to give more insights info the whys. With telemetry one ends up having to guess

         You could just count downloads. You could also describe better what your telemetry is on. I think people would be fine if you specified something like well report your version when you log in through our extension.

         And for general stats about how much sense continued investment makes sense wouldn’t downloads and paying users be enough?

         I really want to emphasize here that you could just make a l list of what and why you tell it. You likely have that internally anyways. Why not publish it unless it’s something you don’t want your users to know? Sure it might be a bit of effort but it could almost be seen as marketing/gaining trust.
   2. 14

      hauleth 1 month ago | link

      Automatic error reporting is also telemetry, and for non-developer focused tools, it is immensely useful to be able to record what errors people encounter without their interaction.

      1. 1

         minus 1 month ago | link

         Fair point, though I wouldn’t really consider crash/error reports telemetry. And judging by 1Password’s wording, this isn’t about that anyway.
   3. 6

      driib edited 1 month ago | link

      The first thing that comes to mind is to check which features they could drop without aggravating too many users. Also, you could prio the backlog by feature group as well as to see which new features get attention and which don’t.

      1. 26

         ploum 1 month ago | link

         From what I’ve seen in my professional life (as a product manager, thus making decisions about features), this is telemetry goal but, unless done at a Facebook scale where you truly experiment with a very large user base of “identical” users, it’s bullshit.

         — Following the data, we could remove A.
         — Wait, if you remove A, it breaks B.
         — We could also remove C.
         — No, C is the pet feature from the CEO.
         — So let’s remove D.
         — Ok.
         — …
         (phone rings)
         — It’s our biggest customer. He’s angry we removed D. The CEO is using the feature once a year but it’s important to him.
         — Sigh.…
         — So let’s remove E
         — Hey, no way, I did E, I like it. I’m simply blocking the telemetry.

         I’ve never seen telemetry data being useful. You need to use it in conjunction with user survey and interviews. Oh, guess what? Once you take the time to do proper user interviews, telemetry are mostly useless.

         And the cost associated with telemetry is very high: you need to maintain the telemetry infrastructure. You augment the risk surface (remember when OSX was bugging because it was trying to phone home?). You need to keep this data reliably to avoid any privacy breach. You need to sort this data, to analyze it. The cost is important.

         Which means that when a product add telemetry, there are only two options: either they are doing that “because everybody does it” or they have some way to recoup for the high cost. Which is rarely a good thing for the end user.

         Fortunately, in 1password case, I switched to bitwarden years ago and bitwarden is cheaper, simpler, opensource multiplatform and works even better for me.

         1. 1

            minus 1 month ago | link

            Thank you for this insightful comment.

            Out of curiosity: What kind of development team size, product/code size and customer base size are we talking about here?
      2. 4

         technomancy 1 month ago | link

         without aggravating too many users

         Without aggravating the users who leave telemetry enabled, you mean.
      3. 2

         NoahTheDuke 1 month ago | link

         The first thing that comes to mind is to check which features they could drop without aggravating too many users.

         I would simply not remove features.

         1. 9

            friendlysock 1 month ago | link

            While admirable, that is not ultimately a sustainable policy depending on market factors.
   4. 3

      sky 1 month ago | link

      Collecting user data, including anonymous data collection, is insaney valuable and it will keep getting more valuable in the future. Even if they don’t know as of now what they’ll be able to do with it, they certainly can harvest it as an “investment” of some sorts.

1. 2

   hauleth 1 month ago | link | on: Why I like Clojure as a solo developer

   I have pretty much the same feeling about working with Elixir.

   1. 1

      emoses 1 month ago | link

      I tried to build a small application in Elixir a while ago and found myself really confused about how to set up supervision trees and how to factor my data/genservers in general. It’s an unfamiliar architecture pattern that seems fantastic, but I felt like I really needed more guidance than I could find even reading an intro book or two.

1. 8

   bdesham 1 month ago | link | on: Fleek: A human-friendly Nix wrapper

   Sometimes it feels like Nix was built as an expression language first, with package management features tacked on afterward. (Certainly the “I’m here for the language” contingent of the community is larger, or at least louder, than the “I just want a package manager with a sane user experience” contingent.) It’s refreshing to see this being reimagined, if only by a third party.

   1. 7

      Corbin 1 month ago | link

      Of course, it is the other way around. Quoting p61 of the thesis introducing Nix:

      The Nix expression language is a simple, untyped, purely functional language. It is not a general purpose programming language. Its only purpose is to describe components and compositions.

      The Nix expression language is a bare-bones purpose-built description language for the Nix store. It was always intended to facilitate the construction of something like nixpkgs, and not much else.

      1. 4

         bdesham 1 month ago | link

         Maybe the expression language is a red herring. My real complaint is that there are weird gaps in the package-manager usability, while the community puts tremendous effort into other things. To be sure, I have no right to complain: it’s a FOSS project, to which I’ve donated little time and no money. But it’s frustrating when I get weird behavior like this (some output omitted for clarity):

         $ nix-env --upgrade --dry-run
         upgrading 'nix-2.13.3' to 'nix-2.14.1'
         
         $ nix-env -iA nixpkgs.nix
         replacing old 'nix-2.13.3'
         installing 'nix-2.13.3'
         

         Like… what’s going on here? I could try to debug it, and maybe I would even be able to figure out the answer, but it feels like the UI has lots of sharp edges like this.

         1. 2

            alper 1 month ago | link

            There are a bunch of two year old tickets in Github where project members themselves can’t agree on what needs to be done here. Lots of weird stagnation and infighting going on there.
   2. 5

      hauleth 1 month ago | link

      I think that this way (expressive base that you can build “sane user experience” on top) is quite good approach. That way Nix do not need to cover and workaround any issue, but you can build tools like one above on top of it. That allows you to “get dirty” when needed and fix problems on your own when you need it, while being “clean” when you do not need it.
   3. 1

      alper 1 month ago | link

      Honestly the nix language is a dog syntactically and a stone around the neck of what is a really good idea around package management.

1. 5

   hauleth 1 month ago | link | on: Jailbreaking a Dreametech Z10 Pro vacuum

   I like the click baiting original title.

1. 15

   asenchi 1 month ago | link | on: The funniest performance regression you've seen

   Not necessarily an application performance but a team/infra performance increase. Some of the details are foggy, but I worked at a company that wanted to add search to a long list of items in a separate PostgreSQL database. The main database was approximately 1.5T in size. The design the team came up with was this new service that would slurp up all the data into an Elasticsearch cluster and then build specific searches around templates. Seems reasonable, and often what is reached for right? Sure, until our database guy chimed in one day and said, “You know… instead of spending $20k on this application’s infrastructure, you could just use the full-text search capabilities in PostgreSQL.”

   Thankfully the team listened and not only did we save a bunch of cash, development time went from 3 months to build the service to a month or so to get the frontend assets right. It was awesome to see. PostgreSQL is great.

   1. 14

      alicebob edited 1 month ago | link

      our database guy chimed in one day and said

      When people say “10 * time programmers don’t exist” I think of cases like these. They absolutely do exist: they are the people who take a step back, think about another solution, and avoid building the whole system altogether.

      (And yeah, postgres is great)

      1. 5

         david_chisnall 1 month ago | link

         I’ve started writing a blog on this exact topic, because I’ve been complaining about it for 20 years: tech companies focus on hiring problem solvers, they don’t focus on hiring people who can identify the right problems to solve.

         1. 2

            vladislavp 1 month ago | link

            would be good to read! I am of opinion that the hiring practices are even less relevant than the above.

            I think the companies are hiring ‘Jeopardy contestants’ – not book/novel writers. Both types deal with ‘words’ – but at a different level of composition, obviously.

            Certainly there are different tasks needed a varying spectrum of the composition skills – but testing for a Jeopardy-like skills, does not reveal much of the composition skills.

            However, these are the interviewing processes of today.
      2. 5

         hauleth 1 month ago | link

         10x programmer isn’t someone who write 10x more code. It is someone who is willing to spend their time on finding solution that will suffice and will take 10x less time to implement.

1. 48

   apg 2 months ago | link | on: WebMutex

   This API is in a new class of products I like to call “Deadlock as a Service”, or DaaS for short.

   1. 18

      hauleth 2 months ago | link

      And DaaS ist nicht gut

1. 7

   knl 2 months ago | link | on: Nuenv: an experimental Nushell environment for Nix

   How would replacing bash with nushell play with bootstrapping of nix and nixpkgs? When comparing guix and nix, guix did quite a good job on that topic and there is really a minimal set of packages to build everything from scratch. I’m wondering if bringing Rust in, just to build nushell, just to build stdenv based on it, would make bootstrapping nearly impossible.

   1. 5

      NinjaTrappeur edited 2 months ago | link

      100% agree, this article completely eludes this central question. Bash is semi-trivial to bootstrap!

      Nixpkgs does not bootstrap rustc, we’re currently using a binary distribution: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/rust/rustc.nix#L28

      Adopting this as a default stdenv would require to push this massive bindist to the Nixpkgs bootstrap seed. That seed is already massive compared to what Guix has, I don’t think we want to degrade this further.

      Rust is definitely source-bootstrapable, as a matter of fact, Guix manages to do it, there’s no reason we can’t do the same. The bootstrap chain is pretty long though. On top of what we already bootstrap (gcc, openssl, etc.), we’d need to bootstrap llvm, mrustc, then rust 1.54 -> 55 -> 56 -> 57 -> 58 -> 60 -> 61 -> 62 -> 63 -> 64 -> 65.

      So yeah, pushing this to stdenv would considerably degrade the bootstrap story in any case.

      From my perspective, Bash is a local optimum, I personally wouldn’t change it, it’s certainly a good balance between a language that is easy to bootstrap and a good-enough expressiveness to express builds.

      If we really want to move to something more modern, Oil could be a more serious contender, they seem to take bootstrapping seriously. There’s a drafted RFC wrt. Oil adoption.

      ---

      [Edit]: The question is eluded, but I don’t think the author expects this to replace stdenv, at least it’s not mentionned in the article. Don’t take this comment as an overwhelming negative “this project is worthless”. Cool hack!

      1. 4

         matklad 2 months ago | link

         This made me realize that Rust is fundamentally non-bootstrapable. It’s definitely going to produce a release every six weeks for quite a number of years, and Rust’s release N needs release N-1 to build, so the bootstrap chain, by design, grows quickly and linearly with time. So it seems that, in the limit, it is really a choice between:

         • using a fully bootstrapped system
         • using Rust
         1. 2

            Corbin 2 months ago | link

            Is there a reference interpreter, perhaps? I imagine that that can’t be a complete solution since LLVM is a dependency, but it would allow pure-Rust toolchains to periodically adjust their bootstraps, so to speak.
         2. 2

            hauleth 2 months ago | link

            There is mrustc which is written in C++ and allows you to bootstrap Rust. There is also GCC Rust implementation in the works, that will allow bootstrapping.
      2. 1

         lucperkins edited 2 months ago | link

         In my defense, I do use the term “experimental” several times and I don’t make any suggestion of replacing stdenv. I could be wrong, but I think that flakes are going to decentralize the Nix ecosystem quite a bit. While the Nixpkgs/stdenv relationship is seemingly ironclad, I don’t see why orgs or subsets of the Nix community couldn’t adopt alternative builders. Any given Nix closure can in principle have N builders involved; they’re all just producing filesystem state after all.

         As for bootstrapping, yes, the cost of something like Nushell/Nuenv is certainly higher than Bash, but it’s worth considering that (a) you don’t need things like curl, jq, and coreutils and (b) one could imagine using Rust feature flags to produce lighter-weight distributions of Nushell that cut out things that aren’t germane to a Nix build environment (like DataFrames support).
      3. 1

         andyc 2 months ago | link

         Yes the new Oil C++ tarball is 375 kilobytes / 90K lines of compressed, readable C++ source :)

         https://www.oilshell.org/release/0.14.2/

         The resulting binary is about 1.3 MB now. I seem to recall that the nushell binary is something like 10 MB or 50 MB, which is typical for Rust binaries. rustc is probably much larger.

         There was some debate about whether Oil’s code gen needs to be bootstrapped. That is possible, but lots of people didn’t seem to realize that the bash build in Nix is not.

         It includes yacc’s generated output, which is less readable than most of Oil’s generated code.

         1. 1

            lucperkins 2 months ago | link

            Nushell is certainly larger! Although I would be curious how much smaller Nushell could be made. Given that it’s Rust, you could in principle use feature flags to bake in only those features that a Nix builder would be likely to use and leave out things like DataFrames support (which is quite likely the “heaviest” part of Nushell).
   2. 1

      qbit 2 months ago | link

      For sure it would make bootstrapping much harder on things like OpenBSD. Good luck if you are on an arch that doesn’t have rust or LLVM. That said, I don’t think this would replace stdenv.. for sure not any time soon!

      Also the article does mention exactly what you are pointing out:

      it’s nowhere near the range of systems that can run Bash.

      1. 1

         knl 2 months ago | link

         My question is orthogonal to this, and maybe I should have specify what I mean by bootstrapping. It’s “how many things I have to build first, before I can have working nixpkgs and build things users ask for”. So if we assume that nushell runs wherever bash runs, how much more effort is to build nushell (and rust and llvm) than bash? I would guess order of magnitude more, thus really complicating the initial setup of nixpkgs (or at least getting them to install without any caches).

1. 4

   ansible-rs 2 months ago | link | on: GitHub unexpectedly rotates its RSA SSH host key

   As yetanotherjosh mentions on the orange site, please take a moment to verify the SHA256 of the new key after you remove the old one and try connecting to github.com again.

   1. 1

      hauleth 2 months ago | link

      You can also just copy and paste their keys from the docs

      github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl
      github.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=
      github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndNxQowgcQnjshcLrqPEiiphnt+VTTvDP6mHBL9j1aNUkY4Ue1gvwnGLVlOhGeYrnZaMgRK6+PKCUXaDbC7qtbW8gIkhL7aGCsOr/C56SJMy/BCZfxd1nWzAOxSDPgVsmerOBYfNqltV9/hWCqBywINIR+5dIg6JTJ72pcEpEjcYgXkE2YEFXV1JHnsKgbLWNlhScqb2UmyRkQyytRLtL+38TGxkxCflmO+5Z8CSSNY7GidjMIZ7Q4zMjA2n1nGrlTDkzwDCsw+wqFPGQA179cnfGWOWRVruj16z6XyvxvjJwbz0wQZ75XK5tKSb7FNyeIEs4TT4jk+S4dhPeAUC5y+bDYirYgM4GC7uEnztnZyaVWQ7B381AK4Qdrwt51ZqExKbQpTUNn+EjqoTwvqNj4kqx5QUCI0ThS/YkOxJCXmPUWZbhjpCg56i+2aB6CmK2JGhn57K5mj0MNdBXA4/WnwH6XoPWJzK5Nyu2zB3nAZp+S5hpQs+p1vN1/wsjk=
      

1. 10

   hauleth 2 months ago | link | on: e69de29bb2d1d6434b8b29ae775ad8c2e48c5391

   I am surprised to see this article translated by author to Polish. It is not perfect Polish (I am native speaker) but still super impressive for someone that is learning Polish as 2nd language (let’s say, that this is not the easiest language to learn).

   1. 8

      arxanas 2 months ago | link

      My wife and her family are Polish— although I rarely discuss hash functions with them!

      I see my articles posted to Polish forums and sites sometimes. I assumed that the Polish tech scene, although burgeoning, is presently somewhat underserved in terms of accessible content, since many of the largest tech organizations work and write primarily in English. I’ve started to translate my blog posts even (especially) when they are particularly technical. Several kind Polish speakers have provided generous corrections and discussion of my grammar and word choice, including a commenter on this very blog post.

      Some of the Polish leaks into my English word choice, such as using “some” instead of “a” or “any” in more situations than I would have before learning Polish.

      I also see my articles on Chinese and Japanese sites, but those languages are a little too much for me to learn right now 😂

      1. 4

         jzb 2 months ago | link

         “although I rarely discuss hash functions with them!”

         This implies that you do, occasionally, discuss them.

         1. 4

            arxanas 2 months ago | link

            My wife and her father work in tech, so sometimes technical topics do arise! Perhaps I once had to explain the blockchain in Polish?
   2. 6

      altano 2 months ago | link

      The post author, @arxanas, is the creator of git-branchless, which is the most ambitious layer over git that I’m aware of. He ported revsets from mercurial ffs. I don’t think he likes easy things :p

1. 3

   hauleth edited 2 months ago | link | on: Things DBs Don't Do - But Should

   For the Democratising Change Events and Soft Deletes section, there is SQL:2011 standard which contain description of “System Temporal Tables” which allows for exactly that (for now I ignore the fact that Democratising Change Events is supported by some DBs, like PostgreSQL, but in non-standard way).

   And about SQL composability - we have libraries like Java’s jOOQ or Elixir’s Ecto which allows exactly that while still using SQL.

1. 3

   hauleth 3 months ago | link | on: What domain registrar is worth using?

   I use OVH and so far had no major issues.

1. 3

   felixgallo 3 months ago | link | on: Phoenix 1.7.0 released: Built-in Tailwind, Verified Routes, LiveView Streams, and what's next

   I’ve bounced off phoenix a few times, and unfortunately this is, in my personal opinion, exemplary why. It appears to be disappearing, like so much front end stuff, into a deeply inscrutable and arcane rabbit hole in which the common language of software is discarded and deep historical understanding of all prior revisions is required to understand this one. What is it to inflect something? Why would you use the word ‘stream’ to denote local front end collections? The profusion of concepts and divergent naming schemes. It’s tough to engage with.

   1. 8

      gamache 3 months ago | link

      What is it to inflect something?

      It’s the process of transforming the case (e.g., PascalCase to camel_case) and/or pluralization of something, and things of that nature. In this example, it’s being able to go from OAuthCallbackController to oauth_callback_url. IIRC the word comes from Rails: https://api.rubyonrails.org/v7.0.4/classes/ActiveSupport/Inflector.html

      Why would you use the word ‘stream’ to denote local front end collections?

      The “stream” denotes that neither the frontend nor the server intends to load all objects in the collection into memory at once.
   2. 8

      srbaker edited 3 months ago | link

      I understand “stream” and “inflect” to be common language of software, not Phoenix specific.

      I really don’t like this idea that we should be able to develop software today without knowing how software has been developed up to now. I’m convinced ours is the only trade that places value on this idea; every other trade seems to place value on learning the past so they can grow.

      This ignorance of the past is how we got “Server Side Rendering” pitched as some kind of new idea that had never been done before. It’s also how we got a whole bunch of new phrases (actor model, micro-services, etc) to describe things we already should have known (OOP).
   3. 3

      hauleth 3 months ago | link

      What is it to inflect something?

      Process of word formation to express different grammatical categories such as […] number, […]. I know that it is not common knowledge among English speakers that such word exists, but in many languages it is used all the time to change gender of the word depending on the context.

      Why would you use the word ‘stream’ to denote local front end collections?

      It is not name for front end collections, but it is coming from verb to stream as you stream events that is happening on the collection instead of operating on collection itself.

      I do not find it hard to understand without understanding background as everything has established meaning already outside of the Phoenix itself.
   4. 2

      yalexr edited 3 months ago | link

      I’m pretty much in the same boat. Phoenix was my goto solution for a while, and then LiveView happened. There’s a lot happening in there, clearly, but I find it less of a load to combine Svelte talking to an Elixir backend for my current project. I wish this were otherwise, i.e., that there were more hours in a day or that I had more energy…

      1. 2

         friendlysock 3 months ago | link

         Luckily, you should still be able to enjoy the subset of Phoenix that you use. I might suggest that the path literals now (~p) are a nice replacement for the somewhat gross Routes.user_session_path(socket, :new)) instead of just ~p"/users/login". The other stuff, especially Tailwind and LiveView proper, you can leave out of your project IIRC. :)

1. 17

   felixyz 3 months ago | link | on: rtx: Runtime Executor (asdf rust clone)

   I have become one of those boring people whose first thought is “why not just use Nix” as a response to half of the technical blog posts I see. The existence of all those other projects, package managers, runtime managers, container tooling, tools for sharable reproducible development environment, much of Docker, and much more, when taken together all point to the need for Nix (and the need for Nix to reach a critical point of ease of adoption).

   1. 28

      Student 3 months ago | link

      Well, maybe there’s a reason why nix hasn’t seen significant adoption?

      1. 10

         domenkozar 3 months ago | link

         The Nix community has been aware of the DX pitfalls that prevented developers to be happy with the tooling.

         I’ve made https://devenv.sh to address these and make it easy for newcomers, let me know if you hit any issues.

         1. 3

            WilhelmVonWeiner 3 months ago | link

            +1 for devenv. It’s boss. The only thing I think it’s truly “missing” at the moment is package versioning (correct me if I’m wrong).
         2. 2

            felixyz 3 months ago | link

            Love it! (As in: I haven’t had a reason to try it yet, but this is definitely the way to go!)
         3. 1

            jdxcode edited 3 months ago | link

            it doesn’t appear to support using different versions of runtimes—which is the entire point of asdf/rtx in the first place. I’m not sure why I would use devenv over homebrew if I didn’t care about versions.

            1. 5

               lygaret 3 months ago | link

               I think the idea is a devenv per-project, not globally, like a .tool-versions file; as you say, it’d be a bit of a non-sequitor otherwise
            2. 2

               hauleth 3 months ago | link

               Devenv, just like Nix, support that OOTB. You simply define different shell per project.
      2. 19

         tonnydourado 3 months ago | link

         No, it’s the children who are wrong
      3. 6

         WilhelmVonWeiner 3 months ago | link

         Primarily the bad taste the lacking UX and documentation leaves in people’s mouths. Python development is especially crap with Nix, even if you’re using dream2nix or mach-nix or poetry2nix or whatever2nix. Technically, Nix is awesome and this is the kind of thing the Nix package manager excels at.

         1. 2

            felixyz 3 months ago | link

            I’ve found mach-nix[1] very usable! I’m not primarily working with Python though.

            [1] https://github.com/DavHau/mach-nix
      4. 5

         felixyz 3 months ago | link

         Yes, it’s way too hard to learn!
      5. 6

         ngp 3 months ago | link

         because the documentation is horrible, the UX is bad, and it doesn’t like it when you try to do something outside of it’s bounds. It also solves different problems from containers (well, there’s some overlap, but a networking model is not part of Nix).
   2. 12

      gamache 3 months ago | link

      I’ll adopt Nix the moment that the cure hurts less than the disease. If someone gave Nix the same UX as Rtx or Asdf, people would flock to it. Instead it has the UX of a tire fire (but with more copy-paste from people’s blogs) and a street team that mostly alienates 3/4 of the nerds who encounter it.

      1. 7

         domenkozar 3 months ago | link

         Curious did you try https://denvenv.sh yet?

         1. 4

            joshka 3 months ago | link

            https://devenv.sh for those clicking…
         2. 3

            gamache 3 months ago | link

            No, thanks for the link! This looks like a real usability improvement. I don’t know if I am in the target audience, but I could see this being very useful for reproducing env in QA.
   3. 10

      jdxcode 3 months ago | link

      It’s like using kubernetes. Apparently it’s great if you can figure out how to use it.

      I’ve given up twice trying to use nix personally. I think it’s just for people smarter than me.

      1. 7

         felixyz 3 months ago | link

         Heh, that’s a good counterpoint. I would say, unlike with k8s I get very immediate benefits from even superficial nix use. (I do use k8s too, but only because I work with people who know it very well.) I assure you (honest) I’m not very smart. I just focus on using nix in the simplest way possible that gives me daily value, and add a little something every few months or so. I still have a long way to go!

         The How it works section of the rtx README sounds very much like nix + direnv! (And of course, I’m not saying there’s no place for tools like rtx, looks like a great project!)
   4. 4

      TeddyDD 3 months ago | link

      Nix is another solution that treats the symptoms but not the disease. I used asdf (and now rtx) mainly for Python because somehow Python devs find it acceptable to break backwards compatibility between minor versions. Therefore, some libraries define min and max supported interpreter version.

      Still, I’d rather use rtx than nix. Better documentation and UX than anything Nix community created since 2003.

      1. 4

         felixyz 3 months ago | link

         It’s clearly out of scope for Nix (or adf, rtx…) to fix the practices of the Python community?

         1. 1

            TeddyDD 3 months ago | link

            Sure. It’s good that a better alternative for asdf exists, although it would be better that such a program wasn’t needed at all.
   5. 4

      maxm 3 months ago | link

      Isn’t it somewhat difficult to pin collections of different versions of software for different directories with Nix?

      1. 7

         adamcstephens 3 months ago | link

         Yes it is difficult. Nix is great at “give me Rust” but not as great at “give me Rust 1.64.0”. That said for Rust itself there aren’t third party repos that provide such capability.

         1. 5

            winter 3 months ago | link

            I think you meant s/aren’t/are :)

            1. 2

               adamcstephens 3 months ago | link

               Correct. Bad typo. :)
         2. 4

            mat 3 months ago | link

            I think you are pointing out that nixpkgs tends to only ship a single version of the Rust compiler. While nixpkgs is a big component of the Nix ecosystem, Nix itself has no limitations prevent using it to install multiple version of Rust.

            1. 4

               adamcstephens 3 months ago | link

               Obviously nix itself has no limitation as I mentioned there are other projects to enable this capability. While you are correct I was referring to nixpkgs, for all intents nixpkgs is part of the nix ecosystem. Without nixpkgs, very few people would be using or talking about nix.
         3. 3

            gnubison 3 months ago | link

            I thought that was the point of Nix, that different packages could use their own versions of dependencies. Was I misunderstanding?

            1. 5

               winter 3 months ago | link

               What Adam means here is that depending on what revision of Nixpkgs you pull in, you will only be able to choose one version of rustc. (We only package one version of rustc, the latest stable, at any given time.)

               Of course, that doesn’t stop you from mixing and matching packages from different Nixpkgs versions, they’re just… not the easiest thing to find if you want to be given a specific package version.

               (Though for Rust specifically, as Adam mentioned, there are two projects that are able to do this easier: rust-overlay and Fenix.)

               1. 3

                  felixyz 3 months ago | link

                  This is a great tool to find a revision of Nixpkgs that has a specific version of some package that you need: https://lazamar.co.uk/nix-versions/

                  That said, it’s too hard, and flakes provides much nicer DX.
               2. 3

                  5d22b 3 months ago | link

                  for Rust specifically, […] there are two projects that are able to do this easier: rust-overlay and Fenix

                  The original https://github.com/mozilla/nixpkgs-mozilla still works too, as far as I know. I use it, including to have multiple versions of Rust.
               3. 3

                  gnubison 3 months ago | link

                  Alright, thanks!
      2. 3

         felixyz 3 months ago | link

         No I wouldn’t say so, especially using flakes. (It gets trickier if you want to use nix to pin all the libs used by a project. It’s not complicated in theory, but there are different and often multiple solutions per language.)
   6. 2

      flying_dutchman 3 months ago | link

      Any pointers on how I can accomplish the same functionality of asdf in Nix?

      1. 8

         joshka 3 months ago | link

         nix-shell -p nodejs-18_x jq
         nodejs -v
         jq --version
         

         Docs https://nixos.org/guides/declarative-and-reproducible-developer-environments.html or use https://devenv.sh/
      2. 3

         felixyz 3 months ago | link

         These are some quick ways to get started:

         Without flakes: https://nix.dev/tutorials/ad-hoc-developer-environments#ad-hoc-envs

         With flakes: https://zero-to-nix.com/start/nix-develop

         And add direnv to get automatic activation of environment-per-directory: https://determinate.systems/posts/nix-direnv

         Or try devenv: https://devenv.sh/

         (Pros: much easier to get started. Cons: very new, doesn’t yet allow you to pick all old versions of a language, for example.)
   7. 1

      southclaws 3 months ago | link

      I have become one of those boring people who just downloads an installer and double clicks it.

1. 9

   soatok 3 months ago | link | on: What are you doing this week?

   I’m working on a draft for a blog post about database cryptography, as a follow-up to a footnote on a friend’s blog post.

   Database cryptography is hard. The above sketch is not complete and does not address several threats! This article is quite long, so I will not be sharing the fixes.

   1. 9

      lorddimwit 3 months ago | link

      I knew a guy who believed that setting the column encoding to “binary” meant that the column was encrypted and safe for directly storing passwords.

      He was a senior developer. He got paid very well while believing this.

      1. 7

         tonyarkles 3 months ago | link

         I wish I was making this up… Back in the day (before JWT was a well-established thing), we had a JSON blob that the the client needed to retain and send back to the server for reasons, and I pointed out that the JSON had data in it that was a) slightly sensitive and b) could be modified by the untrusted client to do bad things. The following sprint he stated that he’d addressed the problem by encrypting it. I had a look and it was literally ROT-13 “encrypted”. When I raised the following concern… his response was “you only know that it’s ROT-13 encrypted because you looked at the code. No one’s going to guess that.”

         1. 6

            david_chisnall 3 months ago | link

            This kind of thing is why I am glad I always work with a red team when building anything I want to make security claims about. They may not find everything I’ve done wrong, but they at least find anything I’ve done embarrassing wrong.

            1. 3

               tonyarkles 3 months ago | link

               That’s the right mindset! I mean, when someone’s learning or has a knowledge gap, I’m more than happy to help out. When they double down on being wrong… that’s a frustration I’m not great at dealing with.

               1. 3

                  soatok 3 months ago | link

                  “Tell me more. When _____ why do you believe _____?” is a good tool to use.
         2. 5

            soatok 3 months ago | link

            Lmao rot13 is literally one of the first cryptography challenges you encounter when learning

            1. 3

               hauleth 3 months ago | link

               I am no cryptographer, but I probably could solve the ROT-13 using just pen and paper.