-
For new comers I recommend to ask questions and search the Discourse forum at https://discourse.nixos.org/
We are starting to have good content there and questions get answered. The longer form allows to better expand on questions in my experience.
-
I then turned to the Nix manual to learn more about working with and creating packages and failed, even with help from the nixos IRC channel and issue tracker. I think the fundamental cause is that it wasn’t written for newbies to learn nix from; there’s a man-page like approach where it only makes sense if you already understand it.
I very much agree with that. Like it’s mentioned in the intro Nix has been mostly designed bottom-up and it shows in the UX. In that regard it’s a bit like Git. It solves a fundamental problem (no more cache invalidation issues!) but requires the user to go through the long and painful learning journey until they emerge from the other side.
On the other end of the spectrum, Homebrew has the best package manager UX that I know of. It’s very easy to create new packages, the CLI is easy to use and has a nice and colored outputs with helpful error messages. It’s just not that robust so one has to cleanup things regularly. Or link/unlink packages when different versions of things are needed.
By the way the Nix community is aware of those issues but we don’t have a UX-first team that has been formed yet. And fixing things like that takes a long time but I still have hope.
-
Logging is a user interface for the developers. Like any UX some care has to be taken to design it properly so that it becomes useful.
-
The issue of copyright and DMCA being used to prevent repair in the transport industry is more widespread that the article mentions. One of the first instances that I have seen is John Deer using it to prevent tractor repair1. As more and more logic moves to software it becomes easier for them to abuse this power.
-
The author wants to use the tool for tasks it is not designed for, or does not know how to use the tool correctly.
Passing on “Distributed version control sucks for distributing software” which is just non-sense.
-
Distributed version control sucks for distributed development
The problem shows up when I’m sitting in my hotel room and need to re-create the local repository over the poor connection. Now I’m not just downloading the one revision I want to work on; I’m downloading every revision ever.
Nothing forces you to do that in git. Just download the one revision you want to work on. This point reinforces also the idea that the author is certainly not working in the industry (academic setting) and seems to have no idea of necessary more advanced features used in developping complex systems collaboratively (with several products / features, in stable branches in which new features have to be backported).
-
Distributed version control sucks for long-lived projects
The history continues to grow; a single version doesn’t. This may be a smooth progression rather than a sudden state change: over time it becomes more the case that the history grows faster than the current version. And so a system that forces every copy to contain all of history will eventually, inevitably, have bigger copies than a system that only stores current versions.
And then the author rants about DVCS sucking for archiving. And sees absolutely no contradiction in those two positions. If you forget your history because you only keep current versions, you are nor archiving anything. It is impossible to replicate a past version of a system, for historical purpose, exploration or just to help a user stuck with an old system.
-
Distributed version control sucks for archiving
Use a database.
The author is just closed off in his own environment and has a poor grasp of the tools he is using on top of it. This rant is useless, and his peers were right to shut him off.
-
-
I didn’t like it either, but you’re strawmanning one criticism. I think OP’s claim is that a centralized repo is easier to archive because everything stays on a primary copy, with people subsetting into secondary copies. So it’s clear what to back up. There’s no contradiction there.
-
-
Is it possible to start a supervisor as a user? Think docker-compose but without the containers. For now only supervisord seems to handle that use-case.
-
It is. I have an ~/svc directory I use, amongst other things, for starting ssh “tunnels.” That is, ssh connections that exist purely to port forward.
my boot script is more-or-less taken from the docs and customized to this circumstance. Here I define a lockfile where I place the process id of s6-svscan and set up s6-log as a catch-all logger. The script exits once s6-log has come up:
#!/usr/bin/env execlineb importas -i HOME HOME cd ${HOME} background -d { s6-setlock -n ${HOME}/run/a.pid getpid pid importas -u pid pid foreground { redirfd -w 1 ${HOME}/run/a.pid echo ${pid} } foreground { rm -f ${HOME}/svc/s6-log/fifo } foreground { mkfifo -m0600 ${HOME}/svc/s6-log/fifo } redirfd -r 0 /dev/null redirfd -wnb 1 ${HOME}/svc/s6-log/fifo fdmove -c 2 1 s6-svscan -S -t0 ./svc } importas -i BOOT ! wait { ${BOOT} } s6-svwait -u ${HOME}/svc/s6-logI have a wrapper around pstree I use to show my service process tree:
#!/usr/bin/env execlineb importas -i HOME HOME backtick -i -n pid { head -n1 ${HOME}/run/a.pid } importas -u pid pid pstree -acplu ${pid}Which produces output like:
s6-svscan -S -t0 ./svc |-s6-supervise s6-log | `-s6-log t n9 s1048576 ./log |-s6-supervise ssh-host1 | `-ssh -MNT -oControlPersist=no host1 `-s6-supervise ssh-host2 `-ssh -MNT -oControlPersist=no host2ssh by itself doesn’t strictly need all this hair: I have additional services other than ssh I’m not showing here. It does permit me to start more than one ssh session to a particular host concurrently (i.e., as a side effect in a script) which s6 turns in to a single start. That lets me bring a tunnel up every time I need it without worrying about whether it’s started or not. Or even whether s6 has been started yet.
-
-
It was a mistake to add a pastebin URL in the AUR build script, it’s way too easy to detect and it looks fishy from the first glance. Instead, re-package the upstream tarball with the script and put it in a github release. AUR users might review the BUILD script but probably won’t look at the upstream code and might not notice the URL switch if it comes from a legit-looking source like github.
That’s what a more sophisticated attacker would do.
-
vvvv is pretty much what the author is talking about. The base data structure is the list and everything is built on top of that. Components can be connected together and then grouped into new components, with some fallback on code with the more complicated parts.
see https://vvvv.org/screenshots
The main caveat is that it only works great if your data is lists or stream-oriented.
The language is not very well knows but also actively used by some V-jay and other artists.
-
I’m not familiar with this, but it looks very cool!
There are actually a lot of environments that are sort of along the same lines. A poster here has written a system like this in Lua that’s quite impressive, and as I mentioned, there’s the Smalltalk environments.
A lot of systems built along these lines are for graphics, which is fine, but I’m more interested in optimizing for the use-case of letting people who don’t identify as programmers do real work in a way that has graphical representation that makes sense to them.
A lot of people have specific, complex, idiosyncratic systems for organizing their every-day tasks that they’ve built on top of inflexible things like spreadsheets or collections of existing websites, and I’d like to cater to that kind of thing – making it possible for them to automate some of that stuff and have an easier time because they can produce a structure that’s a better fit.
-
Excel is actually the most successful programming environment in that regard. The builtin tracing (intermediate values stored in cells) and the spacial representation makes it approachable for people who think that they are not programmers. I think that this is amazing as those same people’s mind would often freeze if they assumed it was a programming language. It’s also deployed in countless of environment and powers a lot of business processes around the world.
But as you noticed, Excel only works for specific types of problems. So far, all the attempts to produce a generic visual programming language have been a failure in my opinion so maybe the solution is to keep the language specialized and then make many of them so all the use-cases can be covered. Create a bunch of UI DSLs.
-
-
-
-
One of the biggest griefs in Go is that channels and IO are not composable in a select statement. For example this is not possible:
select { case msg := <-myChan: case msg := <-myFile.Read(): }Instead, the file reader has to be wrapped in a goroutine and then forwarded to a channel. Then make sure to handle the file closing and shutdown the goroutine properly. I am aware that there are implementation details that make this difficult but since Go went to the trouble of implementing the NxM multiplexing, it’s a missed opportunity to make communication patterns very simple.
-
FOSS isn’t capitalism and capitalism is built on FOSS.
It’s quite amazing how much free and open source software is being produced every day. People that aren’t even paid build amazing libraries, software, … Just look at GitHub and how much stuff there is! Obviously quantity isn’t everything but the power law needs a lot of crap software to produce something good :)
Most companies are pretty much build on top of FOSS now. If you took all the FOSS, companies wouldn’t be able to operate and would have to invest huge sums of R&D to make up for all of this. Typically a company will only sponsor one project. At the same time, there are 10 concurrent experiments going on on GitHub. It’s really hard to compete with that.
We should celebrate this.
Take anyone in the street, they will have no idea that everything they use is built on top of free or open source software. For them, all they see is the end-product. Attribution is part of licenses but usually it’s tucked away in a 10 level deep menu. For them we are a bunch of magicians that make good money and they probably don’t see that selfless and passionate part that is in a lot of us.
-
That would be an appropriate response to the article had the article been appropriately titled! (Instead, the title is a reference to an article that hasn’t made the rounds here, about class division created by technical knowledge & how it impacts the design decisions in Mastodon & Pleroma.)
FWIW, I agree with you that the open source reframing has made FOSS a great boon for capital. (Whether or not this boon for capital has gone on to benefit the rest of us moreso than had capital retained their earlier disgust reaction to free software is hard to say. Being encouraged to make use of free software has made it possible for programmers within industry to do more in less time, but I suspect this was being done during the 90s with BSD-licensed software and just not discussed with management.)
-
-
-
Where YAML gets most of it’s bad reputation from is actually not from YAML but because some project (to name a few; Ansible, Salt, Helm, …) shoehorn a programming language into YAML by adding a template language on top. And then try to pretend that it’s declarative because YAML. YAML + Templating is as declarative as any languages that has branches and loops, except that YAML hasn’t been designed to be a programming language and it’s rather quite poor at it.
-
In the early days, Ant (Java build tool) made this mistake. And it keeps getting made. For simple configuration, YAML might be fine (though I don’t enjoy using it), but there comes a point where a programming language needs to be there. Support both: YAML (or TOML, or even JSON) and then a programming language (statically typed, please, don’t make the mistake that Gradle made in using Groovy – discovery is awful).
-
-
There is also UCL (Universal Config Language?) which is like nginx config + json/yaml emitters + macros + imports. It does some things that bother me so I stick to TOML but it seems like it is gaining some traction in FreeBSDd world. There is one thing I like about it which is there is a CLI for getting/setting elements in a UCL file.
-
-
-
-
Yep! People haven’t learned from mistakes. Here’s a list of XML based programming languages.
-
-
I recently discovered how horribly complicated traditional init scripts are whilst using Alpine Linux. OpenRC might be modern, but it’s still complicated.
Runit seems to be the nicest I’ve come across. It asks the question “why do we need to do all of this anyway? What’s the point?”
It rejects the idea of forking and instead requires everything to run in the foreground:
/etc/sv/nginx/run:
#!/bin/sh exec nginx -g 'daemon off;'/etc/sv/smbd/run
#!/bin/sh mkdir -p /run/samba exec smbd -F -S/etc/sv/murmur/run
#!/bin/sh exec murmurd -ini /etc/murmur.ini -fg 2>&1Waiting for other services to load first does not require special features in the init system itself. Instead you can write the dependency directly into the service file in the form of a “start this service” request:
/etc/sv/cron/run
#!/bin/sh sv start socklog-unix || exit 1 exec cron -fWhere my implementation of runit (Void Linux) seems to fall flat on its face is logging. I hoped it would do something nice like redirect stdout and stderr of these supervised processes by default. Instead you manually have to create a new file and folder for each service that explicitly runs its own copy of the logger. Annoying. I hope I’ve been missing something.
The only other feature I can think of is “reloading” a service, which Aker does in the article via this line:
ExecReload=kill -HUP $MAINPID
I’d make the argument that in all circumstances where you need this you could probably run the command yourself. Thoughts?
-
Where my implementation of runit (Void Linux) seems to fall flat on its face is logging. I hoped it would do something nice like redirect stdout and stderr of these supervised processes by default. Instead you manually have to create a new file and folder for each service that explicitly runs its own copy of the logger. Annoying. I hope I’ve been missing something.
The logging mechanism works like this to be stable and only lose logs in case
runsvand the log service would die. Another thing about separate logging services is thatstdout/stderrorare not necessarily tagged, adding all this stuff torunsvwould just bloat it.There is definitively room for improvements as
logger(1)is broken since some time in the way void uses it at the moment (You can blame systemd for that). My idea to simplify logging services to centralize the way how logging is done can be found here https://github.com/voidlinux/void-runit/pull/65. For me the ability to execsvlogd(8)fromvlogger(8)to have a more lossless logging mechanism is more important than the main functionality of replacinglogger(1). -
-
Instead you can write the dependency directly into the service file in the form of a “start this service” request
But that neither solves starting daemons in parallel, or even at all, if they are run in the ‘wrong’ order. Depending on network being setup, for example, brings complexity to each of those shell scripts.
I’m of the opinion that a dsl of whitelisted items (systemd) is much nicer to handle than writing shell scripts, along with the standardized commands instead of having to know which services that accepts ‘reload’ vs ‘restart’ or some other variation in commands - those kind of niceties are gone when the shell scripts are individually an interface each.
-
The runit/daemontools philosophy is to just keep trying until something finally runs. So if the order is wrong, presumably the service dies if a dependent service is not running, in which case it’ll just get restart. So eventually things progress towards a functioning state. IMO, given that a service needs to handle the services it depends on crashing at any time anyways to ensure correct behaviour, I don’t feel there is significant value in encoding this in an init system. A dependent service could also be moved to running on another machine which this would not work in as well.
-
It’s the same philosophy as network-level dependencies. A web app that depends on a mail service for some operations is not going to shutdown or wait to boot if the mail service is down. Each dependency should have a tunable retry logic, usually with an exponential backoff.
-
-
But that neither solves starting daemons in parallel, or even at all, if they are run in the ‘wrong’ order.
That was my initial thought, but it turns out the opposite is true. The services are retried until they work. Things are definitely paralleled – there is not “exit” in these scripts, so there is no physical way of running them in a linear (non-parallel) nature.
Ignoring the theory: void’s runit provides the second fastest init boot I’ve ever had. The only thing that beats it is a custom init I wrote, but that was very hardware (ARM Chromebook) and user specific.
-
-
-
Dependency resolving on daemon manager level is very important so that it will kill/restart dependent services
Why? The runit/daemontools philsophy is just to try to keep something running forever, so if something dies, just restart it. If one restarts a service, than either those that depend on it will die or they will handle it fine and continue with their life.
-
either those that depend on it will die or they will handle it fine
If they die, and are configured to restart, they will keep bouncing up and down while the dependency is down? I think having dependency resolution is definitely better than that. Restart the dependency, then the dependent.
-
-
-
-
It’s a computer, it’s meant to do dumb things over and over again
I would rather have my computer do less dumb things over and over personally.
And presumably that faulty component will be fixed pretty quickly anyways, right?
Maybe; it depends on what went wrong precisely, how easy it is to fix, etc. We’re not necessarily just talking about standard daemons - plenty of places run their own custom services (web apps, microservices, whatever). The dependency tree can be complicated. Ideally once something is fixed everything that depends on it can restart immediately, rather than waiting for the next automatic attempt which could (with the exponential backoff that proponents typically propose) take quite a while. And personally I’d rather have my logs show only a single failure rather than several for one incident.
But, there are merits to having a super-simple system too, I can see that. It depends on your needs and preferences. I think both ways of handling things are valid; I prefer dependency management, but I’m not a fan of Systemd.
-
I would rather have my computer do less dumb things over and over personally.
Why, though? What’s the technical argument. daemontools (and I assume runit) do sleep 1 second between retries, which for a computer is basically equivalent to it being entirely idle. It seems to me that a lot of people just get a bad feeling about running something that will immediately crash.
Maybe; it depends on what went wrong precisely, how easy it is to fix, etc. We’re not necessarily just talking about standard daemons - plenty of places run their own custom services (web apps, microservices, whatever).
What’s the distinction here? Also, with microservices the dependency graph in the init system almost certainly doesn’t represent the dependency graph of the microservice as it’s likely talking to services on other machines.
I think both ways of handling things are valid
Yeah, I cannot provide an objective argument as to why one should prefer one to the other. I do think this is a nice little example of the slow creep of complexity in systems. Adding a pinch of dependency management here because it feels right, and a teaspoon of plugin system there because we want things to be extensible, and a deciliter of proxies everywhere because of microservices. I think it’s worth taking a moment every now and again and stepping back and considering where we want to spend our complexity budget. I, personally, don’t want to spend it on the init system so I like the simple approach here (especially since with microservies the init dependency graph doesn’t reflect the reality of the service anymore). But as you point out, positions may vary.
-
Why, though? What’s the technical argument
Unnecessary wakeup, power use (especially for a laptop), noise in the logs from restarts that were always bound to fail, unnecessary delay before restart when restart actually does become possible. None of these arguments are particularly strong, but they’re not completely invalid either.
We’re not necessarily just talking about standard daemons …
What’s the distinction here?
I was trying to point out that we shouldn’t make too many generalisations about how services might behave when they have a dependency missing, nor assume that it is always ok just to let them fail (edit:) or that they will be easy to fix. There could be exceptions.
-
-
-
Perhaps wandering off topic, but this is a good way to trigger even worse cascade failures.
eg, an RSS reader that falls back to polling every second if it gets something other than 200. I retire a URL, and now a million clients start pounding my server with a flood of traffic.
There are a number of local services (time, dns) which probably make some noise upon startup. It may not annoy you to have one computer misbehave, but the recipient of that noise may disagree.
In short, dumb systems are irresponsible.
-
But what is someone supposed to do? I cannot force a million people using my RSS tool not to retry every second on failure. This is just the reality of running services. Not to mention all the other issues that come up with not being in a controlled environment and running something loose on the internet such as being DDoS’d.
-
I think you are responsible if you are the one who puts the dumb loop in your code. If end users do something dumb, then that’s on them, but especially, especially, for failure cases where the user may not know or observe what happens until it’s too late, do not ship dangerous defaults. Most users will not change them.
-
-
-
-
-
If those thing bother you, why run Linux at all? :P
-
-
-
-
USB keyboard/mouse polling doesn’t eat CPU because it isn’t done by the CPU. IIUC the USB controller generates an interrupt when data is received. I feel like this analogy isn’t a good one (regardless). Checking a USB device for a few bytes of data is nothing like (for example) starting a Java VM to host a web service which takes some time to read its config and load its caches only to then fall over because some dependency isn’t running.
-
-
-
-
-
I really like runit on void. I do like the simplicity of SystemD target files from a package manager perspective, but I don’t like how systemd tries to do everything (consolekit/logind, mounting, xinet, etc.)
I wish it just did services and dependencies. Then it’d be easier to write other systemd implementations, with better tooling (I’m not a fan of systemctl or journalctl’s interfaces).
-
You might like my own dinit (https://github.com/davmac314/dinit). It somewhat aims for that - handle services and dependencies, leave everything else to the pre-existing toolchain. It’s not quite finished but it’s becoming quite usable and I’ve been booting my system with it for some time now.
-
-
-
-
Runits
sv(8)has the reload command which sendsSIGHUPby default. The default behavior (for each control command) can be changed in runit by creating a small script under$service_name/control/$control_code. -
I was thinking of the difference between ‘restart’ and ‘reload’.
Reload is only useful when:
- You can’t afford to lose a few seconds of service uptime (OR the service is ridiculously slow to load)
- AND the daemon supports an on-line reload functionality.
I have not been in environments where this is necessary, restart has always done me well. I assume that the primary use cases are high-uptime webservers and databases.
My thoughts were along the lines o: If you’re running a high-uptime service, you probably don’t care about the extra effort of writing ‘killall -HUP nginx’ than ‘systemctl reload nginx’. In fact I’d prefer to do that than take the risk of the init system re-interpreting a reload to be something else, like reloading other services too, and bringing down my uptime.
-
-
I hoped it would do something nice like redirect stdout and stderr of these supervised processes by default. Instead you manually have to create a new file and folder for each service that explicitly runs its own copy of the logger. Annoying. I hope I’ve been missing something.
I used to use something like logexec for that, to “wrap” the program inside the runit script, and send output to syslog. I agree it would be nice if it were builtin.
-
-
But it’s pretty annoying to have to tell all your daemons not to fork
Most daemons should not fork. The double-fork-exec hack to re-parent the daemon on top of PID 1 and then write a PID file is a legacy thing. There are all sorts of ways it can go wrong and was only really useful for systems that didn’t have any process manager.
In this specific use-case though it makes sense because Nginx is using fork to do zero-downtime restarts.
-
If the strings are immutable, why is string.Split(”,”) allocating so much? In Go for example it would returns a slice to the original strict which is only a few bytes long.
-
I have been following Factor from afar since the early beginning around 2008. The main thing that stands out to me is how much libraries a few developers were able to create. While most developers are happy if they publish 3-4 libraries here we have 3-4 core developers publishing hundreds of libraries. This goes from a rich C FFI, a network stack with all common protocols, a graphical stack with OpenGL rendering, a UI library, sound, all sorts of inputs, a great documentation system, … Each of these categories require great domain expertise. This is an incredible achievement.