Threads for asymmetric

  1. 3

    This was both hilarious and adorable

    1. 2

      If anyone else is confused, this is not about synapse or dendrite, the home server implementations, but about front-ends.

      1. 1

        Speaking of JSONFeed, have you seen actually simple syndication (ass)?

        It’s the simplest thing that can work.

        1. 4

          It looks like if you want a full-content feed from ASS you need to fetch the data from the URI and somehow render it?

          The advantage of Atom and JSONfeed is that the content is actually in the feed itself so the reader client doesn’t need to implement any extra steps to get it.

          1. 1

            I have now, I’ll have to think if I can use that.

          1. 2

            We’re anxious to continue solving thorny, un-fun problems for developers

            That sounds like… fun? ;)

            On a more serious note: I see several projects popping up using Nix under the hood, with the explicit intention of hiding it away from users. I wonder if this means that people have given up on Nix ever becoming more ergonomic, and if these kind of projects actually undermine that kind of effort.

            It’s as if git UI tools had become more mainstream instead of git becoming more usable/people just swallowing the complexity. Would we be better off in the end?

            1. 12

              Eelco and I haven’t cofounded DetSys together to hide Nix away. We’re looking at a major problem of Nix’s onboarding process being a bit of a slog, and we’re looking at ways to get people value from Nix as a taste. We see a glorious future for Nix, but to bring it to light we’re trying to meet potential future users where they are.

              There definitely are companies and projects trying to use Nix as part of their product, and don’t have much interest in it being about Nix. Eelco and I aren’t doing that.

            1. 0

              Why is this tagged nix?

              1. 1

                Because we don’t have an immutable-x tag and chatting with a bunch of Nix folks was fairly instrumental in growing my thinking and this design. Dropped since evidently you object.

                1. 1

                  I came to ask the very same question :) I don’t think cram uses Nix, as it states:

                  That No previous statefile warning is the secret sauce of Cram. Cram works in terms not just of this log of what changes it will make, but in terms of a persisted log of what changes it has made. This allows Cram to optimize repeated executions to remove installation steps that haven’t changed, while still retaining a precise log of how to get where you are now from an empty slate. This also allows Cram to clean up after itself.
                  

                  Repo doesn’t contain anything nix-related, either. Interestingly, this python project is built using Bazel, that’s something I rarely see.

                  1. 1

                    That’s a story for another time :P

                    1. 1

                      Having tried to use Bazel for Python… ouch, that’s gotta have some very complicated justification because Bazel’s python support is terrible on top of the already pretty terrible user experience of Bazel.

                  1. 5

                    For me the most interesting aspect, and actually a killer app, of OCaml is MirageOS. I haven’t done anything with it yet, but it seems insanely cool.

                    1. 10

                      Users have to be “trusted” to upload store paths to another host because of this specific risk. Content-addressed store paths do mitigate this, but another way to solve this would be to have no local building at all, where the users don’t ever upload NARs. Derivations are already content-addressed, and users don’t have to be trusted to upload a derivation.

                      1. 1

                        Users have to be “trusted” to upload store paths to another host because of this specific risk.

                        Yes, but if the user can’t upload store paths, then it isn’t very useful as a remote builder of paths that are not available elsewhere (e.g. in development). But it is a nice nuance, thanks for pointing it out.

                        Also, the point I wanted to stress in the post (might have to emphasize it more) is that in case of a compromise, the breach of trust could be hard to detect, as the breach could have happen in any of a number of developer machines.

                        Content-addressed store paths do mitigate this

                        Is this because the CA hash is reflected in the store path hash? Because otherwise, either the attacker could replace the nix binary and not perform the check, or they could simply alter the ca field in the SQL DB to match the new hash.

                        I’m not 100% satisfied with how I handled this in the post.

                        1. 1

                          Users have to be “trusted” to upload store paths to another host because of this specific risk.

                          I’m not sure if these two issues are related or not, but when trying to build a derivation on a remote builder where I’m not a trusted user, Nix throws an error stating that I’m “not privileged to build input-addressed derivations”. Since (I believe) all derivations that aren’t content-addressed are input-addressed, what is the use of a remote builder for an untrusted user?

                          Edit: there’s an issue that details the whole issue of not being able to build derivations remotely as a non-trusted user.

                        1. 10

                          I’m learning Nix lately, and I don’t (yet?) hate the configuration syntax. But I do find the user experience to be on the unfamiliar side, particularly when it comes to terminology. For instance, derivations are Nix’s packages, and everyone calls them packages, but the software doesn’t, so new users can get hung up on that. Better to cut down the learning curve by calling them packages, then teach that Nix packages are different in such and such way.

                          Regarding the configuration language, I wonder if there have been any attempts to translate code in another, more palatable language to Nix? I’m not suggesting a new language like TypeScript. I mean a library in a common language for emitting Nix configuration, the way imperative CDK code yields the CloudFormation YAML that’s so clumsy to write by hand.

                          1. 9

                            If you might like parens more than an MLish DSL (or if you want a full language and not a DSL) come check out Guix :)

                              1. 6

                                If I had a Nickel for every new config language I came across…

                              2. 6

                                For instance, derivations are Nix’s packages, and everyone calls them packages, but the software doesn’t, so new users can get hung up on that.

                                There’s a subtle difference, in that what people usually call “packages” (see e.g. nixpkgs) are higher level expression which get translated to derivations (by nix-instantiate). So the word package is actually quite ambiguous.

                                1. 5

                                  I think generating Nix is a bad idea, because Nix already generates shell code, and invokes many subprocesses. So then you’re going to have a leaky abstraction on top of a leaky abstraction.

                                  It would be better just to build something directly on top of shell, which is fundamental since all packages themselves use it .

                                  And all distros use it, even Nix does. Nix somewhat covers it up, but this long debate shows how central shell is to Nix:

                                  https://github.com/NixOS/rfcs/pull/99#

                                  From my understanding every package build basically depends on this, and then in the Nix package defs you’ll also see templated shell to fix up specific issues, just like any other package manager:

                                  https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh

                                  1. 6

                                    So then you’re going to have a leaky abstraction on top of a leaky abstraction.

                                    I still have scars from this. Nix has some cool functionality that basically reimplements Rust’s Cargo in Nix + shell scripts (buildRustCrate). Since Cargo actually supports features with spaces in them (which I hope no crate will ever use), I once worked on a patch that added support for features with spaces to buildRustCrate. IIIRC I eventually got it working, but it was a terrible quotation hell.

                                    1. 11

                                      Yup exactly, this is the “rewriting upstream” problem, which I mentioned with Bazel here:

                                      http://www.oilshell.org/blog/2021/04/build-ci-comments.html#two-problems-with-bazel-and-gg-again

                                      Both Bazel and Nix have strong models of the world and assume it’s more homogeneous than it actually is. On the other hand, shell embraces heterogeneity and you can always find a solution without rewriting, and containers add to that flexibility.

                                      They should enable correct parallelism, incremental builds, and distribution. (It looks like https://earthly.dev/ is pointing in this direction, though I’ve only read about it and not used it.)

                                      Just like Nix prefers rewriting in its expression language; Bazel prefers rewriting with Starklark. You generally throw out all the autoconf and write Starlark instead, which is a lot of work.

                                      Many years I reviewed the R build rules for Bazel which my coworker wrote, which is analogous to the Rust problem in Nix (thanks for the example).

                                      There is some argument that rewriting is “good”, but simply from spinning our wheels on that work, I no longer have any appetite for it. I’ll concede that upstream doesn’t have all the nice properties of Nix or Bazel, but I also want to spend less time messing with build systems, and I don’t want to clear this hurdle whenever trying out ANY new language (Zig, etc.) I think language-specific package managers are something of a anti-pattern, but that’s the world we live in.

                                      We need a “meta-build” system that solves the reproducibility/parallelism/distribution/incrementality problem for everything at once without requiring O(N) rewrites of upstream build systems. I don’t really have any doubt that this will be based on containers.


                                      Nix and Bazel have something else in common which I mentioned here – they have a static notion of dependencies that has to be evaluated up front, which can be slow.

                                      https://lobste.rs/s/ypwgwp/tvix_we_are_rewriting_nix#c_9wnxyf

                                  2. 3

                                    The solution some friends and I were kicking around was similar to this. At this point, we’ve got a relatively strong sense of how we like to build our servers and appliances, and we figured it’d be nice just to focus in on the parts we care about (mainly, the ease of specifying packages and users and keys and whatnot) in a JSON blob and then have a script barf out the relevant Nix files and flakes.

                                    Sure, we give up like 90% of the bizarre shit Nix gives you, but I honestly don’t need most of that.

                                  1. 5

                                    I have a feeling (untested, unproven) many of the pain points that led the author to k8s/k3s could be solved by a combination of declarative OS config (NixOS) + systemd.

                                    EDIT: TFA mentions NixOS at the end.

                                    1. 2

                                      There’s also Harbormaster https://gitlab.com/stavros/harbormaster. But I wanted to get more knowledge of k8s too as it’s an increasing force in the DevOps world, and I like to be up to date on that side of things.

                                    1. 3

                                      This is so great. The bus factor on the Nix tool is ~1, and this has been causing lots of issues.

                                      Is this meant to be flakes-aware?

                                      Also, what’s it being implemented in? If there’s any code available, I couldn’t see it - navigating the SourceGraph-based forge on mobile was an exercise in frustration.

                                      1. 6

                                        Is this meant to be flakes-aware?

                                        We’re not planning to support experimental features for now beyond what is required for nixpkgs, but note that experiments like flakes can also be implemented in the Nix language - you don’t need tooling support for them.

                                        Also, what’s it being implemented in?

                                        The evaluator is in Rust, store/builder are undecided.

                                        navigating the SourceGraph-based forge on mobile was an exercise in frustration

                                        There’s also a cgit instance at https://code.tvl.fyi - we have a search service that takes a cookie for redirecting to the preferred code browser.

                                        1. 1

                                          I think it’s a shame that Flakes are still considered experimental.

                                          1. 3

                                            I disagree, but opinions are split on this :-)

                                            1. 4

                                              what are the problems you see with flakes?

                                        2. 1

                                          I think I recall reading on Matrix earlier that whatever current source is available is from the ~failed fork attempt, and that the reimplementation would likely be in rust?

                                          (For that matter, I also saw the suggestion that they may be over-estimating how readily ~compatible their effort may be with guix–though I don’t recall for sure if that was on Matrix or orange site.)

                                          1. 2

                                            how readily ~compatible their effort may be with guix

                                            You’re probably referring to this thread. The thing is that we don’t explicitly intend to be compatible with the derivation files, but the fundamental principles of Guix (language evaluation leads to some sort of format for build instructions) has - to our knowledge - not diverged too much to be conceptually compatible.

                                            Note that we haven’t really sat down with anyone from Guix to discuss this yet, for now our efforts are focused on Nix (and personally I believe that a functional, lazy language is the right abstraction for this kind of problem).

                                        1. 2

                                          work: Deep dive on some painful AWS IAM and AWS Glue stuff. It is just no fun.

                                          private project: recently discovered how cool https://datasette.io/ is and I have been building a tool that extracts playgrounds from OpenStreetMap and visualizes them in datasette. This is all driven by having a one year old that likes playgrounds and me wanting to do something with map data again. Good fun so far.

                                          1. 1

                                            Could you expand on this a bit? What would the visualization provide, compared to laying it out on a map?

                                            1. 1

                                              Maybe I was a bit too unclear: I am currently using the datasette cluster-map to discover them on the map. I find osm.org a bit too clunky for a search like that.

                                              The process right now looks like this: I am taking a protobuf osm dump from geofrabrik (http://download.geofabrik.de/) and extract/convert all the playgrounds (only for my state) into sqlite (I wrote this in java, since there are excellent libraries available for all things osm). Then I use https://github.com/simonw/datasette-cluster-map to visualize/discover them. The next goal is to add photos and notes about them as my son and I go to them. It is a bit meant as a log of our playground adventures. Does that make sense?

                                            2. 1

                                              I love datasette! But for your specific use-case, are you aware of overpass turbo?

                                              You could go to https://overpass-turbo.eu/ , search for your location and run a simple query like

                                              node
                                                [leisure=playground]
                                                ({{bbox}});
                                              out;
                                              

                                              to find all playgrounds in the bounding box. The export button allows you to fetch that data, e.g. as a GeoJSON file or as a PNG image :)

                                              1. 2

                                                Yeah, I saw that the other day, but I already had all the code written and too much fun doing it. Also, playgrounds can be of type Node OR Way, so it is a bit more than the above. ;)

                                            1. 6

                                              Any idea why one should switch to Lua for their config?

                                              1. 11

                                                As far as I know, the reasons are any/all of:

                                                1. LuaJit runs much faster than vimscript (vimscript is entirely unoptimized, although not sure how much of a difference this will make in relatively small amounts of code like a config).
                                                2. Lua is a full programming language, which could have utility for more complex configurations.
                                                3. You prefer Lua syntax to vimscript.
                                                1. 8
                                                  1. Lua is a mainstream programming language with lots of great tooling, libraries, etc. that continues to improve at a great pace.

                                                  A big part for me was the frustration of learning vimscript while knowing that I will never use this knowledge and code anywhere else. Now when I mess with my nvim configs or writing a new plugin, I’m also practicing my lua which I can use in other projects. :)

                                                  1. 1

                                                    Yeah good point. I do know (some) vimscript but have converted (most) of my configs to lua specifically to learn lua.

                                                2. 5

                                                  I would switch in a heartbeat to Lua for my configs in all software I use instead of the usual JSON, YAML, TOML.

                                                  Why? Because it allows me to script and abstract things I might want. For the cases I don’t want to abstract anything, I can simply use Lua table notation instead of JSON and serve basically the same purpose.

                                                  It also allows software to move from configuration files to initialization files. The change might seem subtle, but with initialization files, you don’t need to outsmart your user and provide all the features they might ever want or need as you do with dumb configuration files. Instead, you can provide a flexible API and let them build initialization files to serve their unique personal needs.

                                                  1. 3

                                                    In https://changelog.com/podcast/457 TJ (neovim maintainer) says don’t switch to lua just for the sake of it. Search for “to write your entire configuration”.

                                                    I took a snippet of mine that I thought was confusing (to me) in vimscript and ported it over. It didn’t take too long and I had to learn the vim apis to invoke lua etc which was new to me.

                                                    1. 2

                                                      This strikes me as accurate. I was able to get most of my existing init.vim ported to lua, with a couple of small bits not working correctly (or at least me not knowing how to make them work correctly yet). As I mentioned in my above comment, I could leave those small pices in vim.cmd([[ ... ]]) blocks in init.lua and have them work the same as they did before, which is fine by me. This podcast was recorded in August 2021, and I’m looking forward to further improvements in the configuration APIs from the neovim contributors.

                                                      My own motivation for switching was some mix of wanting to try out this new Lua-in-Neovim thing I’d been hearing about, partially wanting to move to some fancy new post-Neovim-0.5 plugins that all had configuration documentation in Lua, and partially wanting to break up my lengthy init.vim into several smaller files. I don’t actually know if it’s impossible to do that in Vimscript, but it was certainly straightforward in Lua.

                                                    2. 2

                                                      Lua is simple and logical, vimscript seems to be the opposite of that.

                                                    1. 12

                                                      This reminds me of meli (https://meli.delivery/), but the idea is to provide tools to build various UIs. Kind of like git porcelain for email. Cool project. :)

                                                      1. 3

                                                        I’m excited to try meli, I’m interested in its JMAP support among other things.

                                                        I hope that it remains under active development, with a lot of these little tools I fear their lone developer running out of time and I worry about making them an important part of my workflow.

                                                        1. 11

                                                          I’m the lone meli dev, I’ve stopped my activity merely because it was the only thing I had to do every day under quarantine lockdown and I kind got sick of it. I haven’t given up on it though and it’s not dead.

                                                          1. 1

                                                            It looks like it hasn’t received much attention in the last year or so, by looking at the mailing lists.

                                                            1. 1

                                                              If you look at the source repo, there’s a bit more activity. Also the author commented above you that it’s not dead :)

                                                        1. 9

                                                          My logic behind making my NixOS config repo a public one is to act as an example for others to take inspiration from. I also wanted to make it harder for me to let the config drift. It also gives me a bunch of fodder for this blog.

                                                          I’ve definitely used your blog as a resource in the process of learning about NixOS and using it in my own home network, so thanks for writing this stuff up.

                                                          1. 12

                                                            I’m going to eventually end up writing a book about administering NixOS servers. What kinds of things would you like to see in such a book, or as future articles on my blog?

                                                            1. 5
                                                              • the process for creating a new VM that uses NixOS is still pretty manual for me, even if I can configure it automatically once I use Nixops for the first time and it generates a key. I would like to see information in how you solve this problem on your infrastructure.
                                                              • Nixops itself seems kind of clumsy. I gather that there’s an upcoming 2.0 release that will change a lot of things about the software, but its still not ready to go yet. I’d be interested in reading about how to best use Nixops today; or if there are other tools for applying a nix configuration to many remote boxes. Also, is there any way to query the state of a NixOS machine to see what’s deployed on it?
                                                              • Hints about how to take software that isn’t yet packaged for nix and making it run on your NixOS infrastructure. Some of the services I run are apparently hard to package for Nix (peertube being a good example), and I’m not sure what I should do to work around this.
                                                              1. 2

                                                                if there are other tools for applying a nix configuration to many remote boxes

                                                                There’s a bunch of other tools (deploy-rs, krops, morph), each with their specific use-case and strengths.

                                                                I ended up using nixos-rebuild with flakes, and it works really well for me (apart from currently not being able to build the derivation remotely). Admittedly, I manage very few hosts, so I can afford to run it for each individual machine.

                                                                But it should be pretty easy to write a little bash script that iterates over a few hosts. Maybe even borrowing some ideas from TFA (which I haven’t yet read).

                                                              2. 5

                                                                As another point of reference, I refer to your blog posts all the time, and mostly use NixOS because of those. I’d buy your book in a heartbeat!