Threads for lucasew

  1. 6

    Just a little nitpick

    Instead of import file.nix { inherit pkgs; } you can pkgs.callPackage file.nix { }.

    That way you can destructure the same way as people do in nixpkgs and can use the function override if you want later.

    You can also use a recursive attrset instead of that let expression.

    1. 2

      Thanks! I’m new to the ways of Nix.

      1. 1

        What would be the benefit of using rec attrset instead of the let expression?

        1. 2

          DRY

          rec attrset works like both an attrset and a let expression

          1. 2

            I prefer let … in … over rec attrsets, you can always use inherit for the fields to not repeat yourself that much and I had bad experience with rec in the past (the errors were not that helpful).

      1. 15

        I am a Computer Science undergrad Nix/NixOS user with over 2 years of experience in this [0], and I tried to apply Nix at $WORK.

        We use it to set up dev environments to use Blender with some extra sauce and libraries, so we can replicate the same environment basically everywhere. We found a way to use Blender as a Jupyter Kernel based on someone else’s code [1].

        Initially, we used Google Colab to run some stuff we need, and we were able to run it there by using a script I did [2].

        Now we use a beefy machine for the number crunching, so we use Nix to build container images to run there. The same common code is reused, the difference is basically the entry point.

        My advice is to not try to use it everywhere. Is it cool? Yes! But it can get overkill very quickly. It is giving very nice fruit for this Blender specific use case, but for training stuff it’s better to just stick with one of the many ready-made supported containers.

        Another advice is that for iterative work just build the base container and when you are using that in the final machine just pass the code folder as a volume. We had a lot of iterations to set it up, but after it’s done we didn’t have to rebuild and upload the image anymore.

        A not so cool thing about Nix containers is that it’s not integrated with Docker. When you run docker build it builds and consequently adds to the local storage. With Nix, you use buildLayeredImage to generate that script, then you must docker load it later. It also sucks if you want to set up stuff to run on Kubernetes. Another reason to stick with that code volume strategy.

        If you need GPU stuff you will need NixGL and you need to be careful because the way NixGL is built is based on impure derivations, so if you build the image in your Intel you have to figure out how to generate the NixGL for that Nvidia card or your code will probably not find the GPU. The nixpkgs Blender has no support for CUDA by default, so you will want to stick with nix-warez’s Blender [3].

        [0] https://github.com/lucasew/nixcfg/

        [1] https://github.com/cheng-chi/blender_notebook

        [2] https://github.com/lucasew/nix-on-colab

        [3] https://github.com/edolstra/nix-warez/blob/master/blender/flake.nix

        1. 1

          That’s a really nice functional reproducible workaround! xD

          1. 2

            After reading the landing page I’m still unsure what this project is about. What would definitely help me are some examples on the page, because the given example is without much annotations.

            1. 3

              From what I can tell from the docs and source code, the idea seems to be a language independent CLI parsing library. Essentially, you write a nix script that identifies all your sub commands, flags, and positional parameters. It then generates a Bash script which handles all the command line parsing and validation and runs the correct executable.

              Some things like allowing a CLI to default back to a certain environment variable is a nice touch that isn’t present in every CLI parsing libarry. I can also imagine that it would be about the only sane option if you have multiple utilities written in different languages that you want to expose behind a single subcommand interface.

              Again, though, this is based on a five minute skim of the code.

              1. 2

                Exactly

                BTW I was doing some module work to allow polyglot codegen, so far it only generates shell script but implementing more languages should be trivial, your own code could be readFiled to the prelude then it generates your main function for C for example. I might move some target options to somewhere else.

            1. 2

              Oh yes. SQLite runs everywhere, it’s more like a storage format than a DBMS that the apps connect to and we don’t see much stuff with other databases around embedded use cases.

              1. 2

                I had a hard time until I realized this but solved this, at least partially [1], on nixpkgs by sending a pull request [2].

                [1] https://github.com/NixOS/nixpkgs/pull/133807#issuecomment-921712627 [2] https://github.com/NixOS/nixpkgs/pull/133807

                1. 2

                  I want to finish my Gemini browser for mobile programming classes.

                  Now it’s crashing with some specific links. I think it’s when it parses the URLs and finds out that it starts with a space the parser raises an exception. I need to make something to normalize that, probably a common class.

                  I plan to release AGENA after I present it. It’s a fully native app that uses native elements to render gmi.

                  I think it can be easily expanded to work with other small web protocols like gopher, if it doesn’t know how to handle some file or protocol it just runs the intent so if you tap in an HTML button it opens as if you clicked a link on Telegram, for example.

                  Yesterday I was experimenting with creating a PowerShell library using F# [1].

                  [1] https://github.com/lucasew/playground/tree/21d6725247b2a4af28be7673ead4a1931359d1c2/f-sharp/powershell-get-hello

                  1. 7

                    How will you ensure that you can still build zig from sources in the future?

                    1. 27

                      By forever maintaining two implementations of the compiler - one in C, one in Zig. This way you will always be able to bootstrap from source in three steps:

                      1. Use system C compiler to build C implementation from source. We call this stage1. stage1 is only capable of outputting C code.
                      2. Use stage1 to build the Zig implementation to .c code. Use system C compiler to build from this .c code. We call this stage2.
                      3. Use stage2 to build the Zig implementation again. The output is our final zig binary to ship to the user. At this point, if you build the Zig implementation again, you get back the same binary.

                      https://github.com/ziglang/zig-bootstrap

                      1. 7

                        I’m curious, is there some reason you don’t instead write a backend for the Zig implementation of the compiler to output C code? That seems like it would be easier than maintaining an entirely separate compiler. What am I missing?

                        1. 2

                          That is the current plan as far as I’m aware

                          1. 1

                            The above post says they wanted two separate compilers, one written in C and one in Zig. I’m wondering why they just have one compiler written in Zig that can also output C code as a target. Have it compile itself to C, zip up the C code, and now you have a bootstrap compiler that can build on any system with a C compiler.

                            1. 2

                              In the above linked Zig Roadmap video, Andrew explains that their current plan is halfway between what you are saying and what was said above. They plan to have the Zig compiler output ‘ugly’ C, then they will manually clean up those C files and version control them, and as they add new features to the Zig source, they will port those features to the C codebase.

                              1. 2

                                I just watched this talk and learned a bit more. It does seem like the plan is to use the C backend to compile the Zig compiler to C. What interests me though is there will be a manual cleanup process and then two separate codebases will be maintained. I’m curious why an auto-generated C compiler wouldn’t be good enough for bootstrapping without manual cleanup.

                                1. 7

                                  Generated source code usually isn’t considered to be acceptable from an auditing/chain of trust point of view. Don’t expect the C code generated by the Zig compiler’s C backend to be normal readable C, expect something closer to minified js in style but without the minification aspect. Downloading a tarball of such generated C source should be considered equivalent to downloading an opaque binary to start the bootstrapping process.

                                  Being able to trust a compiler toolchain is extremely important from a security perspective, and the Zig project believes that this extra work is worth it.

                                  1. 2

                                    That makes a lot of sense! Thank you for the clear and detailed response :)

                                  2. 2

                                    It would work fine, but it wouldn’t be legitimate as a bootstrappable build because the build would rely on a big auto-generated artifact. An auto-generated artifact isn’t source code. The question is: what do you need to build Zig, other than source code?

                                    It could be reasonable to write and maintain a relatively simple Zig interpreter that’s just good enough to run the Zig compiler, if the interpreter is written in a language that builds cleanly from C… like Lua, or JavaScript using Fabrice Bellard’s QuickJS.

                                    1. 1

                                      Except that you can’t bootstrap C, so you’re back where you started?

                                      1. 2

                                        The issue is not to be completely free of all bootstrap seeds. The issue is to avoid making new ones. C is the most widely accepted and practical bootstrap target. What do you think is a better alternative?

                                        1. 1

                                          C isn’t necessarily a bad choice today, but I think it needs to be explicitly acknowledged in this kind of discussion. C isn’t better at being bootstrapped than Zig, many just happen to have chosen it in their seed.

                                          A C compiler written in Zig or Rust to allow bootstrapping old code without encouraging new C code to be written could be a great project, for example.

                                          1. 5

                                            This is in fact being worked on: https://github.com/Vexu/arocc

                              2. 1

                                Or do like Golang. For bootstrap you need to:

                                1. Build Go 1.4 (the last one made in C)
                                2. Build the latest Go using the compiler from step 1
                                3. Build the latest Go using the compiler from step 2
                              3. 3

                                Build the Zig compiler to Wasm, then run it to cross-compile the new compiler. Wasm is forever.

                                1. 11

                                  I certainly hope that’s true, but in reality wasm has existed for 5 years and C has existed for 50.

                                  1. 2

                                    The issue is building from maintained source code with a widely accepted bootstrapping base, like a C compiler.

                                    The Zig plan is to compile the compiler to C using its own C backend, once, and then refactor that output into something to maintain as source code. This compiler would only need to have the C backend.

                                    1. 1

                                      I mean, if it is, then it should have the time to grow some much needed features.

                                      https://dl.acm.org/doi/10.1145/3426422.3426978

                                    2. 1

                                      It’s okay if you don’t know because it’s not your language, but is this how Go works? I know there’s some kind of C bootstrap involved.

                                      1. 4

                                        The Go compiler used to be written in C. Around 1.4 they switched to a Go compiler written in Go. If you were setting up an entirely new platform (and not use cross compiling), i believe the recommended steps are still get a C compiler working, build Go 1.4, then update from 1.4 to latest.

                                    3. 2

                                      How do we build C compilers from source?

                                      1. 3

                                        Bootstrapping a C compiler is usually much easier than bootstrapping a chain of some-other-language compilers.

                                        1. 4

                                          Only if you accept a c compiler in your bootstrap seed and don’t accept a some-other-language compiler in your seed.

                                          1. 3

                                            Theoretically. But from a practical point of view? Yes, there are systems like Redox (Rust), but in most cases the C compiler is an inevitable piece of the puzzle (the bootstrapping chain) when building an operating system. And in such cases, I would (when focused on simplicity) rather prefer a language that depends just on C (that I already have) instead of a sequence of previous versions of its own compilers. (and I say that as someone, who does most of his work in Java – which is terrible from the bootstrapping point-of-view)

                                            However, I do not object much against the dependence on previous versions of your compiler. It is often the way to go, because you want to write your compiler in a higher language instead of some old-school C and because you create a language and you believe in its qualities, you use it also for writing the compiler. What I do not understand is why someone (not this particular case, I saw this pattern before many times) present the “self-hosted” as an advantage…

                                            1. 2

                                              The self-hosted Zig compiler provides much faster compile times and is easier to hack, allowing language development to move forward. In theory the gains could be done in a different language, but some of the kind of optimizations used are exactly the kind of thing Zig is good at. See this talk for some examples: https://media.handmade-seattle.com/practical-data-oriented-design/.

                                            2. 1

                                              But you could make a C compiler (or a C interpreter) from scratch relatively easily.

                                      1. 1

                                        Yesterday and today I was experimenting with approaches to create a command pallete. So far just the basic: each item with a icon and a title. I did experiment with Electron and PySimpleGUI.

                                        Using nix-shell shebangs is pretty useful for these quick PoCs.

                                        I’m also doing some university stuff, one of them is something I will share after the semester ends.