1. 4

    Nice! Gunna chime in here and tell you to check out my collection of Nix expressions for my fully declarative, functional, atomic configuration I use across my macOS systems:
    https://github.com/cmacrae/.nixpkgs

    Start by digging through the darwin-configuration.nix if you’re curious, it’ll lead onto the other expressions with include statements.

    Of course, this is all achievable on NixOS (and a lot more), and any system that supports Nix.
    Hope someone finds this helpful/inspirational!

    1. 6

      Woo! Excited to see some Nix stuff being mentioned.

      I had been running NixOS on my work Macbook for a while, but due to issues with a mix of HiDPI & non HiDPI displays on Linux - I went back to macOS.

      With a new found love though, I was delighted to find out about nix-darwin - LnL has always been really friendly and helpful on #nixos, when I had questions about achieving something with it.

      As such, here’s my collection of expressions that declare the system configuration I use across my macOS machines: https://github.com/cmacrae/.nixpkgs

      I’d certainly consider myself an absolute novice, but as you can see - even with little experience you can cobble something fairly comprehensive together.

      My next plans are formed around my home infrastructure. Currently, I have a little rack - with one shelf occupied by a little Joyent Triton cluster made up of 3 intel NUCs. On top of Triton I run a number of home media services which are in lx-branded OS containers. Right now, I’ve formed a workflow around Packer, Ansible, and Terraform for creating images and deploying services.

      I’m planning to introduce NixOS as a base lx-branded OS image, which you could then “inject” Nix system expressions into for declarative, reproducible images for varying deployments and services.

      1. 2

        I find nix-darwin and your nixPGS for macOS really interesting. I want to try it out, but am pretty much non-nix smart.

        I’ve just ordered a new MacBook, and will need to move everything over.
        Is there a way to take your existing configuration and put it into nix-darwin?

        Is there an idiot’s guide to getting started and making this all work somewhere?

        How do I know the name of the variables I can set?

        Keyboard

        system.keyboard = { enableKeyMapping = true; remapCapsLockToControl = true; };

        this is awesome, but how could I have figured it out except seeing it in your config?

        I’m not exactly an idiot, but around nix, I definitely am :)

        1. 4

          I’m afraid there’s no good answer to that at the moment, I should probably look into how nixos builds the configuration.nix manpage. Currently you’ll need to use the darwin-option command or look at the sources.

          1. 3

            Hey, I’m pretty much non-nix smart too!

            The only means of taking your existing config and putting it into nix-darwin is to express it yourself.

            The NixOS manual and Nixpkgs manual are the best resources for learning.

            Regarding knowing the expressions you can use, I’ve found it’s a combination of what you learn from the NixOS manual and digging through the available modules in nix-darwin. It’s pretty easy to understand what’s possible with these when reading them. Nix is pretty verbose, so it should be clear what values you can/cannot pass to module parameters.

            On top of that, you also have my config to go on! And LnL has an ‘examples’ directory in the ‘modules’ part of nix-darwin that you may also be able to learn a bit from.

            1. 2

              Poking me on irc is also an option, I’m happy to answer any questions you have. As for things that are not supported yet feature requests or contributions are very welcome.

        1. 1

          Personally for me (as a user of OpenSolaris) what was disappointing for me is when the project forked, it was basically the kernel with a few (at the time partially broken) build scripts. I am glad you posted this, as it now looks like OpenIndiana (among others) is fairly stable with a half decent community, when I get chance, Ill test it out on my Desktop. Since OpenSolaris/Belenix development was stopped, I moved to FreeBSD (zfs support mainly), I did check back in every now and again on the illumos community, but never really saw progress towards a stable server platform.

          Having used FreeBSD for 6 years now, I doubt I will switch back to an OpenSolaris based platform, but it will be interesting to see where the project is at.

          1. 2

            never really saw progress towards a stable server platform

            SmartOS has been an extremely stable server platform for a long time

          1. 2

            I’m thinking on moving from gnome ti i3/sway. How did you do the step and why?

            1. 6

              If you’re moving from GNOME to i3, you may be interested in reading about running gnome-session with i3 so you still have access to GNOME features like auto-mounting removable devices, media keys, screen-locking, etc.

              1. 1

                Your reply touches something that I don’t really understand because I always used gnome. So for me it’s hard to tell where gnome ends and linux starts. Can you explain quickly why I want to run a gnom-session inside i3?

                1. 5

                  It’s not to run gnome-session inside i3, it’s to replace GNOME’s window manager with i3 in a GNOME session.
                  GNOME is much more than just a window manager, it’s a Desktop Environment. It provides a suite of softwares to manage your desktop. i3, along with most other dynamic/tiling window managers are just window managers.
                  Meaning: they will only provide you with a means of managing X windows. On their own, they will not provide a dock, menu/status bar, application launcher, notification system, etc. However, some (like i3) will actually have ready made solutions to replace some these components, and if one doesn’t come directly from the project, you just build up your suite of components yourself :)

                  It’s very much a more modular approach: GNOME is kinda like a flat-packed Desktop Environment, whereas if you go down the dynamic/tiling window manager route, it’ll be more like building your own thing with LEGO - which can be really fun, and beneficial in some ways.

                  I think Screwtape’s suggestion to try out i3 inside of a GNOME session is so that you could try i3 for what it is: a window manager - but still have the comfort of GNOME (the menu/status bar, notification system, application launcher, workspace manager).

                  1. 2

                    I think he already did: to get the listed features that gnome implements.

                    That said, there are other implementations too.

                2. 1

                  i3 was pretty easy to get used to, but I had to spend an hour or so practicing after reading https://i3wm.org/docs/userguide.html

                1. 8

                  Ah, Bryan - articulate and energetic as ever. For any unfamiliar with bcantril and Joyent, I implore you to go and watch any of his talks on YouTube, truly engaging, informative and entertaining - not to mention they make kickass open source platforms…

                  One of the world’s largest companies (!) sees what you yourself saw, and has seen fit to supercharge the technology with the strength of its size.

                  This hits the proverbial nail on the head for me. I’ve been a Joyent fanboy for quite some time, and I’m so excited to see where this new team-up takes them and their incredible solutions/innovations.

                  I’m sure I speak on behalf of all Joyeurs, when I say: congratulations, guys!

                  1. 4

                    I find it very difficult to care about filesystems. It’s about as exciting to me as printer drivers. I currently use ext4 because it was a default and I had no reason to try anything else. Can someone explain what appreciable difference a filesystem would make on my everyday usage of computers?

                    1. 16

                      I’m only really excited because Apple (might) get rid of the .DS_Store files finder creates when viewing directories.

                      1. 2

                        I was hoping the new FS would be case sensitive by default, just because it’s what I’m used to from Linux. But it won’t be.

                        1. 4

                          https://developer.apple.com/library/prerelease/content/documentation/FileManagement/Conceptual/APFS_Guide/UsingtheAppleFileSystem/UsingtheAppleFileSystem.html#//apple_ref/doc/uid/TP40016999-CH4-SW1

                          APFS defaults to case sensitive currently, I’m not sure if that’s going to stay the default going forward but it seems like progress on that front.

                          1. 5

                            It likely won’t stay the default.

                            Consider that there’s a lot of legacy software in the Apple ecosystem that isn’t very careful about case normalization because it doesn’t have to be. You can format a volume case-sensitive HFS+ today (and you could also do case-sensitive UFS in the past), but a ton of stuff is broken, including big-name apps like Steam. Apple has always stuck with case-insensitivity in the default install because doing anything else breaks too much.

                            1. 1

                              Is a case-sensitive filesystem considered a good thing?

                              In other words, I’m curious what benefits there are to be had in the ability to have Foo.txt and foo.txt side-by-side.

                              1. 6

                                For “normal” people, having a case-insensitive file system is nice, so if they fat-finger the caps-lock key they still get the file they want.

                                For programmers, having a case-sensitive file system is nice, so the file you create with a given name is always distinct from other files with logically different names.

                                Imagine writing some sort of “cache” files to disk that are named using some hash that produces a combination of upper and lower-case letters. On a case-insensitive file system, you’re eventually going to end up with collisions. That’s a bummer (and hours of debugging time lost) to have to worry about, especially when your program needs to work cross-platform.

                                1. 5

                                  Normal people never type the names of existing files.

                                  1. 4

                                    What Windows does (as a compromise) is NTFS and the kernel are case sensitive, but the Win32 subsystem is not by default. Users get what they expect, and other subsystems can get the semantics they want. (Case sensitivity is toggleable for Win32 as well, but I wouldn’t recommend this.)

                          2. 10

                            Sure:

                            • Snapshots
                            • Self-healing files

                            Snapshots are particularly useful. Sorta like a git commit, you can always go back to that point, even if you delete files.. etc. With HammerFS(2? DragonflyBSD Only), ZFS, BTRFS and FreeBSD FFS (maybe more..) you get the snapshotting.

                            1. 12

                              Hey @qbit, been a while!

                              I figure I’ll weigh in here too.

                              As already mentioned: snapshots. If you ever try SmartOS/SmartDataCenter (recently renamed to Triton) by Joyent, you might end up playing around with container snapshots (zones). It’s crazy that I can go into a zone, or a bunch of zones, completely destroy the filesystem, then roll back to a safe snapshot in a matter of seconds (data type/size would make “seconds” vary here, but it’s always fast).

                              I remember being absolutely blown away by the notion of VM “flavours” being available in a repo, just like packages, this was before the big hit of Docker becoming widely known and adopted. The fact I could go onto my SmartOS headnode and do a imgadm avail | fgrep redis then grab that “image” in no time from Joyent’s remote repo, then deploy straight to a zone was just baffling. Why am I harping on about this? Because this framework revolves around ZFS snapshots bundled up with some metadata compressed into a tarball. Pretty damn cool.

                              Which then leads me to some of the utilities ZFS has available, like zfs send and zfs receive: https://duckduckgo.com/?q=zfs+send+receive&ia=web

                              I won’t babble on about that, check out those search results.

                              There’s also the ability to implement file sharing protocols, like NFS/smb, at the filesystem level with some simple flags when making/modifying volumes. I use a SmartOS server at home with a few NFS shares set up directly when I made the volumes.

                              Another thing: on the fly expansion/shrinking of volumes. Really cool when you want to chuck some extra space at some zones, or bring them down.

                              All of this is largely from my own experience of administering servers with ZFS, I’ve never used it on a desktop/laptop. However, were I to go down that path (if it was presented to me in a simple, solid manner), I’d be using snapshots, send/recieve to make backups all the damn time.

                              ZFS is fucking great - I’ll end on that.

                              1. 1

                                From the perspective of an OSX user, would there be much of an improvement over the kind of snapshotting Time Machine does? I realize it’s not at the filesystem level, and not nearly as flexible if you’re managing big storage arrays and such, but for a desktop/laptop user it seems like a “good enough” solution.

                                1. 3

                                  It might be that TM does a good enough job, but other things like the self-healing stuff can take it a step further. Say you have a “raid” volume (used quotes because zfs has its own naming) and a file gets corrupted on one of the raid mirrors, zfs will check the file’s checksum against other mirrors and replace the broken file with a known good copy. TM in this example would just put the corrupt file into your TM backup.

                                  All that said, my main FS is OpenBSD’s FFS, which has none of these features, and I have never had issues :P

                                  1. [Comment removed by author]

                                    1. 2

                                      iMacs come with this SSD/spinny disk fusion, which ZFS has pretty good support for out of the box (where the SSD becomes a cache rather than extra storage space). On top of that, for things like video editing (which Apple still has a strong presence on, I think?), something like ZFS gives you a lot of options to increase the throughput of storage. And, while ZFS supports a RAID setup well, it does have a lot of value for single disk setups. I use it on my desktops without a hitch. The COW-semantics of APFS, from what I can tell skimming the docs, are very similar to what ZFS does, so at the very least a subset of ZFS is useful enough for Apple devices.

                                      1. 1

                                        It can be done with a single drive as well (obviously this isn’t recommended :P). Also I am not arguing that people are or should be doing this (running any of these FS2.0 file systems). I am just expanding on my previous list of “distinguishing features”.

                                2. 4

                                  I use ZFS everywhere I can and am really pleased with it and don’t think I’d feel comfortable going back to something else. The two main values I get from ZFS are that it ensure data is valid via checksums. I have been stung by hardware or software corrupting my data and ZFS has protections against that. The second is snapshots. Snapshots are cheap in ZFS, so on my workstations I take snapshots at various units of 5 min, 15 min, 1 hour, 1 day, 1 week, 1 month and retain them for various periods of time. Then transferring snapshots around is easy so I can back up these to other machines really painlessly.

                                  With snapshots, you can do other really powerful things that you might not realize you want to do until you have them. The biggest one is boot environments. This makes it so you can snapshot your installation and switch between them on boot. The usecase for this is if you’re going to do a big upgrade you can role it back if it breaks. The power that something like ZFS gives you is that you can ensure the packages and kernel are always in sync. While existing OS’s like Ubuntu might keep multiple kernel versions laying around, you don’t have any guarantees that the rest of the system still makes sense if you rollback. You do have those guarantees with boot environments.

                                  Then there are other nice things you can do, for example if you have a lot of data and you want to experiment with it, you can clone it (cheap), play with it, and destroy it, without harming the original data. If you are using Solaris or FreeBSD, there are things like jails which are whole-system containers, that become much easier and more powerful with ZFS (creating new ones becomes fast and cheap, so you can make heavy use of them).

                                  Then, if you’re admining any system, ZFS lets you do a lot of useful things, even delegate control of portions of the filesystem to users so they can do the work they want themselves. Running any serious storage box benefits from ZFS on basically every axis (performance, durability, operationally).

                                  So, of course, it depends. For myself, ZFS has given me the ability to do things I didn’t realize I wanted to do before as well as given me increased safety about my data. On top of that, it’s benefiting me as a person who admins some machines and as regular-joe-user. I used to rsync data to multiple USB drives as backup, now I can just transfer incremental snapshots around which is much safer and significantly faster.

                                1. 9

                                  Shameless plug: if you want to get up and running with pkgsrc/pkgin with Joyent’s repo, you can use the Save OS X bootstrap script: https://github.com/cmacrae/saveosx

                                  1. 3

                                    SaveOSX is the best way to bootstrap pkgsrc on OS X, I’ve been using it for a long time!

                                  1. 3

                                    Thanks for sharing. I always enjoy it when urbit talks/papers come around.

                                    1. 2

                                      Cool - definitely trying this out.

                                      Minor typo in the shell example at the bottom of the front page:
                                      infinit-volume --mount --name company --ountpoint /mnt/company/

                                      Guessing --ountpoint should read --mountpoint :)

                                      1. 2

                                        Woops, will be fixed soon, thanks. Let us know how that went when you have some time!

                                      1. 4

                                        Such a great extension. I used to work with /u/lili2311 - you guys really did an awesome job on this.
                                        Beautiful design and quite frankly, now, a necessity when using such services. Surprising to think that I hadn’t even thought about it before; but this is the sort of information that the guys running the services should be providing in the first place!

                                        1. 1

                                          Cheers! Yeah, they really should be. The cynic in me suspects the reason they don’t is monetary, given how many badly rated places are at the top of their listings.

                                          1. 2

                                            I think you hit the nail on the unfortunate-head there.
                                            Keep up the good work!

                                          1. 1

                                            Dude, nice. Thank you!

                                            Edit: It seems that this isn’t quite the same idea – the website definitely could use a new design, that’s for sure.

                                            1. 3

                                              Also worth mentioning explainshell.com

                                          1. 1

                                            Brilliant post, thanks for sharing

                                            1. 4

                                              Excellent talk, thanks for sharing!

                                              1. 2
                                                1. 1

                                                  This is awesome. Thank you for your rump kernel posts, keep ‘em coming :)

                                                  1. 1

                                                    Here’s my IDM/Chillwave/Ambient playlist. It’s a compilation of my favourite tracks from my favourite genres (both in no particular order). It’s very chill, easy to concentrate when listening to, there’s not much in the way of vocals.

                                                    Enjoy!

                                                    1. 15

                                                      I wasn’t sure sharing my tools/workflow would help you much, as you state you’re on Windows, but I saw your comment saying you’re waiting for an Ubuntu install.

                                                      I don’t use anything particularly out of the ordinary: i3 window manager, rxvt-unicode, zsh, vim. But what I will certainly advocate the use of: tmux. If you’re not already a user, pick it up, use it heavily. It changed the way I work for the better.

                                                      It really helps keep things organized - for instance, I always have 3 sessions running: management, support, and personal. Each session will dictate what kind of tools are open or what kind of actions I carry out. I’m a sysadmin, so maybe these session names don’t speak wonders to you, but I’m sure there’s certain ‘categories’ that segment your workflow, and embodying that in a few shells can really help manage how you conduct your work.

                                                      I am, of course, only pointing out one tiny feature that I find really useful, tmux is so much more, go read up on it if you’re unfamiliar.

                                                      1. 4

                                                        Yup I’ve been using tmux for a decent amount of time, it’s great!

                                                        1. 1

                                                          And a tip for anyone who has been a heavy user of screen long time, you can change the keybindings to similar of screen. (CTRL+A etc)

                                                          With this the switch from screen is really simple.

                                                          1. 1

                                                            As it’s similar, I’ll join mine right here.

                                                            Tools:

                                                            • FreeBSD 11.0-CURRENT on a 13.3" UltraBook
                                                            • i3 window manager – the WM_CLASS to workspace magic is gold.
                                                            • xterm
                                                            • zsh
                                                            • ssh-agent (with different ssh keys for different purposes, think login to
                                                            • hosts, pull from git repos, …) and this helpful snippet in ~/.ssh/config:

                                                              Host *

                                                              ControlPath ~/.ssh/sockets/master-%l-%r@%h:%p

                                                              ControlMaster auto

                                                              ControlPersist yes

                                                              TCPKeepAlive yes

                                                              ServerAliveInterval 5

                                                              ServerAliveCountMax 1

                                                            • tmux (with exposed SSH_AGENT_PID and SSH_AUTH_SOCK variables)

                                                            • git (for work and fun)
                                                            • svn (for FreeBSD purposes)
                                                            • mutt
                                                            • gpg
                                                            • irssi

                                                            When it comes to development practices:

                                                            … and of course, the practice of not doing releases on Fridays.