Threads for uvavueranu

  1. 2

    I was wondering if this patch would make vterm output faster as well?

    1. 17

      Its package ecosystem is in excellent condition and packages such as org-mode and eglot / lsp-mode make even the most demanding programming languages a joy to work with in Emacs.

      I work on a large C/C++ codebase as part of my day job and use lsp-mode/eglot (currently eglot) to navigate the code, with very few extensions. I also use the latest mainline Emacs with native compilation. I have been using Emacs for over 25 years and my customization is best categorized as “very light”. In short, my Emacs set up is not much beyond what ships with it.

      And it’s still just… slow. GCC has some pretty big files and opening them can take up to 10 seconds thanks to font-lock mode. (Yes, I know I can configure it to be less decorative, but I find that decoration useful.) It’s much worse when you open a file that is the output from preprocessor expansion (easily 20000+ lines in many cases).

      Log files that are hundreds of megabytes are pretty much a guaranteed way to bring Emacs to a crawl. Incremental search in such a buffer is just painful, even if you M-x find-file-literally.

      I had to turn off nearly everything in lsp-mode/eglot because it does nothing but delay my input. I can start typing and it will be 3-4 characters behind as it tries to find all the completions I’m not asking for. Company, flymake, eldoc are all intolerably slow when working with my codebase, and I have turned them all off or not installed them in the first place.

      M-x term is really useful, but do not attempt to run something that will produce a lot of output to the terminal. It is near intolerable. Literally orders of magnitude slower to display than an xterm or any other terminal emulator. (M-x eterm is no better.)

      The problem, of course, is that Elisp is simply not performant. At all. It’s wonderfully malleable and horribly slow. It’s been this way since I started using it. I had hopes for native compilation, but I’ve been running it for a few months now and it’s still bad. I love Emacs for text editing and will continue to use it. I tried to make it a “lifestyle choice” for a while and realized it’s not a good one if I don’t want to be frustrated all the time. Emacs never seems to feel fast, despite the fast hardware I run it on.

      1. 6

        The performance was the reason for me to leave Emacs. I was an evil mode user anyways so the complete switch to (neo)Vim was simple for me. I just could not accept the slowness of Emacs when in Vim everything is instant.

        E.g. Magit is always named as one of the prime benefits of Emacs. While its functionality is truly amazing its performance is not. Working on a large code base and repository I was sometimes waiting minutes! for a view to open.

        1. 3

          What did you find slow on Emacs aside from Magit?

          I actually use Emacs because I found it really fast compared to other options. For example, the notmuch email client is really quick on massive mailboxes.

          Some packages might be slow, though. I think the trick is to have a minimal configuration with very well chosen packages. I am particularly interested in performance because my machine is really humble (an old NUC with a slow SATA disk).

          1. 2

            To be fair it was some time ago and I don’t remember all the details but using LSPs for code completion/inspection was pretty slow e.g.

            Compared to IDEs it might not even have been slow but similar. I however have to compare to Vim where I have equal capabilities but pretty much everything is instant.

            My machine was BTW pretty good hardware.

            1. 1

              lsp-mode became much more efficient during the last year or so. Eglot is even more lightweight, I think. Perhaps it is worth giving it another go.

              I think there was some initial resistance to LSP in the Emacs community and therefore they were not given the attention they deserve.

              1. 2

                Thanks for the notice! I may try it again in the future but currently I am very happy with my Neovim setup, which took me a long time to setup/tweak :)

          2. 2

            Out of curiosity, were you using Magit on Windows?

            I use Magit every day and my main machine is very slow. (1.5GHz 4 core cortex A53) Magit never struck me as particularly slow, but I’ve heard that on Windows where launching subprocesses takes longer it’s a different story.

            1. 3

              but I’ve heard that on Windows where launching subprocesses takes longer

              Ohh you have no idea how slow in a corporate environment. Going through MSYS2, Windows defender, with windows being windows and a corporate security system on top, it takes… ages. git add a single file? 20 seconds. Create a commit? Over a minute. It’s bonkers if you hit the worst case just right. (On a private Windows install, MSYS2 + Exceptions set in Windows Defender it’s fine though, not much slower as my FreeBSD laptop) I asked around and there is a company wide, hardcoded path on every laptop, that has exceptions in all the security systems just to make life less miserable for programmers. Doesn’t solve it completly, but helps.

              Either wait an eternity or make a mokery of the security concept. Suffice to say I stopped using Windows and Cross-Compile from now on.

              1. 1

                Can confirm. I use Magit on both Linux and Windows, and it takes quite a bit of patience on Windows.

                1. 1

                  With Windows I think it’s it’s particularly git that is slow, and magit spawns git repeatedly. It used also to be very slow on Mac OS as well because of problems with fork performance. On linux, it used to be slow with tramp. There are some tuning suggestions for all of these in the magit manual I think.

                  1. 1

                    Nope on Linux. As mentioned our code base is big and has many branches etc. Not sure where exactly Magit’s bottleneck was. It was quite some time ago. I just remember that I found similar reports online and no real solution to them.

                    I now use Lazygit when I need something more than git cli and it’s a fantastic tool for my purpose. I also can use it from within Vim.

                  2. 1

                    Working on a large code base and repository I was sometimes waiting minutes! for a view to open.

                    This happens for me as well with large changes. I really like Magit but when there are a lot of files it’s nearly unusable. You literally wait for minutes for it to show you an update.

                  3. 4

                    I know you’re not looking to customise much but wrt. terminals, vterm is a lot better in that regard.

                    1. 1

                      I actually switched to M-x shell because I found the line/char mode entry in term-mode to be annoying (and it seems vterm is the same in this respect). shell-mode has all the same slowness of term-mode, of course. I’ve found doing terminal emulation in Emacs to be a lost cause and have given up on it after all these years. I think shell-mode is probably the most usable since it’s more like M-x shell-command than a terminal (and that’s really its best use case).

                      1. 1

                        If you need ansi/curses there’s no good answer and while I like term it was too slow in the end and I left. I do think that for “just” using a shell that eshell is fine though.

                    2. 3

                      Do you use the jit branch of emacs? I found once I switched to that and it had jit compiled things my emacs isn’t “fast” but its pretty boring now in that what used to be slow is now at least performant enough for me not to care.

                      1. 2

                        Is there a brew recipe or instructions on compiling on Mac? Or does checking out the source and running make do the business?

                        1. 3

                          I use the emacs-plus1 package. it compiles the version you specify. currently using emacs-plus@29 with --with-native-comp for native compilation, and probably some other flags.

                          1. 2

                            Thanks again, this is appreciably faster and I’m very pleased 😃

                            1. 2

                              Awesome! also, check out pixel-scroll-precision-mode for the sexiest pixel-by-pixel scrolling. seems to be a little buggy in info-mode, can’t replicate with emacs -Q though, so YMMV.

                            2. 1

                              Thank you that sounds perfect

                            3. 1

                              I’m a Mac user and I found it very hard to compile Emacs.

                              This might be a good starting point however:

                              https://github.com/railwaycat/homebrew-emacsmacport

                              1. 1

                                I honestly don’t know I use nix+home-manager to manage my setup on macos, this is all I did to make it work across nixos/darwin:

                                Added it as a flake input: https://github.com/mitchty/nix/blob/7e75d7373e79163f665d7951829d59485e1efbe2/flake.nix#L42-L45

                                Then added the overlay nixpkgs setup: https://github.com/mitchty/nix/blob/7e75d7373e79163f665d7951829d59485e1efbe2/flake.nix#L84-L87

                                Then just used it like so: https://github.com/mitchty/nix/blob/6fd1eaa12bbee80b6e80f78320e930d859234cd4/home/default.nix#L87-L90

                                I gotta convert more of my config over but that was enough to build it and get my existing ~/.emacs.d working with it and speedy to the point I don’t care about emacs slowness even on macos anymore.

                              2. 1

                                Do you use the jit branch of emacs?

                                Yes. I’ve been using the libgccjit/native compilation version for some time now.

                              3. 2

                                The problem, of course, is that Elisp is simply not performant.

                                That’s half of it. Another half is that, IIRC, Emacs has rather poor support for asynchrony: most of elisp that runs actually blocks UI.

                                1. 1

                                  In short, my Emacs set up is not much beyond what ships with it.

                                  Can share your config? I’m curious to know how minimal you made it.

                                  1. 1

                                    Here you go. It changes a little bit here and there with some experiments.The packages I currently have installed and use are: which-key, fic-mode, counsel, smartparens, magit, and solarized-theme. There may be a few others that I was trying out or are only installed for some language support (markdown, yaml, and so forth).

                                    1. 1

                                      Thank you very much.

                                    2. 1

                                      Quick addendum on the config: that’s my personal config, which morphs into my work setup. My work one actually turns off flymake and eldoc when using eglot.

                                    3. 1

                                      Is there anything that has prevented a Neovim-style rewrite of Emacs? A Neomacs?

                                      I keep hearing about the byzantine C-layer of Emacs and the slowness of Elisp. And Emacs definitely has the community size to develop such an alternative. Why do you think no one has attempted such an effort? Or maybe I should add “recently” to the question. As I know there are other Emacs implementations.

                                      1. 4

                                        As crusty as Emacs source can be, it’s nowhere near as bad Vim source was, which was a rat’s nest of #ifdef. That’s why Neovim had to basically rewrite their way to a fork. The Emacs implementation is surprisingly clean, as long as you can tolerate some of the aged decisions (and GNU bracing).

                                        1. 2

                                          There is Climacs, which isn’t exactly the same, but is close.

                                          The problem for any new Emacs clone will that it has to run all the Elisp out there. Unless there is a substantial speed improvement to Elisp or a very good automatic translation tool, any such project will be doomed from the start.

                                      1. 12

                                        The whole Goggle office app suite is just really bad, and it’s amazing because I’m pretty sure it’s gotten worse over time?

                                        1. 13

                                          I thought there must be something better until I used Microsoft Office 365 and was much more annoyed.

                                          I quite like docs and sheets. I guess it depends on your use case. I just need something simple with not a lot of features.

                                          1. 7

                                            I use Libreoffice and I can’t say I have any complaints. Mind you I am a programmer mostly and I spend less than 1% of my time at work working with office type documents. I have been subjected to the entire google suite by my current company though and I can definitely say I am not a fan.

                                            I am not a fan of cloud services in general. I have a big fat self built pc tower, I have no need for thin clients. I see the value of collaborative tools but I really wish everything was just built on top of git.

                                            1. 8

                                              I value the real time collaboration.

                                              Otherwise, I am a big fan of putting things into version control.

                                              And honestly, the collaboration is 95% about comments which would be handled somewhat well with gitlab, github, another review tool…

                                            2. 6

                                              I find it depends on the tool. For presenter tools, I haven’t found anything better than PowerPoint. Early versions of Keynote had the advantage that they implemented only a subset of the features of PowerPoint (which included the minimum set required to make good presentations) but they gradually copied PowerPoint misfeatures (such as shrinking text if you type more, because slides with 1,000 words are obviously better than slides with 100 words). Google’s thing is awful, so is the Libre/OpenOffice one. PowerPoint’s Design Ideas, morph transitions, and SmartArt largely make up for its other shortcomings (such as awful drawing tools, inability to do syntax highlighting for code, and so on).

                                              I like LaTeX beamer for technical presentations because the combination of the listings package and TikZ lets you make some very clear diagrams including code listing (e.g. control-flow graphs with code in each node) fairly easily and it’s also easy to have a single document that generates the slides and the handouts.

                                              The only better thing I’ve found is Sozi, which will never gain widespread use because it’s too different. Sozi is inspired by Prezzi. It takes an SVG file as input and creates a presentation by panning, zooming, and (the feature that makes it much better than Prezzi) making layers appear and disappear.

                                              When it comes to spreadsheets, they’re all pretty bad. Lotus had two spreadsheet products. The bad one, 123, and the good one, Improv. Everyone copied the bad one. Quantrix Modeller is the only surviving Improv clone and it is orders of magnitude better than any of the others. Apple’s Numbers is probably the best for tiny toy spreadsheets, none of them are appropriate for real work. Jupyter notebooks and Pandas are often a better tool than a spreadsheet for a lot of things people use a spreadsheet for. I’d love to see a good open-source Improv clone but the only one I know of is an unmaintained GNUstep app that is very unfinished.

                                              Word processors are uniformly bad but I’m hugely biased against WYSIWYG. LyX is the only WYMIWYG editor that I’ve tried and it was less slower for me than typing semantic markup directly into a text editor. I’d really like to see a good visual editor for semantic markup. I’d also like to see something with a decent typesetting engine (in 2022, Word still uses a greedy algorithm for line breaking), such as SILE.

                                              M365 does the collaborative editing pretty well. I can edit in the desktop app at the same time someone else edits on the web and we can see each other’s edits live. I can also turn on track changes and be able to review all of their changes before merging them. I wish it made versioning more explicit though.

                                              1. 4

                                                Docs and Sheets are pretty good for basic stuff. The only downside is I’ve never had them work offline correctly, even on a Chromebook.

                                                1. 1

                                                  Strange.

                                                  Docs has definitely worked for me offline a couple of times on train rides but haven’t used that recently.

                                                2. 2

                                                  I thought there must be something better until I used Microsoft Office 365 and was much more annoyed.

                                                  Why would trying those two options give you the impression that there isn’t anything better? LOL just kidding. Try Office 2003.

                                                  1. 3

                                                    We probably have different use cases.

                                                    I mostly have docs with a simple format that I want to collaborate on. E.g. comments.

                                                    I believe that Office 2003 is very capable for traditional office tasks.

                                                    1. 2

                                                      I used Office in 2003 as a student. It might be fine for “office” tasks, but it was totally inadequate for students. Basic reference management just wasn’t there.

                                                      1. 1

                                                        Automated reference management seems like more trouble than it’s worth for the sorts of papers I wrote as a student! Until grad school at least, where everything is LaTeX.

                                                3. 2

                                                  I’m needing to use Gsuite at my new gig and I think on the whole it’s better than the competition.

                                                  But why oh why does everything default to ‘/edit’ and is there an experienced Gsuite user that can point me to a setting or even Firefox plugin or something to fix that? I suppose I could use Tridactyl to rewrite. I very very often just want to read design docs, not accidentally mash some keys into them.

                                                  1. 2

                                                    IDK, I think Google Docs is pretty good. Especially with the pageless mode. I just stick to the default styles and the only formatting I do is setting headings.

                                                    It’s a bit slow but rock solid and the collaboration is top notch.

                                                    1. 3

                                                      Especially with the pageless mode

                                                      It blows my mind that they just introduced this now. Been using this software for over a decade (not by choice) and have never once used it to produce anything that ended up on paper, but for over a decade, every document I worked on had page breaks in it that you couldn’t turn off.

                                                      Absolutely boggling.

                                                      1. 1

                                                        I do agree. I think this is a huge step and hopefully they really take advantage of it. I tried Dropbox Paper which ironically is not so focused on actual paper but it was far too buggy and the collaboration was weak. I guess this will be enough to keep using Google Docs for my D&D notes and stuff.

                                                  1. 3

                                                    Nice project!

                                                    On cursory glance, I think for the Emacs folks, magit-wip-mode can do a lot of this for you

                                                    1. 3

                                                      Can any users familiar with both talk to a comparison of a Nix flake + direnv? Just trying to build my own mental model of Hermit

                                                      1. 4

                                                        I’ve used direnv, and Nix, but not Nix flake. Hermit is more akin to asdf.

                                                        It differs from Nix primarily in that there’s no installation required at all when using a Hermit-enabled repo. You just clone it down and Hermit will auto-install itself and any packages in the repo as they’re used.

                                                        The FAQ has a section comparing it to different package managers including Nix.

                                                        1. 2

                                                          Hermit seems to carve out a smaller scope. In particular, it doesn’t model the process of building tools–just downloading them. And it doesn’t try to manage low level dependencies like libc nor higher level stuff like recreating pypi, npm, and crates.io

                                                          And it doesn’t try to provide security beyond https. No hashes, signatures, etc.

                                                          1. 1

                                                            This is mostly accurate, except there are opt-in SHA-256 hashes.

                                                        1. 2

                                                          Great post. Thanks for sharing. I learned a lot more about Avy. Keen to try to work them into my muscle memory.

                                                          1. 7

                                                            Thanks for sharing. Similarly, I’ve been using deploy-rs to deploy Flake-based configurations to my own low powered devices (PCEngines APU, RPi). Works really well!

                                                            1. 2

                                                              Nice, thank you! I haven’t taken the time to grok flakes yet but I’ll be sure to take a look at deploy-rs when I do. It might take me a while though…

                                                              1. 1

                                                                I only spent an afternoon or two with deploy-rs (had been quite new to Nix in general), and enjoyed it, but didn’t need multiple profiles and found it to be a bit slow. Secret management did not seem to be included either, so that would require an extra tool and while hacking on that, I found https://github.com/zhaofengli/colmena which I am using since. It’s still a prototype officially but works well for me

                                                                1. 1

                                                                  For secrets management agenix (or rangenix) seems to be the best option for me since I can push the secrets to the public repository. Still, it’d be nice to have features of both agenix and deploy-rs in a standard tool like NixOps.

                                                                  1. 2

                                                                    colmena has support for secrets included. One can even specify custom commands, which use together with pass with great pleasure :)

                                                                    1. 0

                                                                      That’s neat. I assume that keyCommand is executed on the local machine instead of the deployment target (though I couldn’t find it in the docs). Agenix, on the other hand, decrypts secrets using the target’s SSH host key at activation time. Both approaches make sense, I’m just more conformable with agenix way—“stateless” encrypted secrets in the Nix store so I can rollback secrets along with the system configuration.

                                                                      I also like that deploy-rs uses flake’s nixosConfigurations output, meaning that I can use plain nix for local deployments. Is it possible to do something similar with colmena-based configuration?

                                                                      1. 1

                                                                        Yes, key command is executed locally - which i personally prefer for my use case but does not allow for unattended reboots - that could be regarded as a limitation or a feature depending on the project.

                                                                        Is it possible to do something similar with colmena-based configuration?

                                                                        Yes, colmena uses NixOS modules like anything else, but does not use nixosConfigurations, so one needs a smallish wrapper to support both, nixosConfigurations and colmena in the same flake. I could try to polish mine a bit and publish it, but haven’t done so yet because I am still new to nix so there might be better ways to do so ;)

                                                                        1. 1

                                                                          Ouch, doesn’t look like colmena supports automatic rollbacks for borked configurations. https://reddit.com/comments/kgj6ir/_/gggcmyo

                                                                          That’s an absolute deal breaker for me since I tend to tinker with network configs way more than I probably should.

                                                                1. 16

                                                                  For me it’s mu4e in Emacs. The speed of mailutils, convenient keybindings and sane composition defaults you don’t have to fight to submit patches.

                                                                  1. 3

                                                                    Another vote here for mu4e. It helps me focus on getting through my inbox to have it outside of my browser and be able to use even more keyboard shortcuts than the gmail interface.

                                                                    1. 3

                                                                      I also use mu4e. I haven’t found another email client that offers the same speed of execution and of user input. It connects with my password manager with a single line of configuration: (auth-source-pass-enable) which is builtin to Emacs. I also have the ability to define custom bookmarks to, with a single keystroke, show me all my inboxes, just my flagged emails, etc.

                                                                      The big feature for me though is contexts. For each email account I have, I define a :match-func function. I actually used a macro to create the functions to match on the account’s given Maildir. A large part of the mu4e workflow is marking messages to delete/flag/move/etc and then executing those marks (similar to dired). When I realized the contexts automatically reassign for each message you mark in “real time”, I was pleasantly surprised. This means, for example, if I there are a bunch of emails in a row from potentially different accounts, I can just spam the d key to mark them for deletion, then x to actually delete, and they will all go to their respective trash folders, not just the trash folder of the context you selected when you launched mu4e.

                                                                      1. 2

                                                                        Yet another vote for mu4e. Been using it for a few years and it’s great. A bonus is that it integrates especially well with orgmode; e.g. it’s trivial to link to emails from within orgmode TODOs, which is exceptionally helpful when a lot of TODOs come in via email :)

                                                                        1. 1

                                                                          I used to use mu4e, but I could never get the moving parts of mu, mbsync and Office365 to play nice together

                                                                          1. 1

                                                                            Same! Would love to hear from anyone with an Emacs-Office365 workflow they’re happy with to be honest.

                                                                            1. 1

                                                                              I’m using Gnus/nnimap now, which works reliably, if sometimes a wee bit slow due to O365 throttling

                                                                            2. 1

                                                                              I use it primarily with office365/exchange via offlineimap.

                                                                          1. 2

                                                                            If I understand correctly, this is meant for use by individual devs, each signing their own commits and tags. How is this better than each dev having their own signing key living on their own machine? I see a lot more of “going rogue” potential in this system.

                                                                            1. 3

                                                                              I understood it a bit differently, this approach is trying to add the same guarantees (as developers signing their commits and tags) to the outputs of the CI.

                                                                              With it, the CI system can sign a commit or, more commonly, a release tag at the same time and using the same role and key material as the other deployable artefacts in the release.

                                                                              1. 1

                                                                                Yep sorry, ended up replying on top of you. This is what was meant in the post.

                                                                              2. 3

                                                                                Not quite, this is more useful for the CI build agents that are often next in the chain. So after your devs have signed individual commits and those commits are being rolled up into deployable artefacts by the CI system.

                                                                                The CI system can authenticate to Vault as its “CI system identity” to sign those artefacts to mark them as deployable after having passed whatever rigour is appropriate, and the subsequent deployment environments verify the signature upon admission.

                                                                                This tool is one way to do that when the “deployable artefacts” in question are actually just e.g. IaC files on a branch instead of binaries or tarballs. There’s other ways to achieve this chain of custody of course. One simply being tarballing up the branch or copying its HEAD SHA into a file and signing that. Another I’ve read about (but can’t remember from where) is the propagation of dev’s user identity into the CI stage if you’re on the AWS stack, using their SSO and Code* products.

                                                                                1. 2

                                                                                  Ah, so if I understand correctly this would be more likely to be used for signed tags than commits? That makes sense, thanks!

                                                                              1. 4

                                                                                I saw someone using just ‘;’ as a prompt on twitter, apparently cribbed from the rc shell. It means you can copy multiple lines from history and re-execute them on paste.

                                                                                It’s significantly more minimal than my own minimal prompt, and inspiring.

                                                                                1. 3

                                                                                  BTW this tickled a bug, which I fixed. The next release of Oil will be able to run punctual :)

                                                                                  https://github.com/oilshell/oil/issues/853

                                                                                  $ bin/osh
                                                                                  [osh] lisa ~/git/oilshell/oil$ . /home/andy/git/wild/shell-prompt/punctual/punctual.sh 
                                                                                   lisa ▶ true
                                                                                   lisa ▶ false
                                                                                   lisa ▶ 1 ▶ true
                                                                                  
                                                                                  1. 1

                                                                                    Glad it was of some use, thanks!

                                                                                  2. 1

                                                                                    Yeh I’m one of those ‘;’ people these days. I really like the lack of noise as others have called out in this thread. I also colour the ‘;’ red on non-zero exit code (don’t really care for knowing the code value straight away). For dir tracking I use zsh to signal the title of my terminal emulator (currently vterm in Emacs but works anywhere else)

                                                                                  1. 2

                                                                                    I like exa instead of ls. And few more tools I mention here.

                                                                                    1. 3

                                                                                      I really want to like exa, but I get tripped up every single time using exa -t when what I want is ls -t. It’s such a productivity killer.

                                                                                      1. 2

                                                                                        I’d make an alias. I have alias a=‘exa’. Can do specific alias for the -t flag.

                                                                                        1. 2

                                                                                          Yeah. My ls -ltr muscle memory needs to go to exa -lrsold and I don’t have it yet. I have an alias t for a tree like list that is great.

                                                                                          t='exa -l -T -L 2 --header --git-ignore -F -d -I node_modules'
                                                                                          
                                                                                          1. 1

                                                                                            I agree, my muscle memory is a super power and a prison.

                                                                                            This is a decent solution. I could alias it to lt

                                                                                            1. 1

                                                                                              I do lt for the same. Funny how this particular muscle memorised incantation catches so many of us.

                                                                                            2. 1

                                                                                              How is this any better than the fully POSIX compliant tree?

                                                                                              tree -L 2 -C -I node_modules
                                                                                              
                                                                                              1. 2

                                                                                                Non-measurable preference? I have tree too. But to extoll this alias: It reads my .gitignore (if there is one). It has headers. Here is an output example. You can’t see the underlines of the column headings.

                                                                                                 tmp/foo $ exa -l -T -L 2 --header --git-ignore -F -d -I node_modules
                                                                                                Permissions Size User    Date Modified Name
                                                                                                drwxr-xr-x     - you     22 Oct 12:23  ./
                                                                                                .rw-r--r--     0 you     22 Oct 12:23  ├── blech.txt
                                                                                                .rw-r--r--     0 you     22 Oct 12:22  ├── bleep.txt
                                                                                                .rw-r--r--     0 you     22 Oct 12:22  └── bleh.txt
                                                                                                

                                                                                                It understand git. Has some nice other options in the manual.

                                                                                        1. 3

                                                                                          We’ve run into and had to deal with at least a couple additional cases to consider:

                                                                                          1. Terminating a pod triggers various async state changes in the cluster, including propagating network config updates where each host updates their iptables network overlay configurations, load balancer updates, and the like. It’s possible, perhaps likely in some setups, that a pod terminates well before the network changes have converged. When this happens requests may still be forwarded to the now terminated endpoint resulting in failures. Using a preStop hook with a sufficiently long delay works around this. They suggest using a preStop hook for other reasons, and show a ~4 second delay, which may be implicitly sufficient to worth around this on their end.

                                                                                          2. Your service must handle SIGTERM properly. For example, your service may be handling a long running request. If your service tears itself down immediately upon receiving the SIGTERM, these in-flight requests will fail. Instead you’ll need to trap the SIGTERM, wait for in-flight requests to complete, and only then shutdown. Assuming you’ve addressed the networking issues in #1 you do not need to worry about handling new requests arriving after the initial SIGTERM since at that point no new traffic is routed to the pod. Luckily more and more frameworks these days have support for graceful termination. Note that there’s a configurable hard limit, terminationGracePeriodSeconds, that limits the max duration a service is given to complete shutting down cleanly.

                                                                                          1. 4

                                                                                            Great points and I would add that I don’t see enough of this advice out in the wild. Proper use of preStop and handling of signals is absolutely imperative for zero dropped packet deploys in Kube. I expect this catches out organisations the world over.

                                                                                            1. 2

                                                                                              Indeed. That’s the reason for the 2 second delay at the beginning of our preStop

                                                                                              Lots of trial and error!

                                                                                              1. 2

                                                                                                2 seconds ain’t all that bad! We use 15, which works but I’d sure like a deterministic way to know when it’s safe to start terminating (I appreciate the complexity of this as there’s lot of moving parts and even more so at scale). I’ve seen anecdotes that GKE deployments need 30+ seconds… trial and error as you say : )

                                                                                            1. 5

                                                                                              I’ve taken a bit of career detour these last ~5-6 years, finding myself at the helm of teams building out internal platforms for a large (previous gig) then medium enterprise (current gig)—so called “platform engineering” or “technical infrastructure” teams.

                                                                                              It could be Australia being a bit behind the times, but both instances have been when the organisation in question is transitioning from a “Dev throwing over the wall to on-premises Ops” to a “You build, you run (in a cloud)” way of working. So my focus has been on generally introducing self-servicable cloud tech foundations, then tailoring a paved road “platform” atop.

                                                                                              The idea being that should a product engineering team choose to ride (and it is a choice) at the platform rather than the foundational level then a considerable number of cross-cutting concerns are handled on their behalf (security, reliability, observability etc.), allowing them to spend more time on the differentiating business logic.

                                                                                              Anyway, I mention this because in both instances a container orchestration platform has been at the crux of what I’ve built, early on with Rancher 0.x-1.x and then the Kubernetes choochoo train after that, but critically, always with the teams coming in at an abstraction above. That is, I’m of the opinion that Kubernetes should be considered a platform for building platforms—otherwise there are just too many concepts for a product engineering team to grok.

                                                                                              As a concrete example, at the moment we have a singular Kube CRD that hides multiple clusters per environment and distills the myriad of Kube/Istio/ArgoCD/Prom/Vault/OPA (blah blah…) resources down to a small easy-to-reason-about config (averaging about ~30 lines of YAML). Apart from the low cognitive load for teams, it also allows us to manage underlying tech upgrades or introduce new features at our pace.

                                                                                              I realise this might sound over-engineered, turtles all the way down, or science project like… so figure it is worth mentioning a few things:

                                                                                              • Both these orgs were/are operating (micro-)service ecosystems > 150 services, acutely, with private on-premises connectivity requirements as well. I feel that’s when this sort of platform atop a cloud starts to shine. I don’t discourage our teams from reaching for cloud FaaS or the likes, but the inter-connectivity with the rest of the ecosystem does start to break down a little. Especially when met with our security/risk/compliance posture (bank). A common pattern I see is a brazen team going foundational with gusto then jumping up to the paved road after learning some things the hard way. YMMV but I would go as far as saying Kube is not suited for a relatively homogeneous, single tenant/team/startup/static type environment.

                                                                                              • As someone else mentioned in this thread, definitely don’t operate Kube yourself if you can avoid it. Outsource as much of this stuff to SaaS as possible.

                                                                                              • It is imperative for these platform engineering teams to act like internal product teams that are both in completely tune with the user needs, and also comfortable with trying to automate/outsource their way out of a job. It doesn’t personally fill me with joy to be building a platform abstraction on top of a cloud but unless you can wire those constituent cloud services together in a way that suits your org, it is where we are.

                                                                                              (This reply getting on a bit. Would be glad to talk more about this stuff in DM if you like.)

                                                                                              1. 1

                                                                                                I should try Nix on Mac again; I took against it a while ago for … reasons? … and I do love it on the Linux machine in the basement. I fear it would be fighting the platform too much on Darwin, but it can’t hurt to try.

                                                                                                1. 2

                                                                                                  I have used it quite successfully on MacOS. I do only use it for per project shells though, I still install all “system level” applications with Brew.

                                                                                                  1. 1

                                                                                                    I am in a kinda inbetween phase: I use home-manager, and use nix as a “primary” package manager. Meaning, I first try to install stuff via nix; but when something is not available in nixpkgs, I resort to brew. Which seems to be much more often than I’d like, unfortunately… :(

                                                                                                    Didn’t try nix-darwin yet; I feel not brave enough yet…?

                                                                                                    1. 3

                                                                                                      Curious, any recent examples come to mind where you had to resort to brew instead of nixpkgs?

                                                                                                      Did you open a package request on the nixpkgs repo?

                                                                                                      1. 2

                                                                                                        Hm, haven’t installed anything over last 3+ months so don’t remember now; but I’m starting a new job just now, so I can try and collect a list for you if something shows up - and if I remember :/ (will try to put up a todo for that).

                                                                                                        Didn’t request, I don’t feel like bothering people, esp. as I seem to believe the Nixpkgs maintainers tend to be overburdened anyway - or has that changed since a few years ago?

                                                                                                        edit: Also, sometimes I’m trying stuff and not yet sure if I won’t uninstall, so if it’s not on nixpkgs, should I ask someone to maintain it just for the sake of trying and uninstalling?

                                                                                                        1. 2

                                                                                                          There’s thousands of contributors and package maintainers. I wouldn’t say you should open a package request for something you’re just trying out on a whim, but if it’s something you actually care about and would be willing to commit some time helping through testing and such then sure.

                                                                                                        2. 1

                                                                                                          One example I had recently was Karabiner-Elements. I wasn’t sure how to handle the use of Darwin kernel extensions.

                                                                                                    2. 2

                                                                                                      I’m a fan of Nix and use it across both NixOS and macOS.

                                                                                                      I find nix-darwin is useful for configuring the macOS desktop experience, and between nixpkgs and home-manager you can get quite far towards a declarative macOS setup. I only reach for Homebrew to install some macOS UI applications (so casks, really) that I can’t easily Nix-ify, and I still manage the Brewfile using Nix (and a special casks list type that allows me to declare the cask near the respective Linux pkgs statement).

                                                                                                      I recently updated my dotfiles to build both platforms using CI on push to ensure I haven’t broken either with a change. I also store the resultant binaries in Cachix which often saves me needing to locally build.

                                                                                                    1. 1

                                                                                                      The Control key moved to the Caps Lock keys usual position. […] I can’t imagine why anyone thought Caps Lock should have had this prominent of placement.

                                                                                                      That’a one of the first things I do on new system installs. Also, most X11/xkb configs allow mapping left + right Shift to act as a Caps Lock (which is IMO more convenient and intuitive than a dedicated key).

                                                                                                      Section "InputClass"
                                                                                                      	Identifier "Keyboards"
                                                                                                      	MatchIsKeyboard "yes"
                                                                                                      	Option "XkbLayout" "us,ru"
                                                                                                      	Option "XkbVariant" "mac,mac"
                                                                                                      	Option "XkbOptions" "grp:rwin_toggle,ctrl:nocaps,shift:both_capslock"
                                                                                                      EndSection
                                                                                                      

                                                                                                      Note: grp:rwin_toggle switches layouts using right meta key.

                                                                                                      1. 3

                                                                                                        I always throw this into my .xprofile:

                                                                                                        setxkbmap -option caps:swapescape
                                                                                                        

                                                                                                        That swaps the esc and caps keys. Very handy for vim.

                                                                                                        1. 3

                                                                                                          I’ve always mapped caps to ctrl, but recently have been forced to use a MacBook (touchbar) at work. As a user of vi bindings everywhere I can, the lack of a tactile escape key was too much to bear so I’m using Karabriner Elements to use caps as ctrl when held and esc when pressed alone.

                                                                                                          I have to say that there’s no going back after this switch—total game changer for me. Of course, I now can no longer use anyone else’s machine.

                                                                                                          The same can be achieved in X.org with setxkbmap as above for your caps/control switch, combined with xcape for the esc tap feature.

                                                                                                        2. 1

                                                                                                          YEEESSSSS! You can also remap caps lock to control on macOS under Preferences -> Keyboards -> Modifier Keys… or on Windows via registry edits.