Threads for johnaj

  1. 2

    Emacs shows matching parentheses by default

    It seems that show-paren-mode no longer has to be enabled?

    If so, that’s great. I’m trying to use most tools with as many defaults as possible.

    After 20 years on Emacs, my .emacs / init.el is very small, just a few use-package invocations.

    I wish make-backup-files defaulted to nil or that Emacs shipped with some mode that flipped those few variables that many people alter, such as that one, to more modern defaults.

    1. 1

      I agree that the tilde backup files are annoying, but they have saved my ass a few times. I think they are turned off when using a version-control mode and the file is under version control.

      1. 7

        I used to turn them off as well, until I discovered backup-directory-alist.

        (setq backup-directory-alist '(("." . "~/.cache/emacs/backup")))
        

        Now they don’t clutter my directories, but they’re still available when I need them.

        1. 1

          I’m not on 28 yet, but Emacs (with vc-mode, 99% defaults) does create the backup files even for files in version control for me. I’ve sort of learned to ignore them mentally.

          1. 1

            In my Emacs 27.2, vc.el has this commented-out code:

               ;; (unless vc-make-backup-files
               ;;   (make-local-variable 'backup-inhibited)
               ;;   (setq backup-inhibited t))
            

            …with a preceding comment claiming that this is somehow wrong. I must have learned this behavior on an earlier version of Emacs where that code was still active. Now I’m using Magit and I do have backup-inhibited set in version-controlled files, so I presume Magit is doing that.

            1. 1

              I just checked on emacs -Q (27.2) and backup-inhibited is still t on my version-controlled files. A quick (rip)grep didn’t find anything relevant though, but something must be setting it, and it’s not Magit.

        2. 1

          I honestly expect that a lot of old…experienced Emacs users will complain about this, as with most other changes to the old defaults that’s been done.

          But I agree, it’s great that it’s on by default now! One thing less to enable as a standard.

          1. 10

            As someone whose .emacs file is older than some of my junior devs, I say attack those defaults with a chainsaw. I can fix the defaults I don’t like, but new devs can’t, so the opinions of my fellow foggies shouldn’t count when making these changes.

            1. 2

              I thought that the guiding philosophy for improvements to the out-of-box defaults is that the old settings are still available there and experienced users will know how to revert to them if they wish (whereas new users may have more trouble going the other way). And as an Emacsen user for nearly [mumble!?] years now, I’m perfectly fine with that!

          1. 10

            This looks interesting. On a related note, if you want something closer to vi, but the lack of UTF-8 and bidirectional text in OpenVi is a problem, take a look at neatvi.

            1. 9

              I really enjoy everything the author of neatvi has written (neatroff especially). I reached out to him once via email basically to tell him I was a fan. He was very gracious.

              1. 3

                He was very gracious.

                Agreed. When neatvi was relatively new, I submitted a small number of patches and had several questions. He was always helpful.

              2. 5

                For UTF-8, another option is nvi. OpenBSD’s vi is actually an old version of nvi that lacks UTF-8 support.

                1. 3

                  Don’t confuse OpenVi/OpenBSD-vi, nvi1, and nvi2. These are all different programs that share the same heritage.

                  OpenVi is derived from OpenBSD vi, which derives from nvi version 1.79, released in 1996. There has been 25+ years of independent development as part of the OpenBSD base system and it has diverged greatly in that time, with the development going in a different direction.

                  Nvi1, currently on version 1.8x, is maintained at https://repo.or.cz/nvi.git - I believe the latest version of this editor does have multibyte support, but this is not the OpenVi/OpenBSD version of the editor.

                  Nvi2 shares the same heritage as well, but is also quite far removed from 1996 code. It is actively maintained at https://github.com/lichray/nvi2 and also includes multibyte support.

                  (If I remember correctly) the multibyte support in both Nvi1 and Nvi2 derives from nvi-m17n, developed as part of the KAME project by the late itojun - http://www.itojun.org/itojun.html … the last update to nvi-m17n was about 3 years ago, and is available at https://cgit.freebsd.org/ports/tree/editors/nvi-m17n/files

                  Currently, optimizing for size using link-time garbage collection with GCC 11.2 on an x86_64 glibc Linux system gives a good idea of the changes over time and the different direction these editors have taken. OpenVi is also simplified in structure and does not have the three levels of abstraction of Nvi 1.8x - there is no library interface layer.

                  For OpenVi, the compiled binary is 278K, and for Nvi1 (nvi-1.81.6-45-g864873d3) the compiled binary is 528K (36K for vi, 528K for libvi).

                  OpenVi has a single configuration standard with no dependencies beyond curses.

                  Nvi1 has many options beyond trace/debug (“widechar” “gtk” “motif” “threads” “perl” “tcl” “db3/4” “internal-re”) - so at least 255 different build variations are possible.

                  (I’ve not yet built Nvi2 myself on Linux so I can provide an actually fair comparison yet, but I will, and I’ll summarize the data in an FAQ section of the README)

                  1. 2

                    (Note that I was using the defaults here, I’m sure that it’s possible to trim down Nvi 1.8x further, but I’m comparing the default compilations, optimized for size (GCC, -Os, -fdata-sections, -ffunction-sections, link-time GC enabled), but Nvi 1.8x is a much more complicated program, and has a different feature set, and different supported options.

                    1. 1

                      Well, I allowed myself to omit the fact that OpenBSD’s vi has seen some independent development past nvi 1.79, which is true. A “(based on)” should be inserted before “an old version” in my original comment. But I appreciate the thorough summary of nvi versions!

                    2. 2

                      Nope, the vi in OpenBSD is nvi - you’re confusing it with nvi2. Both are in active development: nvi and nvi2.

                      1. 2

                        It should be noted that DragonFly BSD has imported nvi2, but with some modifications as well.

                        It’s unfortunate there is so much confusion surrounding the various nvi-based editors, mostly due to them all being so similarly named.

                        Part of why I chose to call this project OpenVi was because the name was - suprisingly - available, and does not directly imply that OpenVi is exactly Nvi1/2 or OpenBSD’s vi.

                        (In particular, all bugs in OpenVi should be considered my fault.)

                    3. 2

                      I will confirm that Neatvi is an excellent project, but I’m a bit more interested in Nextvi - https://github.com/kyx0r/nextvi - the RTL/bidi in Neatvi is a huge strength and is done very cleanly when compared to other vi-likes

                    1. 24

                      tl;dr

                      Someone who does not like IDEs, tries out github copilot and finds something they don’t like. They use this to support several notions including the idea that IDEs cause complex code, and that being dependent on tools is bad. (Says the person using a computer.)

                      I haven’t tried out github copilot, but I code for a living and am grateful for all the automation I can get.

                      1. 26

                        I think this is an unfair characterization of the post. IMO the main line of thought – that IDEs guide the programmer in a specific and limited way – is perfectly reasonable. The anecdote about a non-IDE programmer producing very different code from IDE programmers is thought-provoking. The observation about Java being language where the complexity is out of the programmer’s hands, so to speak, is interesting. And I found the post as a whole to be well-written.

                        1. 7

                          I too, dislike IDEs, mainly because I don’t want to learn yet another editor [1], but over the years, I’ve also found out they refuse to deal with the code I wrote [2]. Or they crash. Hard. After a few minutes of use [3]. Over the three decades of attempts, I’m at the point where I pretty much ignore IDEs these days as just not worth it.

                          I also wrote about this in general: When does technology pass from being a tool to being a crutch?

                          [1] I recall using Turbo Pascal 3, which technically was an IDE. I hated using it because the editor used different key bindings from what I was used to, and it couldn’t be changed. Yes, I am seriously dating myself here.

                          [2] One of the first Java IDEs back in 1997. The authors of that piece of crap didn’t expect anyone to write their own Java layout manager for applets.

                          [3] I’m not even talking about ancient history here, but recently. Some popular IDE that claimed to be for C/C++ but was lying because it assumed you were always using C++. I don’t. I use C. And class is a perfectly cromulent C identifier. Yes, I know. I don’t care. I’m writing C code.

                        1. 3

                          1600x1200 :-)

                          But on a more serious note, I think my resolution is not to let the perfect be the enemy of the good. There are a few things that I tend to either do properly or not do at all. One example is cooking and eating well. It’s great if I can do it consistently, but I usually don’t have the energy or time, so instead I end up not trying at all, which is worse than doing it inconsistently.

                          1. 6

                            Serious question for most of these starter configs: How does inhibiting the startup screen without replacing it help anyone?

                            On the startup screen you are given hyperlinks to a guided tour and tutorial, which imo is probably the best thing you can provide new users. Setting them up with a blank screen is counterproductive.

                            1. 8

                              It’s slow and scary, I love the blank notepad-like familiar screen. Not arguing for inhibit to be the default but I was happy to find it, one of the first options I set.

                              1. 1

                                It’s slow and scary? Please elaborate because it takes the same amount of keystrokes to exit the startup screen as it does a blank one.

                                1. 3

                                  I don’t think anyone suggests replacing the splash screen with nothing?

                                  Setting inhibit-splash-screen makes it drop you into a scratch buffer where you can enter elisp directly, which is useful on its own. In my own config I have it drop me into an eshell buffer, which IMO is more useful than scratch, but either one is more useful than the splash screen for people who already know how to open the documentation.

                                  1. 2

                                    for people who already know how to open the documentation.

                                    I think this is the important bit here and it’s people who don’t know how to open the documentation just yet that I find it most useful for.

                                    After learning a bit I’m all for users replacing the splash, but I also worry about the user who is inclined to copy a config that’s branded as sane or better defaults before they get a good grasp of what they’re using. It’s something that happens a lot more in practice than I’m comfortable with in a midly popular emacs community that I’m a part of.

                                    1. 1

                                      In my own config I have it drop me into an eshell buffer

                                      Pray tell! This is relevant to my interests. I went looking for your config to try and steal that bit, but couldn’t find it.

                                      1. 3

                                        I just put a call to (eshell) in my init.el file; nothing much to it.

                                        https://git.sr.ht/~technomancy/dotfiles/tree/master/item/.emacs.d/init.el

                                        1. 2

                                          Part of the problem is you went looking for his config instead of an answer

                                          Try this

                                          (add-hook 'emacs-startup-hook 'eshell)

                                          Not sure if it handles edge cases but I think you will get more value out of figuring out why than copying from someone else’s config wholesale.

                                          1. 1

                                            Yeah, I was lazy. I ended up skimming the docs for the initialisation section and came up with this, which appears to work:

                                            (setq initial-buffer-choice 'eshell)

                                            Edit: it doesn’t work, at least not fully. I do get a nice Eshell on startup, but for some reason Magit can’t find emacsclient:

                                            There was a problem with the editor '/run/current-system/sw/bin/emacsclient --socket-name=/var/folders/mn/y08j0jvs2cv95bthlsgsm9sr0000gn/T/emacs501/server'

                                            Edit2: that issue seems to be a problem with with-editor.el, but I have to do some more debugging to rule out an issue with other parts of my config.

                                      2. 2

                                        Oh, it does?! Ok, then maybe that’s cool. I haven’t had the start up screen uninhibited since the 90s. Maybe you are right. Back then it’d launch you into a really bad tutorial, all focused on C-n, C-b (arrow key basics) and not on stuff like describe-key, describe-mode, describe-function.

                                    2. 6

                                      Even worse is that most starter configs, including this one, disable the menu bar – which is the most helpful feature for new Emacs users!

                                      How, as a new user, are you supposed to know that C-x C-f means “open file”? By far, the quickest, simplest and most intuitive way is to open the File menu and check the corresponding keyboard shortcut there.

                                      1. 3

                                        The menu bar is useful for about your first hour or two.

                                        As soon as you learn how to describe then you need to let go of the menu bar; people assume all the commands will be visible there but it’s just too limited. Most commands aren’t in the menu bar; it’s the wrong place to look for things.

                                        1. 6

                                          The menu bar may be the wrong place to look for things but it’s far easier to navigate than the output of any describe command called on a buffer with non-trivial modes. Commands are organised in sub-menus of a few items rather than in a flat list organised alphabetically by prefix and binding, and oftentimes you really don’t need all commands, just the important ones. Self-documenting features like describes are great if you already know what you’re looking for, but sometimes you don’t. It’s also useful for commands that are accessed so infrequently that you don’t want to bind them to anything, or don’t want to remember the bindings. I’m not gonna bind “Manage Emacs packages” to anything, retrofitting a Space Cadet keyboard on my laptop is just not worth the effort.

                                          At the risk of embarrassing myself in front of the crustacean nerd audience by looking like a puny normie, I’ll gladly admit that I’ve used the Emacs menu bar for like twenty years now and I have no intention to stop. I turn it off by default because I generally use darker themes, and sometimes (coughon Windowscough) the menu bar burns my retina, but I have `menu-bar-mode`` bound to F10 so I can toggle it on. Sue me.

                                          (Or revoke my Church of Emacs membership card, I guess? :-))

                                          Edit: also, the target audience of this 72-line (well…) config really seems to be people who have used Emacs for 1-2 minutes, so maybe it’s worh keeping the menu bar on for the remaining 118 minutes or so :-).

                                          1. 2

                                            I don’t actually use describe very much… what I think is the most useful thing for new-to-intermediate users is:

                                            1. Treat M-x like you treat the Command Palette on a normie editor (those are basically inspired by M-x anyway).
                                            2. Use a reasonable search and completion framework for the minibuffer, preferably with marginalia. I use selectrum + hotfuzz + prescient, but I’m happy to start seeing good solutions show up in core and/or GNU Elpa, like vertico.

                                            I do think absolutely new users should leave the menu-bar and scroll-bars on until they have learned what the better alternatives to those are.

                                          2. 2

                                            Even worse is that most starter configs, including this one, disable the menu bar – which is the most helpful feature for new Emacs users!

                                            How, as a new user, are you supposed to know that C-x C-f means “open file”? By far, the quickest, simplest and most intuitive way is to open the File menu and check the corresponding keyboard shortcut there.

                                            But how am I supposed to look cool if I have a menu bar? It wouldn’t look like an xterm otherwise.

                                            1. 2

                                              Your Emacs looks like an xterm? Mind sharing your config? /s

                                        1. 3

                                          I have also considered this… The only problem is that you need to keep the text and the markup in sync. Have you found this to be amnoying?

                                          1. 1

                                            Have you found this to be annoying?

                                            Not yet. Perhaps the downside is balanced by the upside of paying more attention to each markup. But to be fair I think I need a few more months of personal usage data. It may be annoying.

                                          1. 4

                                            I feel the value-add of CPAN is slightly disingenuous when I bet a majority of those packages are no longer maintained (or work, considering how APIs for services can change), and other ecosystems have leapfrogged it.

                                            1. 1

                                              Yeah, CPAN ain’t what it used to be, and for “package-driven” development, Python is probably the way to go nowadays.

                                              On the other hand, I find that the diminishment in value due to unmaintained API-dependent packages is compensated by the volume of packages that change and augment the language in really powerful ways. See e.g. Keyword::Declare (and the accompanying presentation).

                                              1. 1

                                                I’m one of those people who actually tries very hard to write Perl scripts that don’t have CPAN dependencies, even if I end up reinventing the wheel now and then.

                                              1. 2

                                                This is precisely what I’ve been wishing for – great job! I tried the bookmark and it seems to work perfectly.

                                                1. 10

                                                  Once we move beyond one-liners, a natural question is why. As in ‘Why not use Python? Isn’t it good at this type of thing?’

                                                  The reasons provided are fine, but for me the main reason is speed. AWK is much, much faster than Python for “line at a time” processing. When you have large files, the difference becomes clear. (perl -p can be a reasonable substitute.)

                                                  Once you are writing long AWK programs, though, it’s time to consider Python or something else. AWK isn’t very fun once data manipulation gets complicated.

                                                  1. 4

                                                    (perl -p can be a reasonable substitute.)

                                                    +1. In my eyes, it’s Awk and then Perl. Perl turns out to be much better for these purposes than other scripting languages. The difference in startup time between Perl and Python is very significant. If you don’t use (m)any modules, Perl scripts usually start just as quickly as Awk scripts.

                                                    1. 2

                                                      I’m sure that’s true for some kinds of scripts, but that doesn’t match my experience/benchmarks here (Python is somewhat faster than AWK for this case of counting unique words). For what programs did you find AWK “much, much faster”? I can imagine very small datasets being faster in AWK because it’s startup time is 3ms compared to Python’s 20ms.

                                                      1. 2

                                                        For what programs did you find AWK “much, much faster”?

                                                        Any time the input file is big. As in hundreds of MGs big.

                                                        I used to have to process 2GB+ of CSV on a regular basis and the AWK version was easily 5x faster than the Python version.

                                                        1. 1

                                                          Was the Python version streaming, or did it read the whole file in at once?

                                                          1. 1

                                                            Streaming.

                                                        2. 2

                                                          Regarding your results, 3.55 under awk is with or without -b?

                                                          I get 1.774s (simple) and 1.136s (optimized) for Python. For simple awk, I get 2.552s (without -b) 1.537s (with -b). For optimized, I get 2.091s and 1.435s respectively. I’m using gawk here, mawk is of course faster.

                                                          Also, I’ve noticed that awk does poorly when there are large number of dictionary keys. If you are doing field based decisions, awk is likely to be much faster. I tried printing first field of each line (removed empty lines from your test file since line.split()[0] gives error for empty lines). I got 0.583s for Python compared to 0.176s (without -b) and 0.158s (with -b)

                                                          1. 4

                                                            Also, I’ve noticed that awk does poorly when there are large number of dictionary keys.

                                                            Same here. If you are making extensive use of arrays, then AWK may not be the best tool.

                                                        3. 2

                                                          Once you are writing long AWK programs, though, it’s time to consider Python or something else. AWK isn’t very fun once data manipulation gets complicated.

                                                          I dunno, I think it’s pretty fun.

                                                          I am consistently surprised that there aren’t more tools that support AWK-style “record oriented programming” (since a record need not be a line, if you change the record separator). I found this for Go, but that’s about it. This style of data interpretation comes up pretty often in my experience. I feel like as great as AWK is, we could do better - for example, what about something like AWK that can read directly from CSV (with proper support for quoting), assigning each row to a record, and perhaps with more natural support for headers.

                                                          1. 2

                                                            You are right. Recently I was mixing AWK and Python in a way that AWK was producing key,value output easily readable and processed later by Python script. Nice, simple and quick to develop.

                                                          1. 12

                                                            Yes, cat | grep is a “useless use of cat”, but it allows me to quickly change the grep regex, which is much more annoying if I have to navigate past the filename first.

                                                            1. 8

                                                              I agree, and I think it’s kind of stupid to try to avoid cat outside of scripts, but that said, there is a way to solve this specific problem:

                                                              <file grep regex
                                                              
                                                              1. 1

                                                                That’s cool. I think I’ve read about this shell syntax before in man pages but never saw the use for it. Now I do!

                                                                1. 1

                                                                  I’ve always liked that syntax, especially if you have a redirection at the end of a follow-on pipeline (<foo | bar >baz ) but it can break in bash with some shell constructs in the pipeline (some uses of while)

                                                                  1. 2

                                                                    I think you mean

                                                                    <foo bar | baz >qux
                                                                    

                                                                    I too find this pattern useful, specifically for troff pipelines.

                                                                2. 4

                                                                  “Useless use of cat” is usually bad advice, anyhow.

                                                                  As far as I can tell, it only matters for performance-sensitive scripts. If someone’s just hacking around on the command line, cat ... | ... is extremely useful since it allows quick switching out of inputs. Let’s say I’ve got some_expensive_script.sh | ... – better to capture the output to a file, run cat output.txt | ... and build up the pipeline until it works, then swap back in the expensive script call when I’m ready.

                                                                  (Also, I very often end up accidentally writing < output.txt | grep ... because I forget to delete the pipe, and so grep gets nothing! This leads to a great deal of frustrated debugging until I notice it. Best to just keep it a pipeline while developing rather than adhering to some random internet person’s notions of terminal purity.)

                                                                1. 2

                                                                  At first glance, this seems impossible. But with global warming and the environment in mind, I do wonder if it wouldn’t solve many our problems. It wouldn’t be be economically sane to offer infinite software updates for a one-time purchase, but for a recurring fee? That seems a lot more viable. Just sell the updates instead of giving them away for free. It is not even unthinkable, considering the massive movement towards subscription services that has been going on in the software world.

                                                                  1. 2

                                                                    I don’t think this kind of law would solve any environmental problems. The environmental impact of a piece of consumer electronics like a PSP is that it took some amount of energy to build it in a factory (and build all of its inputs, transitively, until you get to the energy cost of mining them out of the ground), and took some amount of energy to ship it from the factory to the consumer. Exactly like every other physical good that people use - there’s nothing special about consumer electronics.

                                                                    The chain of reasoning that someone might perform to conclude that a law requiring perpetual software support of consumer hardware devices would solve the environmental problem of global warming is something like: a law would force manufacturers to provide software support for a longer period of time -> consumers would use their devices for longer than they currently do in the absence of such a law -> consumers would be less prone to buying new devices -> this would reduce demand at electronics factories to build and sell new consumer electronic devices -> they would use less energy as a production input -> less energy implies less burning of fossil fuels for energy -> less CO2 emitted into the atmosphere.

                                                                    I can see multiple problems with this chain of reasoning; to pick one in particular, I don’t think people actually do buy new electronic devices solely because their old ones no longer receive software updates. I think people buy new electronic devices because the state of the art of electronics devices is actually advancing, and people want to be able to do new things that new devices let them do and old devices can’t. A PSP that still received software updates doesn’t actually replace something like a Nintendo Switch - I can’t use a PSP to play Breath of the Wild or Metroid Prime 4, and if economic conditions compelled video game manufacturers to design modern games like these for the PSP’s hardware, because it was prohibitively expensive to sell new hardware, they would likely be inferior games.

                                                                    Something that people who are concerned about the environmental impact of human economic activity don’t think about clearly is that the environmental benefit of avoiding energy/resource use in a particular economic production process only happens if it is prohibitively expensive to use that particular energy or resources. A world where everyone was still playing video games on a PSP because after 2008 it became prohibitively expensive for companies to build a new video game system, would be one where everyone’s material standard of living was lower because all sorts of electronics made with similar manufacturing techniques would also be prohibitively expensive. If the cheapest smartphones cost $50,000, there would be way fewer smartphones manufactured because way fewer people could afford them at that price, and a billion people in the 3rd would would just not be accessing the internet.

                                                                    1. 2

                                                                      I don’t think people actually do buy new electronic devices solely because their old ones no longer receive software updates. I think people buy new electronic devices because the state of the art of electronics devices is actually advancing

                                                                      Well, the size of the set of people who upgrade because a device improved is not empty, but is the size of the set of people who upgrade due to needless obsolescence fully empty? Two things can both be upgrade reasons to different people. There’s an environmental win if anybody stops upgrading, not necessarily everybody.

                                                                      I’m typing this from a 2013 era desktop that I could easily afford to replace. I don’t because it works, it’s updated, replacing it seems like effort, and I’m just too lazy. So I don’t think it’s true that I’d buy new electronics unless it’s prohibitively expensive - I just need moving to a new thing to be less convenient than using the old thing, which tends to favor the old thing unless the broader ecosystem actively rejects its ongoing use.

                                                                      1. 4

                                                                        The other part of this if were specifically talking about environmental impact is spare parts. Not just in the right-to-repair sense, but how long should Sony be required to keep the manufacturing line for PSP replacement batteries running? LiPos have a bad shelf life in general; they can’t really just produce 15 years worth of spares and stuff them in a warehouse somewhere because 15 years later they won’t be any good.

                                                                        1. 2

                                                                          But your new desktop allows you to use all the things modern desktops do (which is also why Android 2.3 has endured such a long time). Imagine a world where most modern software were not available for your machine. Sure, you get software updates for your 2013era Firefox 18 but nothing new is coming out, since all machines have moved to, say, RISC-V since and support for x86 was dropped entirely. This is the situation with the PSP.

                                                                          Yes sure, I have an old MacPro 2010 that upgraded with a halfway decent graphics card makes it a power-hungry but feasible Steam machine for the kind of gaming I do, but this is because post-2010 software still runs on it. If I had been limited to software from that era it would be way less useful and its main use would be to be carted around to vintage computer festivals to show off how well it runs Hypercard or so.

                                                                          1. 1

                                                                            The fact that this is feasible for you reflects the fact that in many ways desktop PC technology has actually stopped getting better, at least meaningfully better, and the time where it topped out was very roughly around 2010. If the computer you had was of 2008-vintage rather than 2013 (just five years older), there’s a much better chance that you’d be interested in upgrading - you’d see that many things people can do with modern desktop computers, such as visiting modern websites, don’t work very well on that hardware, you’d be much less likely to have a SSD, which really did represent a noticeable performance improvement (and which uses resources and energy to make, just like a PSP or any other piece of electronics).

                                                                            PCs themselves don’t generally have software updates in and of themselves anyway; rather it’s the operating systems on them that do (and various specialized components like the CPU or SSD might have their own separate and less routine firmware update process).

                                                                            If you ran modern Windows on your PC, you would still get updates for as long as Microsoft supported it - so if you ran Windows XP on that machine you’d be out of luck, but if it was Windows 7 you might still be ok. Of course eventually Microsoft will stop supporting Windows 7, but will support some new version of Windows that you can buy (will such a law make it illegal for Microsoft to stop supporting a version of Windows? Force them to start supporting Windows XP again? Windows 95?). Perhaps Microsoft might want to stop supporting hardware configurations that don’t have any USB 3 ports, perhaps by only making it possible to actually install the OS over USB 3. USB 3.1 was released as a standard in 2013, so it’s unlikely that your computer has it (although you could still add it via a PCI card - another electronic component that uses energy and resources to manufature! - if necessary). Maybe the law would have to have a provision that made it illegal not to require OS manufacturers in the mid-2020s to assume that USB 3 exists…

                                                                            Of course if you ran Linux on that machine, it would be supported for longer. But even the Linux kernel dropped support for i386 machines as of kernel version 3.8 - with Linus Torvalds’ full blessing. Dropping software support for 25 year old hardware doesn’t sound like a forever software update to me - maybe the Linux foundation would need to be sued or criminally charged under this law?

                                                                            1. 1

                                                                              If the computer you had was of 2008-vintage rather than 2013 (just five years older), there’s a much better chance that you’d be interested in upgrading

                                                                              As luck would have it, I read your comment from (and am replying from) a 2007 MacBook.

                                                                              you’d see that many things people can do with modern desktop computers, such as visiting modern websites,

                                                                              That’s true, but not because of hardware. This 2007 MacBook is a great example of the lack of software updates forcing unnecessary obsolescence - the newest OS X it supports is 10.7, meaning it can’t run a modern browser and can’t browse a lot of sites. The hardware is fine. It had a user serviceable disk, RAM, and battery, so it’s had an SSD for a long time and its third battery holds a charge fairly well. From an environmental point of view, upgrading to an SSD seems less damaging than replacing the entire device.

                                                                              But to be clear though, I was just taking issue with your comment that the only way to avoid rapid hardware upgrades is to dramatically raise cost. That logic sort of implies that the instant consumers get money we go buy electronics, so if they cost more it’d take longer to get that money and we’d buy less. Tech incomes though mean most of us on this site can afford to buy new hardware tomorrow, so there must be a reason we don’t that’s not about money.

                                                                              Updates “forever” seems unrealistic and I never meant to imply that it should happen. It’s still a valid question though whether it’s appropriate to end software updates when a majority of devices manufactured of a particular model are still in active use. When that occurs, it strongly suggests software updates are being used to drive hardware sales when users are otherwise happy with the hardware.

                                                                          2. 2

                                                                            Something that people who are concerned about the environmental impact of human economic activity don’t think about clearly is that the environmental benefit of avoiding energy/resource use in a particular economic production process only happens if it is prohibitively expensive to use that particular energy or resources.

                                                                            This is a bit of a misrepresentation. All of the climate activism I have been to recently has emphasised the importance of economic and climate justice taking place at the same time. The point is to redistribute resources and to globally reduce emissions. For some this will mean getting by with less. For many more this would mean greater access to resources.

                                                                            More accurate carbon pricing on products doesn’t have to mean that people who are currently poor have to do without, but for that to happen there will have to be massive redistribution through aid and increased wages for poorer people all over the world (which will mean increased product prices on many items that are currently cheap only because human time is valued so differently in the global south versus the north, though automation and efficiencies will also become more competitive as labour becomes reasonably priced and that can help here).

                                                                            1. 1

                                                                              This is a bit of a misrepresentation. All of the climate activism I have been to recently has emphasised the importance of economic and climate justice taking place at the same time. The point is to redistribute resources and to globally reduce emissions. For some this will mean getting by with less. For many more this would mean greater access to resources.

                                                                              Unless a specific anti-carbon-emission scheme entails literally every human being in the world getting by with less resources whose production entails emitting CO2, it won’t actually work for the purpose of reducing CO2 emissions. What activists claim in the course of performing activism bears little relationship to what would actually happen in a world where a given policy actually exists and people make economic decisions in response to it.

                                                                              More accurate carbon pricing on products doesn’t have to mean that people who are currently poor have to do without, but for that to happen there will have to be massive redistribution through aid and increased wages for poorer people all over the world (which will mean increased product prices on many items that are currently cheap only because human time is valued so differently in the global south versus the north, though automation and efficiencies will also become more competitive as labour becomes reasonably priced and that can help here).

                                                                              It does mean that people who are currently poor (along with everyone else) has to do without, because “doing without” is the actual mechanism by which CO2 is prevented from being emitted into the atmosphere.

                                                                              1. 1

                                                                                Less CO2 can be emitted at the same time as the carbon budget for most people increasing because of the massive inequality in global carbon emissions. See e.g. https://ourworldindata.org/grapher/consumption-co2-per-capita

                                                                                1. 1

                                                                                  This might be true; there’s a lot of economic processes producing goods and services that involve emitting CO2 besides consumer electronics manufacturing. Maybe flying planes turns out to dominate CO2 emission linked to economic activity and making it illegal to run an airline would reduce aggregate CO2 emissions more than everyone on earth being able to afford a cell phone. The point remains that if the specific thing you’re trying to reduce is resource consumption associated with inputs to making consumer electronics, the only way to do this is to make consumer electronics more expensive, so fewer people can afford to buy one, so fewer get physically made (making something illegal is one way of making it expensive - you could imagine a law saying that a person could only legally own one of a cell phone or a game console, and that would engender a black market in unregulated cell phones/game consoles, which would cost more money because of the illegality).

                                                                        1. 3

                                                                          I find it interesting how many of these articles mention i3. I see the appeal in theory, but I find myself unable to effectively use these minimalist setups. I absolutely love the idea of vim but I can’t use more than three or four commands.

                                                                          There seems to be some kind of indirection or memorization my brain can’t cope with. Which leads into some interesting observations of how my broken brain successful compensates for many things but not this.

                                                                          My short term memory immediately drops everything if I need to think about what order to do keystrokes in. If I try to look at a line number and use it in a command, I forget the number or the command. If I try to learn two commands one after the other, I mix them up. Sometimes figuring out what key to hit will cause me to forget where the key is on the keyboard.

                                                                          I don’t have this issue when working in a “regular” window manager because I can use spatial recognition, a part of my brain that actually works. For some odd reason, I can use the same lookup table my written vocabulary lives in to store keyboard shortcuts, but only for ones that do a single “action”. You’d think I could learn vim this way, but I find this memory inaccessible when working in a terminal. Probably because my brain can’t both work on writing commands and remembering commands at the same time. When using a mouse or the arrow keys, I’m not “writing”.

                                                                          Interestingly, when the arrow keys are all in the same row, I can’t use them. I forget which one is up and which one is down. No issues using WASD in games.

                                                                          I often wonder what it would feel like to use a more minimalist setup. :)

                                                                          1. 4

                                                                            I do use vi as my main editor, but I totally agree with you: much of it requires a lot of cognitive energy. I mostly use hjklweb{}dy, the occasional [[, ]], / and :s. Things like t and f (go to the next occurrence of a letter in the line) often require way too much thinking than a couple of w’s and h/l’s. I think people overestimate how useful the more advanced functionality of vi is.

                                                                            I generally feel the same way about tiling and keyboard-driven window managers. There’s no way a keyboard shortcut for resizing a window is easier than using the mouse.

                                                                            1. 3

                                                                              I think a lot of the time (and tiling wm experts correct me if I’m wrong here), a tiling window manager is putting stuff in a position for you that you were already expecting (based on your familiarity with which keyboard command is needed to place a new window where you want it, and split/move panels horizontally/vertically and stuff like that). It will definitely be faster than taking your hand up from the keyboard, repositioning the mouse cursor to the window title-bar, dragging to new location, letting go, and bringing your hand back to the keyboard. That’s one of the powerful aspects of tiling wm’s you often hear people hyping up :)

                                                                              1. 3

                                                                                It will definitely be faster than taking your hand up from the keyboard

                                                                                Faster doesn’t mean easier.

                                                                                1. 2

                                                                                  Ah yeah, perhaps not. There’s probably some upfront “cost” or “investment” to get that muscle memory and ultra familiarity with the key combinations. That said, I do think it will pay off long term. Who knows, we’ll see. I intend to spend more time in some tiling (or otherwise keyboard-driven) window manager in the future, since I’ve only dabbled casually so far!

                                                                              2. 1

                                                                                I think people overestimate how useful the more advanced functionality of vi is.

                                                                                That is an interesting point. As an Emacs user I have always envied vi users their rich command language. But if you don’t really use it, does it matter?

                                                                                (Yeah, I know about Emacs easily supports vim bindings. I just don’t use them. Never got around to it, even though I use vi all the time to edit system configuration files.)

                                                                                I generally feel the same way about tiling and keyboard-driven window managers. There’s no way a keyboard shortcut for resizing a window is easier than using the mouse.

                                                                                The key is not having to do either in the first place. I have StumpWM configured so that most of the time I just have two windows, each filling one monitor. I have bindings to raise different windows: S-w for Firefox (‘web’); C-S-w for Chrome; S-c for the terminal (‘console’); S-e for Emacs. So I do not have to use the mouse at all. If I need to split a frame, I do, and fill it with whichever window I need. It works pretty well, better than GNOME, KDE, Windows or Mac OS by far

                                                                                1. 1

                                                                                  That is an interesting point. As an Emacs user I have always envied vi users their rich command language. But if you don’t really use it, does it matter?

                                                                                  What is really useful about vi isn’t the advanced stuff. It’s that the really simple stuff is so easy to do. For example:

                                                                                  1. dd deletes the current line.
                                                                                  2. dj deletes this line and the next.
                                                                                  3. d{ deletes until the end of the (next) paragraph

                                                                                  Simple things like those are usually more complicated in other editors. Let’s see if I can remember the Emacs equivalents:

                                                                                  1. ^A (go to beginning of line), ^K (delete till end of line), ^K (delete the empty line)
                                                                                  2. ^A, ^space (place mark/begin selection), ^N ^N (go two lines down), ^W (delete selection)
                                                                                  3. ^A, ^space, M-} (go to end of paragraph – had to look that one up), ^W

                                                                                  So, in other words, it’s (mostly) the simple stuff that makes the difference. Now, examples 2 and 3 aren’t really a big deal, because they’re fairly uncommon operations, but number 1 is a very common operation.

                                                                                  1. 1

                                                                                    ct is really simple in vim/evil-mode. I think I’d need to reach for the mouse to do that in pure emacs.

                                                                                  2. 1

                                                                                    The key is not having to do either in the first place. I have StumpWM configured so that most of the time I just have two windows, each filling one monitor. I have bindings to raise different windows: S-w for Firefox (‘web’); C-S-w for Chrome; S-c for the terminal (‘console’); S-e for Emacs.

                                                                                    Am I right in assuming you have the positions and sizes of those windows pre-configured? In that case, yes, that is a good way of avoiding the need to manually move/resize windows. In my experience, however, most people who use tiling window managers let the window manager automatically move/resize windows, which usually requires at least the occasional manual intervention in order to be useful.

                                                                                    1. 1

                                                                                      Am I right in assuming you have the positions and sizes of those windows pre-configured?

                                                                                      The keybinding pulls the window into the current frame, and normally I just have one or two frames per screen.

                                                                                      StumpWM is the Emacs of window managers, and in this case its frame-handling mirrors that of Emacs: you split the available space up as you wish, and mostly leave it alone after that, pulling in windows as desired. At least that’s how I use it.

                                                                                      I pretty much never spend time resizing frames.

                                                                                    2. 1

                                                                                      I have bindings to raise different windows

                                                                                      It takes longer for me to remember the shortcuts than to hit command+space and type the first three letters of the application. :)

                                                                                      I know command+option+4 is what I need to hit to take a screenshot… most of the time. I’ve often sat for 3 seconds trying to remember what to hit. And then having to look at my keyboard to hit it. (While I’m hoping I don’t forget what I was taking a screenshot of.)

                                                                                      Yet I have no issues touch typing? My brain is frustrating to work with.

                                                                                  3. 2

                                                                                    i3 (and now sway) has ruined every other desktop for me.

                                                                                  1. 9

                                                                                    They could have quit 10 or 15 years ago and the world would have been no worse off. Let the historical linguists hash things out in their own private-use blocks, and let people send little vector graphics to each other in a way that doesn’t require assigning them as part of an international standard and then burdening every goddamn computing device for the rest of eternity.

                                                                                    1. 15

                                                                                      There is more than historical scripts being added. For example:

                                                                                      Tangsa, a modern script used to write the Tangsa language, which is spoken in India and Myanmar

                                                                                      There are a good number of other, still alive scripts that are yet to be added. There is no reason to burden the internet infrastructure with vector images, when all you need is a decision and a font update. If your application really cares, maybe the character data update as well.

                                                                                      1. 3

                                                                                        Imagine if they had added a Turkish delight emoji back in the 1940s. Everyone loves Turkish delight, right!? Except no, to contemporary tastes it is chalky and gross. Well, every food emoji is a bet that for the remainder of human civilization, we will want to be able to refer to a culturally specific food with a particular 32-bit number. It’s crazy. We already have as an historical accident a bunch of Japanese emoji that make no sense in an American context. Why would you deliberately add more culturally specific junk to humanity’s permanent record? In conclusion, 𓂺.

                                                                                        1. 11

                                                                                          What is in humanity’s permanent record, aside from culturally specific junk? Besides, so far there are only 144,697 characters, 1,404 emoji, and whatever else is in there. There’s still room for a few thousand more Turkish delights.

                                                                                          1. 3

                                                                                            In the interest of various parties I demand Greek delights, Parthian delights, Scythian delights and Armenian delights. Luckily Unicode can handle them all!

                                                                                            1. 1

                                                                                              I’d like to upvote the first sentence of this post 144,697*1404 times just for starters.

                                                                                            2. 2

                                                                                              Is that really the consensus on Turkish delight? :-)

                                                                                              1. 1

                                                                                                This. Emoji is a necessary part of Unicode since it must subsume earlier encodings, but emoji is a Japanese phenomenon. Any new emoji (if ever) should be left up to the Japanese, just like any new kanji.

                                                                                              2. 1

                                                                                                What are fonts? :)

                                                                                                1. 1

                                                                                                  turtle graphics on acid.

                                                                                            1. 7

                                                                                              Hm this is probably heresy, but “out params” to return values and the error being the actual return value solves a lot of these problems.

                                                                                              That style is idiomatic in C and some C++, which don’t really have multiple return values. I believe C# has explicit support for it.

                                                                                              I find myself using it more in Python even though it’s not idiomatic. And it does actually make code shorter and reduces these dependency problems. Surprisingly, the code can be more composable and refactorable.

                                                                                              There are a couple examples from the Oil codebase I could dig up. I started with multiple return values like Go and refactored to out params and I was surprised how much cleaner the code is. A lot of it has to do with the fact the interpreters have more intricate conditional logic than other types of code.

                                                                                              I also use exceptions, but the out params style is nice for specific cases where “I know exactly what error can happen and I want to do something special with it at a specific spot”. The code can also be shorter than exceptions which cause a 3-4 line boilerplate around a function.

                                                                                              1. 2

                                                                                                A sum type would be better–it forces handling two distinct cases with clear separation of concerns.

                                                                                                1. 1

                                                                                                  Hm this is probably heresy, but “out params” to return values and the error being the actual return value solves a lot of these problems.

                                                                                                  I don’t immediately see in which way this is different from multiple return values. Could you give an example?

                                                                                                  1. 4

                                                                                                    This might not be that convincing / intelligible, but it’s the code I was thinking of:

                                                                                                    https://github.com/oilshell/oil/blob/master/osh/word_eval.py#L1332

                                                                                                    Basically vsub_state is an “out param” that is mutated, but there’s also a return value val.

                                                                                                    This is for evaluating:

                                                                                                    • $foo and "$@" – “simple var sub”
                                                                                                    • ${foo} and ${@} – “braced var sub”
                                                                                                    • as well as “var refs” like ${!a} where the value is @, which is yet another way to get $@

                                                                                                    So all of those return a “val”. It is like a typical interpreter that does a switch/match over dozens of cases and returns a different value based on the sum type.

                                                                                                    But then only in minority of cases do you have to convert an array to a string – this is weird shell behavior.

                                                                                                    The out param means you don’t have to “pollute” every code path with that logic; instead I just mutate the out param in the one place I need to. AND sometimes you pass the out param down TWO levels instead of one.

                                                                                                    And it composes well – I can use it from 3 different contexts.

                                                                                                    This is pretty deep inside baseball, but I did actually refactor it from a different style, and this ended up cleaner, shorter, and more composable. It probably looks pretty confusing without detailed knowledge of the problem domain, but I think reading the signatures of the functions now clarifies a lot of things.


                                                                                                    I don’t think this style applies everywhere – sum types and exceptions are more common. But when I saw the Go code with 2 return values I can definitely see it being cleaner, roughly like

                                                                                                    T1 x;
                                                                                                    if (!func1(&x) {  // x is output param, return value is success/fail
                                                                                                      return false;
                                                                                                    }
                                                                                                    
                                                                                                    T2 y;
                                                                                                    if (!func2(x, &y)) {  // x is input, y is output
                                                                                                      return false;
                                                                                                    }
                                                                                                    
                                                                                                    T3 z;
                                                                                                     if (!func3(y, &z)) {  // y is input, z is ouput
                                                                                                       return false;
                                                                                                     }
                                                                                                     return true;  // success, output can be read from z "out param"
                                                                                                    

                                                                                                    This is basically func3(func2(func1())) but with (non-exception) error handling (in C++ syntax).

                                                                                                1. 6

                                                                                                  I like Perl too. (As an aside, I recommend not using GitHub to host code blocks. They render in a huge font size on iPhone and are broken in reader mode.)

                                                                                                  1. 2

                                                                                                    Thanks for the feedback! I’ve replaced them with regular code blocks.

                                                                                                  1. 10

                                                                                                    I do care about cookies

                                                                                                    1. 3

                                                                                                      You’re in luck then. Soon every single useless site will have a banner :)

                                                                                                      1. 12

                                                                                                        Except the sites that don’t try to track you :)

                                                                                                        1. 4

                                                                                                          My hope is that sites will see their numbers drop from having the banners and rethink the policy of tracking everyone. You can still have analytics without saving identifying information, you can still show content without tracking the user. There is no need for this behaviour. Having said that I don’t have an issue with people blocking the banners, apparently the majority don’t even block ads so I can’t see this becoming widespread enough to damage the incentives.

                                                                                                          1. 2

                                                                                                            What about default Apache servers or WordPress installations? I was under the impression that GDPR technically requires them to have a banner, but I find that ridiculous.

                                                                                                      1. 1

                                                                                                        Can I get some remedial explanation for this line to convert hex to decimal:

                                                                                                        SIZE=$( printf “%d\n” 0x${SIZE} )

                                                                                                        Why add the newline to the format?

                                                                                                        Shouldn’t we quote “0x${SIZE}”? I guess it works for me in without it, but it makes me nervous for some reason.

                                                                                                        1. 3

                                                                                                          Newline or not does not matter, as it is automatically stripped anyway.

                                                                                                          0x${SIZE} without quotes is safe if you know it does not contain glob (like *) or IFS characters (like spaces, usually). In this case, we know it does not contain IFS characters (as the read command splits on IFS), and we can be relatively certain it won’t ever contain a glob character.

                                                                                                          Still not best practice, though, as we ultimately do not control the value ourselves.

                                                                                                          1. 2

                                                                                                            You are right the ‘\n’ newline is not needed there, but does not hurt as it seams.

                                                                                                            As for the quoting - its always a good idea to quote variable - which I sometimes forgot - but in that case the size is without spaces so it works without a problem.

                                                                                                            Regards.

                                                                                                            1. 3

                                                                                                              I can add that printf '%s\n' is such a common pattern that it is probably a good idea to include the newline even when it is unnecessary, because it is necessary when piping the result, which is what is usually done:

                                                                                                              printf '%s\n' "$x" | ...
                                                                                                              
                                                                                                          1. 2

                                                                                                            Very cool! It would be super cool if you could post things.

                                                                                                            1. 2

                                                                                                              I’m writing an NNTP server library in python and plan to implement auth (it’s read-only for now). I will make a reddit bridge too, seems they have a well documented API.

                                                                                                              EDIT: Now that I think of it, you don’t need NNTP auth if you run the bridge locally.

                                                                                                            1. 24

                                                                                                              I agree with most of what’s said in the article. However, it misses a bit the forest for the trees, at least considering the introduction. Programs that take seconds to load or display a list are not just ignoring cache optimizations. They’re using slow languages (or language implementations, for the pedantics out there) like cpython where even the simplest operations require a dictionary lookup, or using layers and layers of abstractions like electron, or making http requests for the most trivial things (I suspect it’s what makes slack slow; I know it’s what makes GCP’s web UI absolutely terrible). A lot of bad architectural choices, too.

                                                                                                              Cache optimizations can be important but only as the last step. There’s a lot to be fixed before that, imho.

                                                                                                              1. 16

                                                                                                                Even beyond than that, I think there are more more baseline things going on: Most developers don’t even benchmark or profile. In my experience the most egregious performance problems I’ve seen have been straight-up bugs, and they don’t get caught because nobody’s testing. And the profiler basically never agrees with what I would have guessed the problem was. I don’t disagree with the author’s overall point, but it’s rare to come across a program that’s slow enough to be a problem that doesn’t have much lower hanging fruit than locality issues.

                                                                                                                1. 3

                                                                                                                  I agree so much! I’d even say that profiling is one half of the problem (statistical profiling, that is, like perf). The other half is tracing, which nowadays can be done with very convenient tools like Tracy or the chrome trace visualizer (“catapult”) if you instrument your code a bit so it can spit out json traces. These give insights in where time is actually spent.

                                                                                                                  1. 1

                                                                                                                    Absolutely. Most developers only benchmark if there’s a serious problem, and most users are so inured to bad response times that they just take whatever bad experience they receive and try to use the app regardless. Most of the time it’s some stupid thing the devs did that they didn’t realize and didn’t bother checking for (oops, looks like we’re instantiating this object on every loop iteration, look at that.)

                                                                                                                  2. 9

                                                                                                                    Programs that take seconds to load or display a list are not just ignoring cache optimizations.

                                                                                                                    That’s right. I hammered on the cache example because it’s easy to show an example of what a massive difference it can make, but I did not mean to imply that it’s the only reason. Basically, any time we lose track of what the computer must do, we risk introducing slowness. Now, I don’t mean that having layers of abstractions or using dictionary are inherently bad (they will likely have a performance cost, but it may be reasonable to reach another objective), but we should make these choices intentionally rather than going by rote, by peer pressure, by habit, etc.

                                                                                                                    1. 5

                                                                                                                      The article implies the programmer has access to low level details like cache memory layout, but if you are programming in Python, Lua, Ruby, Perl, or similar, the programmer doesn’t have such access (and for those languages, the trade off is developer ease). I’m not even sure you get to such details in Java (last time I worked in Java, it was only a year old).

                                                                                                                      The article also makes the mistake that “the world is x86”—at work, we still use SPARC based machines. I’m sure they too have cache, and maybe the same applies to them, but micro-optimizations are quite difficult across different architectures (and even across the same family but different generations).

                                                                                                                      1. 6

                                                                                                                        The article implies the programmer has access to low level details like cache memory layout, but if you are programming in Python, Lua, Ruby, Perl, or similar, the programmer doesn’t have such access

                                                                                                                        The level of control that a programmer has is reduced in favor of other tradeoffs, as you said, but there’s still some amount of control. Often, it’s found in those languages best practices. For example, in Erlang one should prefer to use binaries for text rather than strings, because binaries are a contiguous sequence of bytes while strings are linked lists of characters. Another example, in Python it’s preferable to accumulate small substrings in a list and then use the join method rather that using concatenation (full += sub).

                                                                                                                        The article also makes the mistake that “the world is x86”—at work, we still use SPARC based machines. I’m sure they too have cache, and maybe the same applies to them, but micro-optimizations are quite difficult across different architectures (and even across the same family but different generations).

                                                                                                                        I don’t personally have that view, but I realize that it wasn’t made very clear in the text, my apologies. Basically what I want myself and other programmers to be mindful of is mechanical sympathy — to not lose track of the actual hardware that the program is going to run on.

                                                                                                                        1. 4

                                                                                                                          I know a fun Python example. Check this yes implementation:

                                                                                                                          def yes(s):
                                                                                                                            p = print
                                                                                                                            while True:
                                                                                                                              p(s)
                                                                                                                          
                                                                                                                          yes("y")
                                                                                                                          

                                                                                                                          This hot-loop will perform significantly better than the simpler print(s) because of the way variable lookups work in Python. It first checks the local scope, then the global scope, and then the built-ins scope before finally raising a NameError exception if it still isn’t found. By adding a reference to the print function to the local scope here, we reduce the number of hash-table lookups by 2 for each iteration!

                                                                                                                          I’ve never actually seen this done in real Python code, understandably. It’s counter-intuitive and ugly. And if you care this much about performance then Python might not be the right choice in the first place. The dynamism of Python (any name can be reassigned, at any time, even by another thread) is sometimes useful but it makes all these lookups necessary. It’s just one of the design decisions that makes it difficult to write a high-performance implementation of Python.

                                                                                                                          1. 3

                                                                                                                            That’s not how scoping works in Python.

                                                                                                                            The Python parser statically determines the scope of a name (where possible.) If you look at the bytecode for your function (using dis.dis) you will see either a LOAD_GLOBAL, LOAD_FAST, LOAD_DEREF, or LOAD_NAME, corresponding to global, local, closure, or unknown scope. The last bytecode (LOAD_NAME) is the only situation in which multiple scopes are checked, and these are relatively rare to see in practice.

                                                                                                                            The transformation from LOAD_GLOBAL to LOAD_FAST is not uncommon, and you see it in the standard library: e.g., https://github.com/python/cpython/blob/main/Lib/json/encoder.py#L259

                                                                                                                            I don’t know what current measurements of the performance improvement look like, after LOAD_GLOBAL optimisations in Python 3.9, which reported 40% improvement: https://bugs.python.org/issue26219 (It may be the case that the global-to-local transformation is no longer considered a meaningful speed-up.)

                                                                                                                            Note that the transformation from global-to-local scope, while likely innocuous, is a semantic change. If builtins.print or the global print is modified in some other execution unit (e.g., another thread,) the function will not reflect this (as global lookups can be considered late-bound, which is often desirable.)

                                                                                                                            1. 8

                                                                                                                              I think this small point speaks more broadly to the dissatisfaction many of us have with the “software is slow” mindset. The criticisms seem very shallow.

                                                                                                                              Complaining about slow software or slow languages is an easy criticism to make from the outside, especially considering that the biggest risk many projects face is failure to complete or failure to capture critical requirements.

                                                                                                                              Given a known, fixed problem with decades of computer science research behind it, it’s much easier to focus on performance—whether micro-optimisations or architectural and algorithmic improvements. Given three separate, completed implementations of the same problem, it’s easy to pick out which is the fastest and also happens to have satisfied just the right business requirements to succeed with users.

                                                                                                                              I think the commenters who suggest that performance and performance-regression testing should be integrated into the software development practice from the beginning are on the right track. (Right now, I think the industry is still struggling with getting basic correctness testing and documentation integrated into software development practice.)

                                                                                                                              But the example above shows something important. Making everything static or precluding a number of dynamic semantics would definitely give languages like Python a better chance at being faster. But these semantics are—ultimately—useful, and it may be difficult to predict exactly when and where they are critical to satisfying requirements.

                                                                                                                              It may well be the case that some languages and systems err too heavily on the side of allowing functionality that reduces the aforementioned risks. (It’s definitely the case that Python is more dynamic in design than many users make use of in practice!)

                                                                                                                              1. 2

                                                                                                                                Interesting! I was unaware that the parser (!?) did that optimization. I suppose it isn’t difficult to craft code that forces LOAD_NAME every time (say, by reading a string from stdin and passing it to exec) but I find it totally plausible that that rarely happens in non-pathological code.

                                                                                                                                Hm. For a lark, I decided to try it:

                                                                                                                                >>> def yes(s):
                                                                                                                                ...  exec("p = print")
                                                                                                                                ...  p(s)
                                                                                                                                ... 
                                                                                                                                >>> dis.dis(yes)
                                                                                                                                  2           0 LOAD_GLOBAL              0 (exec)
                                                                                                                                              2 LOAD_CONST               1 ('p = print')
                                                                                                                                              4 CALL_FUNCTION            1
                                                                                                                                              6 POP_TOP
                                                                                                                                
                                                                                                                                  3           8 LOAD_GLOBAL              1 (p)
                                                                                                                                             10 LOAD_FAST                0 (s)
                                                                                                                                             12 CALL_FUNCTION            1
                                                                                                                                             14 POP_TOP
                                                                                                                                             16 LOAD_CONST               0 (None)
                                                                                                                                             18 RETURN_VALUE
                                                                                                                                >>> yes("y")
                                                                                                                                Traceback (most recent call last):
                                                                                                                                  File "<stdin>", line 1, in <module>
                                                                                                                                  File "<stdin>", line 3, in yes
                                                                                                                                NameError: name 'p' is not defined
                                                                                                                                
                                                                                                                          2. 5

                                                                                                                            and for those languages, the trade off is developer ease

                                                                                                                            I heard Jonathan Blow make this point on a podcast and it stuck with me:

                                                                                                                            We’re trading off performance for developer ease, but is it really that much easier? It’s not like “well, we’re programming in a visual language and just snapping bits together in a GUI, and it’s slow, but it’s so easy we can make stuff really quickly.” Like Python is easier than Rust, but is it that much easier? In both cases, it’s a text based OO language. One just lets you ignore types and memory lifetimes. But Python is still pretty complicated.

                                                                                                                            Blow is probably a little overblown (ha), but I do think we need to ask ourselves how much convenience we’re really buying by slowing down our software by factors of 100x or more. Maybe we should be more demanding for our slow downs and expect something that trades more back for it.

                                                                                                                            1. 2

                                                                                                                              Like Python is easier than Rust, but is it that much easier?

                                                                                                                              I don’t want to start a fight about types but, speaking for myself, Python became much more attractive when they added type annotations, for this reason. Modern Python feels quite productive, to me, so the trade-off is more tolerable.

                                                                                                                              1. 1

                                                                                                                                It depends upon the task. Are you manipulating or parsing text? Sure, C will be faster in execution, but in development?

                                                                                                                                At work, I was told to look into SIP, and I started writing a prototype (or proof-of-concept if you will) in Lua (using LPeg to parse SIP messages). That “proof-of-concept” went into production (and is still in production six years later) because it was “fast enough” for use, and it’s been easy to modify over the years. And if we can ever switch to using x86 on the servers [1], we could easily use LuaJIT.

                                                                                                                                [1] For reasons, we have to use SPARC in production, and LuaJIT does not support that architecture.

                                                                                                                          3. 7

                                                                                                                            The trick about cache optimizations is that that can be a case where, sure, individually you’re shaving nanoseconds off, but sometimes those are alarmingly common in the program flow and worth doing before any higher-level fixes.

                                                                                                                            To wit: I worked on a CAD system implemented in Java, and the “small optimization” of switching to a pooled-allocation strategy for vectors instead of relying on the normal GC meant the difference between an unusable application and a fluidly interactive one, simply because the operation I fixed was so core to everything that was being done.

                                                                                                                            Optimizing cache hits for something like mouse move math can totally be worth it as a first step, if you know your workload and what code is in the “hot” path (see also sibling comments talking about profiling).

                                                                                                                            1. 6

                                                                                                                              They’re using slow languages (or language implementations, for the pedantics out there) like cpython where even the simplest operations require a dictionary lookup

                                                                                                                              I take issue with statements like this, because the majority of code in most programs is not being executed in a tight loop on large enough data to matter. The overall speed of a program has more to do with how it was architected than with how well the language it’s written in scores on microbenchmarks.

                                                                                                                              Besides, Python’s performance cost isn’t a just an oversight. It’s a tradeoff that provides benefits elsewhere in flexibility and extensibility. Problems like serialization are trivial because of meta-programming and reflection. Complex string manipulation code is simple because the GC tracks references for you and manages the cleanup. Building many types of tools is simpler because you can easily hook into stuff at runtime. Fixing an exception in a Python script is a far more pleasant experience than fixing a segfault in a C program that hasn’t been built with DWARF symbols.

                                                                                                                              Granted, modern compiled languages like Rust/Go/Zig are much better at things like providing nice error messages and helpful backtraces, but you’re paying a small cost for keeping a backtrace around in the first place. Should that be thrown out in favor of more speed? Depending on the context, yes! But a lot of code is just glue code that benefits more from useful error reporting than faster runtime.

                                                                                                                              For me, the choice in language usually comes down to how quickly I can get a working program with limited bugs built. For many things (up to and including interactive GUIs) this ends up being Python, largely because of the incredible library support, but I might choose Rust instead if I was concerned about multithreading correctness, or Go if I wanted strong green-thread support (Python’s async is kinda meh). If I happen to pick a “fast” language, that’s a nice bonus, but it’s rarely a significant factor in that decision making process. I can just call out to a fast language for the slow parts.

                                                                                                                              That’s not to say I wouldn’t have mechanical sympathy and try to keep data structures flat and simple from the get go, but no matter which language I pick, I’d still expect to go back with a profiler and do some performance tuning later once I have a better sense of a real-world workload.

                                                                                                                              1. 4

                                                                                                                                To add to what you say: Until you’ve exhausted the space of algorithmic improvements, they’re going to trump any microoptimisation that you try. Storing your data in a contiguous array may be more efficient (for search, anyway - wait until you need to insert something in the middle), but no matter how fast you make your linear scan over a million entries, if you can reframe your algorithm so that you only need to look at five of them to answer your query then a fairly simple data structure built out of Python dictionaries will outperform your hand-optimised OpenCL code scanning the entire array.

                                                                                                                                The kind of microoptimisation that the article’s talking about makes sense once you’ve exhausted algorithmic improvements, need to squeeze the last bit of performance out of the system, and are certain that the requirements aren’t going to change for a while. The last bit is really important because it doesn’t matter how fast your program runs if it doesn’t solve the problem that the user actually has. grep, which the article uses as an example, is a great demonstration here. Implementations of grep have been carefully optimised but they suffer from the fact that requirements changed over time. Grep used to just search ASCII text files for strings. Then it needed to do regular expression matching. Then it needed to support unicode and do unicode canonicalisation. The bottlenecks when doing a unicode regex match over a UTF-8 file are completely different to the ones doing fixed-string matching over an ASCII text file. If you’d carefully optimised a grep implementation for fixed-string matching on ASCII, you’d really struggle to make it fast doing unicode regex matches over arbitrary unicode encodings.

                                                                                                                                1. 1

                                                                                                                                  The kind of microoptimisation that the article’s talking about makes sense once you’ve exhausted algorithmic improvements, need to squeeze the last bit of performance out of the system, and are certain that the requirements aren’t going to change for a while.

                                                                                                                                  To be fair, I think the article also speaks of the kind of algorithmic improvements that you mention.

                                                                                                                                2. 3

                                                                                                                                  Maybe it’s no coincidence that Django and Rails both seem to aim at 100 concurrent requests, though. Both use a lot of language magic (runtime reflection/metaprogramming/metaclasses), afaik. You start with a slow dynamic language, and pile up more work to do at runtime (in this same slow language). In this sense, I’d argue that the design is slow in many different ways, including architecturally.

                                                                                                                                  Complex string manipulation code is simple because the GC tracks references for you

                                                                                                                                  No modern language has a problem with that (deliberately ignoring C). Refcounted/GC’d strings are table stakes.

                                                                                                                                  I personally dislike Go’s design a lot, but it’s clearly designed in a way that performance will be much better than python with enough dynamic features to get you reflection-based deserialization.

                                                                                                                                3. 1

                                                                                                                                  All the times I had an urge to fire up a profiler the problem was either an inefficient algorithm (worse big-O) or repeated database fetches (inefficient cache usage). Never have I found that performance was bad because of slow abstractions. Of course, this might be because of software I work with (Python web services) has a lot of experiences on crafting good, fast abstractions. Of course, you can find new people writing Python that don’t use them, which results in bad performance, but that is quickly learned away. What is important if you want to write performant Python code, is to use as little of “pure Python” as possible. Python is a great glue language, and it works best when it is used that way.

                                                                                                                                  1. 1

                                                                                                                                    Never have I found that performance was bad because of slow abstractions.

                                                                                                                                    I have. There was the time when fgets() was the culprit, and another time when checking the limit of a string of hex digits was the culprit. The most surprising result I’ve had from profiling is a poorly written or poorly compiled library.

                                                                                                                                    Looking back on my experiences, I would have to say I’ve been surprised by a profile result about half the time.

                                                                                                                                  2. 1

                                                                                                                                    As a pedantic out here, I wanted to say that I appreciate you :)

                                                                                                                                  1. 8

                                                                                                                                    That is amazing! I was unaware of such a project. I’m not asking it to become the default, I just would like such an init system to be available on BSD at the administrator’s choice. (which would be my case)

                                                                                                                                    This is a great result for what it seems to be a one-man project.

                                                                                                                                    1. 7

                                                                                                                                      I’m not asking it to become the default, I just would like such an init system to be available on BSD at the administrator’s choice. (which would be my case)

                                                                                                                                      Interesting – I find BSDs appealing and use them because of a lack of choice. The whole system is “curated”, all the parts fit together. I have to check manpages to know apache uses -t to check its config file whereas haproxy uses -c, but if I use relayd and httpd I use the same flag that I use for pfctl and other builtins.

                                                                                                                                      So for me, the fact that “there’s one way to do it” is quite appealing!

                                                                                                                                      1. 3

                                                                                                                                        In my experience, BSD is more curated than Linux, but I wouldn’t say it follows the “there is only one way to do it” principle. For example, on both FreeBSD and NetBSD, there is more than one firewall included by default (pf, ipfilter, npf etc.), and the administrator can generally choose whichever he prefers.

                                                                                                                                      2. 1

                                                                                                                                        I just would like such an init system to be available on BSD at the administrator’s choice

                                                                                                                                        I replaced the FreeBSD’s rc.d with daemontools 15 years ago. It was just a port at the time (not sure if it still is), and it’s something that’s been possible for a long time if you’re willing to put a bit of effort in it.

                                                                                                                                        Fairly sure you can also do it on OpenBSD etc.