Threads for frkl

  1. 3

    Very interesting, thanks for the submission! This does seem to have some similarties to pyoxidizer / pyembed or at least it looks to me that way (from the outside), that there would be a big potential to re-use code (given that both use Rust). Yet, Pyoxidizer is not mentioned anywhere, so I assume the two projects are separate.

    Even though it would probably make sense to somehow separate the interpreter- from the env-management part, I’d think it would make sense for those two projects to re-use each others code in some way or other, but maybe I’m missing something?

    1. 4

      As someone with maybe 20 hours of experience writing Nim, I’m almost impressed and generally just a little frustrated. For a new language with no baggage it’s just ludicrously inconsistent.

      For loops:

      for x in 0..10:
        echo x
      
      # backwards ranges have a completely different syntax
      for x in countdown(10, 0):
        echo x
      

      Hash tables:

      # making a hash of 1->2 like this works fine...
      var x: TableRef[int, int] = {1: 2}.newTable()
      
      # ... but the empty table is a compiler error?!
      var x: TableRef[int, int] = {}.newTable()
      
      # this is how you're supposed to make an empty table
      var x = newTable[int, int]()
      

      Working with arrays:

      var thing = @[1, 2, 3]
      # Some functions accept a callback:
      thing.sort(compareFunc)
      
      # but some don't, such as maxIndex (and there's no max):
      thing.maxIndex()
      
      # there's also family of "It" functions that take a code template 
      thing.mapIt(it * 2)  # doubles the items in a list
      
      1. 4

        I found the fact that you can set a magic variable “result” or use the return keyword, to be pretty strange too. I’ve been reading through “mastering nim” and have run into a few other things like this that I found likewise strange and/or inconsistent. With so much to otherwise like about the language, these little oddities stand out a bit more starkly. It eventually becomes off-putting.

        1. 1

          I found the ‘result’ thing weird initially too, but after getting used to it I really like it, it feels very natural and I prefer it to the other ways of returning a value.

        2. 3

          For a new language with no baggage it’s just ludicrously inconsistent.

          Nim actually has massive amounts of baggage IMO, with the language going through multiple changes in focus over its lifetime and having quite a bit of fancy extra functionality that was never actually fleshed out.

          1. 1

            Care to elaborate?

          2. 1

            While there is a good deal of variation, much of this is from Nim starting in 2006..8 and being 14..16 years old now with consistency conventions evolving.

            While your main point is not wholly off-track, your examples rub me the wrong way. To take just the first, the countdown iterator is anti-symmetric with the countup iterator, both of which take a step size. `..` follows more natural language notation which is asymmetric. It’s only 2 lines of code to add an anti-symmetric counterpart, but your code readers may not find guessing its meaning as easy:

            iterator `-->`[T](a, b: T): T =
              for x in countdown(a, b): yield x
            for x in 10-->0: echo x
            

            So, the inconsistency you are complaining about on this line item is just one of human language abbreviation. Given the above you would also want a <-- (or maybe the above should already be that with args reversed? Often it’s a choice of what to be consistent with, in this case loop direction OR argument order).

            Anyway, this is maybe a good example for readers of how, unlike many prog.langs, Nim lets users just define their own operators. Unlike Python, there is really no penalty to just adding your own “shims” or layers on top of the stdlib to match whatever consistency you like. (Nim’s macro system could even automate that.)

            While the starting state may not be to your liking, you absolutely have the power to fix everything like the things you complained about in your local code base. (But I can understand not wanting to..some love an out of the box experience that happens to perfectly align with their own tastes while Nim Is Choice is something I often say).

            1. 4

              While the starting state may not be to your liking, you absolutely have the power to fix everything like the things you complained about in your local code base. (But I can understand not wanting to..some love an out of the box experience that happens to perfectly align with their own tastes while Nim Is Choice is something I often say).

              That’s all well and good but I don’t think that’s entirely fair here. A New user (at least this new user) doesn’t yet have the ability to go down a rabbit hole of macros and templates to fix empty table creation. I don’t think it’s unfair to say the language would be objectively better without that particular wart, so why make everyone fix it in a slightly different way?

              And this isn’t purely some “Ew, look how gross that is,” reaction but rather a very real issue of discoverability.

              Anywhoo, I agree that There’s No Accounting For Taste. Maybe if I were more invested in nim I’d be willing to tweak it to my satisfaction…but even then, I’m not super excited about an ecosystem of software where reading code is hard because everyone solves these papercuts in his or her own special way.

              1. 2

                There is no question that there are major discovery issues. Part of that is also “order of discovery” and some patience on the part of any new user of anything is required. Diversity of tutorials (of anything) is valuable partly because different people ask questions in different orders and come from various backgrounds.

                E.g., there is a consistent way to create most any empty container (Table, HashSet, seq, etc.) - just declare it, like var x: Table[int,int]. With value types the convention is an init pfx like var y = initTable[int,int](..).

                If you are coming from a more Python direction, an empty comprehension/builder style is probably a closer match (but verbose):

                import sugar, tables, sets
                let x = collect(for i in countup(0,-1): {1:2})
                echo x # gives you an empty table
                

                Change the {1:2} to whatever two types you need or {oneVal} for a HashSet or a scalar for a seq. Change the countup part to something “more real” and you get a builder notation along those lines. Verbose for “empty” compared to just a declaration, but more extendable to other construction loops.

                ref is more complex, replacing init* with new* and there are other ways using {} with maybe glitchy (open an issue?) behaviors, but not every way is glitchy or inconsistent and there are choices (somewhat consistent along some dimension).

                Anyway, I’m not claiming it’s perfect, as I tried to make clear initially, or trying to be unfair. There are less bazooka ways to go than macros unless you are clinging to “must have XYZ syntax”. I mostly think had the example code/tutorial/whatever you based your quick impressions upon been different then your complaint examples would have been very different and that “Right To Repair” is an important property. Both may be vacuous observations, but, hey, I put some more Nim examples in there for other readers. I’m really only trying to add information for thoughtful consideration. Sounds like your interest in Nim has come & gone, though which is fine. No obligation to like anything.

          1. 1

            For anyone looking for a good Python implementation, I recently discovered https://github.com/hashberg-io/multiformats and couldn’t be happier.

            1. 3

              I always use multibase encoded multihashes for any new project using hashes

              1. 1

                Yes, I started to do this too now.

              1. 4

                The biggest challenge was storage. Hetzner charges around €50/month for a 1 TB volume (others have comparable pricing).

                They also offer dedicated servers with 8TB of storage and 64GB of RAM for around €45 a month. I know because I’ve been looking into a similar setup for hosting my plex library. I’m still trying to decide whether I want to rent a dedicated server, or bite the bullet and build a proper homelab.

                1. 4

                  Why not use a Hetzner storagebox ( https://www.hetzner.com/storage/storage-box ) or even a storage share ( https://www.hetzner.com/storage/storage-share )? 5 TB for €11/month for example, and traffic between that and a hetzner cloud vm (or any of the real servers there) is free (and fast enough for stuff like this). I don’t think there is a real need to have block (nor blob) storage, mounting webdav, samba or sshfs works perfectly fine in my experience.

                  1. 1

                    I actually wasn’t aware of that (apparently I didn’t do enough research). In my particular case though, I want to use the machine to host a few VMs as well. €11/month for 5TB (€12 after VAT for me) is pretty good though. If storage boxes supported iSCSI, I’d definitely consider it, but for now I’m leaning toward the homelab option as I don’t strictly need anything public facing.

                    1. 2

                      Yeah, if they offered block storage via iSCSI, that’d be something else. We can always dream :-)

                      Personally, I have a Hetzner AMD box (can’t remember which exactly) with 2 ssd drives (raided) that host my containers and VMs, and a storage box mounted via CIFS for anything largeish that doesn’t need to be accessed fast, as well as backup of my VM snapshots and other data. That worked out quite well so far, esp. for the price.

                      1. 1

                        That’s a nice solution, and I’ll look into it. A smaller amount RAIDed NVMe attached directly to the server for VMs, and a reasonably sized storage box mounted via CIFS would actually be better for my use case than 8TB of slow HDDs, which now looks like a poor compromise between the two needs.

                        I’m curious now about the rest of your setup. What host OS / hypervisor / container-orchestrator are you using for containers/VMs?

                        1. 1

                          Yes, I thought the same when considering the spinning disk version of this server, and the storage box solution also has the advantage that I don’t have to worry about raiding the hard-disks to get some redundancy, effectively cutting storage space in half. I trust the Hetzner storage box will do a much better job at keeping my data relatively safe than I could. I was a bit concerned about access performance at first, but for what I do with it (which is arguably nothing very demanding), it has been no problem at all.

                          It’s been a while since I set-up that box, it just chugs along without many problems, just doing regular updates on it. This is the second iteration of my setup, for the first one I used Proxmox, which worked quite well, but for this I didn’t even bother, and just installed k3s on Debian stable. That runs most of my services. I also have kvm to host a Mac OS X VM which I use to build Mac binaries, and another special purpose Linux VM if I remember right. It’s a bit more effort to setup than Proxmox, but overall more flexible and nowadays it’s just considerably easier to do containers in kubernetes than lxc.

                          I’ve got my 2 ssds split into half each, and using a raid1 btrfs for system and boot (so if I loose one disk the system will still boot – plus there might be a tiny bit better read performance) and data I want to keep a bit more safe (in addition to backup), and the other half is just for ‘normal’ fast storage that is attached to some of the vms/containers, so effectively I’m getting ~ 750GB of fast storage space, 250 of which are raid-1’ed.

                          Ah, and as I’ve written in another comment, I also use gocryptfs to encrypt part of the storagebox filesystem that is mounted via CIFS (so, basically gocryptfs inside a samba share). Which at first was just an experiement, because I thought the performance would surely be atrocious. But it turned out to not be too bad, every few months it hangs itself, but a force remount usually fixes that.

                          Wireguard for accessing the box and the VMs.

                    2. 1

                      Someone on Twitter (was that you?) also suggested that, that looks great. I think I’ll switch to a storage box, rclone supports SFTP after all.

                      1. 1

                        Or you could just go with CIFS and don’t use rclone at all. That works quite well for my Hetzner root box. I haven’t used it with cloud VMs myself yet, but I don’t think it’d work any different. I’ve encrypted my storagebox using gocryptfs (which probably adds a bit of latency), but even without that I don’t see any advantage of using SFTP over CIFS from a Hetzner VM. Filesystem listings and random access should be quite a bit faster compared to the rclone solution, but that’s just me guessing, maybe rclone is exceptionally good with caching.

                        But I’m probably just over-optimizing, esp. if it’s just for Plex. As long as you make sure to have the Plex metadata locally on your VM disk, the worst that will happen is that a movie starts a second or two later, probably.

                        EDIT: or you could just filesystem-mount via sshfs or webdav if you don’t like CIFS? I haven’t tried that myself, so it might not work with storagebox, but I’d imagine any (more or less) ‘native’ way to mount a filesystem would always be better than rclone, if available?

                    3. 3

                      Homelabs are definitely cool! If you have a public IPv4 this is probably the cheapest option.

                      The dedicated server makes sense when you have media that needs to be re-encoded as you‘ll have access to the hardware encoder.

                      1. 1

                        8TB storage and 64GB of RAM for around €45 a month

                        Really? Is there a catch? That’s a pretty ridiculously low price even just for 8 TB block storage alone without any compute!

                        1. 3
                          1. Spinning rust

                          2. Limited selection of data centers (Germany and Finland)

                          3. You’re literally just renting two hard disks, with no bells and whistles so you don’t have any redundancy or reliability that cloud block storage (e.g. AWS EBS) would provide.

                          By default it’s two 4TB mechanical disks configured as software RAID1, but you should be able to install your own OS and treat it as 8TB of RAID0 block storage if you want.

                          https://www.hetzner.com/dedicated-rootserver/ex42/

                          That works fine for a server storing torrented pirated media linux ISOs, but it’s not ideal for everything.

                      1. 6

                        This is one of the coolest projects I’ve come across in the last few years, and I’ll never not upvote it. Even if it was posted every day.

                        I’m working on this data-science library, and one of my ideas is to wrap exported datasets including their metadata (properties, lineage information, annotations) and a generic html file, and redbean into a zip file. The html page would serve as a sort of ‘data-passport’, showing a preview of the data itself, the metadata, etc. So, basically every dataset would come with a cross-platform way of executing it and showing itself off in a browser if users so choose to do. I haven’t really thought that through entirely, esp. security considerations (making random files executable might not be the best idea to recommend to users), and how that would work having a server process running on an OS where the commandline is not used by most users (Windows) for example.

                        But even if I decide to not go forward with the idea, just the potential that I could blows my mind…

                        1. 3

                          That’s exactly the kind of use case I intended to enable with redbean. It’d be great if every website with Jupyter Notebooks had a button to “download my notebook into a redbean”. It’s why with Cosmopolitan Libc we’ve been focusing on porting other languages too besides Lua, e.g. Python, LISP, JavaScript, etc. Regarding security, redbean is one of the few application servers that’s viable in an air-gapped environment. What could be more secure than that?

                          1. 2

                            It’s why with Cosmopolitan Libc we’ve been focusing on porting other languages too besides Lua, e.g. Python, LISP, JavaScript, etc.

                            Oh, I wasn’t aware of this. Please stop, my imagination is running wild. I’m getting to an age where this sort of excitement is keeping me sleepless for nights on end…

                            Regarding security, redbean is one of the few application servers that’s viable in an airgapped environment. What could be more secure than that?

                            I don’t disagree with that. I meant more the practice of encouraging my less-then-aware users to just set the executable flag on any random (data or not) file people send them.

                            1. 4

                              What is open source but posting files online containing code and encouraging others to download and execute them? If you want a snapshot of things to come, we managed to get Python 3.6 down to a 5.5mb Actually Portable Executable. That’s including all its standard library dependencies which are baked into the ZIP structure of the binary. It took quite a hacking marathon. https://justine.lol/actually-portable-python.png It’s going to do so much to help facilitate a sharing of things like scientific knowledge that can be reproduced and interacted with for years to come, rather than merely read like a PDF file, or being something that breaks the next time Ubuntu upgrades, or requires that people in other parts of the world with less Internet privilege download gigabytes of bundled stuff? The technology that underpins redbean is going to help address all those things. You can read more about it here: https://justine.lol/ape.html If anyone reading wants to help make this vision of eliminating the barriers to knowledge, please consider sponsoring me on GitHub.

                              1. 1

                                What is open source but posting files online containing code and encouraging others to download and execute them?

                                Yeah, I guess you are right.

                                What’s the plan with Python packages? I reckon including pure-Python ones should be feasible, but esp. for scientific data numpy and pandas would probably be much more difficult to support. Having sqlite available will make for a good workaround in a lot of cases, though. Building a more portable version of https://datasette.io/ might be an interesting project idea…

                                Have you ever looked into Apache Arrow? Intuitively, it seems like it could be a good fit with readbean if you wanted redbean to venture into the data science space a bit. It being portable, and memory efficient, mmappable and all that.

                                Either way, I’ll definitely keep an eye out for the Python support, even if everything I would like to do right now would already be possible with Lua.

                                If anyone reading wants to help make this vision of eliminating the barriers to knowledge, please consider sponsoring me on GitHub.

                                Done, good on you for mentioning it, I wouldn’t have thought to check… If I end up using it in our project, I’m pretty sure I can get them to sponsor as well.

                                1. 3

                                  Thank you for your support!

                                  Porting Python packages is something we’ve been doing by hand so far. It’s largely been a community driven effort due to how native dependencies have effectively locked-in so many packages to specific operating systems. Some examples of things we’re doing, is I started writing a Python bindings generator for chibicc which is an embeddable ~300kb C compiler. It’s all part of the Cosmopolitan Libc repo. My intent here is that, when you just want the native code to work, and you don’t care about having the last inch of performance you’d get with GCC and Clang, then all you need to do is plop the .c files into the .com file using a zip tool, and the Python implementation will just magically build them and link them in automatically across platforms. That would be the pain free path. But if you want the full perf for things like scientific computing, then you just need to integrate it with the Cosmopolitan Libc makefile build.

                                  This same concept would naturally also apply to generating Lua bindings in Redbean for any C code where you need extra performance. chibicc is great. Since half as fast as GCC is still 10x faster than scripting languages. Plus its performance will improve too over time as we add things like simple optimization passes.

                                  1. 2

                                    Thank you for your support

                                    Thank you for your work! Honestly, I can’t tell you how much I appreciate people like you spending what looks like a lot of energy, thoughtfulness (and probably love) on really original ideas. Things like this make my days brighter. I thought redbean was an outstanding idea, but the more I learn about Cosmopolitan libc itself,.. When I first heard about it I could not imagine how much further you’d be able to bend this.

                                    chibicc does sound great, I really have to find the time to look into all of this. I don’t want to harp on about Python specifically, but now I’m really intrigued: do you think this could become a viable alternative to things like PyInstaller, nuitka, PyOxidizer in the future? Esp. for cases where performance is not of upmost importance, and even if it’d mean we can’t use every Python library under the sun. Being able to distribute a single-file, cross-platform Python command-line app would be absolutely fantastic.

                                2. 1

                                  So excited about the possibilities with Python support! Amazing!

                          1. 8

                            I would recommend Nix over Conda for new projects. The NixOS community has a comparison between Nix and Conda. While there may be many ways in which Nix is slightly painful compared to Conda, the terms of service alone seem like a motivating reason to avoid Conda.

                            1. 1

                              What problems (particularly ones stated in the article) does Nix solve here?

                              1. 3

                                From the article’s perspective, Nix is like Conda but more reproducible. The running example from the article becomes a single-line expression with nixpkgs:

                                python39.withPackages (ps: [ ps.numpy ps.pandas ps.pillow ])
                                

                                To make a third column in the first table, adding Nix alongside pip and Conda, Nix does everything that Conda can do. The third table is similarly simple; features like reproducibility and virtual environments are baked into Nix’s design, and nixpkgs has a security team. To make a third column in the second table, comparing nixpkgs to PyPI and conda-forge:

                                • package names are maintained by consensus and anybody can contribute
                                • build infrastructure is centralized but optional since Nix can build packages locally
                                • any open-source or Free Software Python packages can be installed as long as source code is available
                                • more tools are available than with any traditional Linux distribution (visual comparison)
                                • Linux is first-class, Darwin is well-supported but has fewer packages, and Windows is unsupported

                                Finally, but non-trivially, the production of Docker-compatible container images is reproducible when using nixpkgs’ tools to assemble the image. This provides a reproducible alternative to Docker’s builder. In contrast, the article gives two non-reproducible Dockerfiles which depend on the state of global package repositories.

                                On serious examination, the only reason to recommend Conda might be for Windows users, but I usually recommend that they change their entire software stack at that point.

                                1. 2

                                  Yes, Nix can do everything Conda can, the problem I see is that you may want to install precise versions of your dependencies and instead if you stick to a certain Nixpkgs release or commit, you have to use the version in the distro. I don’t know if that affects Conda as well. That’s the reason why for most my Python project I use few distro packages and instead i rely on mach-nix which does a very good work together with a private Nix binary cache in order to not have to rebuild the packages every here and then.

                              2. 1

                                Can nix do non-root install now (but still use a binary cache)? That was the one thing keeping me from using nix for this last time I checked (which was a while ago).

                                1. 3

                                  It needs root just to create /nix directory during installation (in single-user mode, in multi-user mode it will need root also to create some additional users to run the nix build service as) and yes, it does support binary caches in such setup

                                  1. 1

                                    Right, thanks. So that’s pretty much what I remember. Without a non-root install option (that also supports binary caches) it’s not a good fit for what I need to do (I can’t assume that I’ll have root access where I need to run my software, or that anyone would install it for me). Conda ticks those boxes, but I’d prefer the stronger guarantees that come with nix. Also the tooling.

                                    1. 3

                                      If the problem is one of distribution alone, then nix-bundle is a great prototyping tool, and nixpkgs supports static linking for many languages. As long as you’re not trying to share development environments too, Nix would still work for that situation.

                                      For the specific case of living in homedirs on somebody else’s hardware as a permanent tenant, I personally think that specialized toolchains should be developed which use “living off the land” techniques. These techniques are typically used by malware, but they could be used for good, too.

                                      1. 1

                                        living off the land

                                        TIL. Thanks for that, very interesting! I guess having scripts setup conda environments (or similar) in a semi-transparent way is not too far off that concept, which is what I’m doing at the moment. miniconda (or a custom installer created via constructor take quite a bit of pain out of that work, because they’ve already taken care of wrapping the install process into scripts, and they also take care of a lot of the inconsistencies between target platforms, and so on. As I said earlier, conceptually I’d prefer having something more rigorous like nix, but there is only so much time I can afford to spend on making deployment as simple as possible for my users…

                                        Also, thanks for mentioning nix-bundle, I think I stumbled upon it when reading about the packaging of Nyxt (or some other tool), but forgot about it again. I could probably make it work somehow, but the advantages compared to Conda would probably not be worth it in my scenario (modular, extendable data-science tool for non- (or not-so)-technical users, basically). And then I’d still have to worry about Darwin and Windows.

                                        If I ever find the time I would really love to look into it even if it’s just for Linux. Being able to create nix-based containers and AppImages and all that easily would be something very nice to have.

                              1. 2

                                Anyone has tried that out? I recently looked into switching from Pocket/Pinboard to wallabag, but I’m running into ‘can’t parse this webpage’ type errors a lot.

                                1. 16

                                  A blanket ban seems like a bad idea since there are certainly some people who post good stuff that is relevant, for example @ltratt’s latest submission.

                                  Drawing the line is really hard. Some accounts are just using the site to promote every little thing they write, regardless of how trivial it is, or they are submitting what are clearly unthought out code ideas with no real implementation behind them, possibly looking for feedback. I really don’t care for these submissions and I think they de-value the site.

                                  I really want to discourage the latter and I really want to see more of the former. I’m content with the system of raising concerns on IRC or to the mods and having a judgement call made that way rather than banning it. Maybe we can stress something on the submission page.

                                  1. 4

                                    I agree with your point more than others’. The issue IMO is not self-posting, it’s low-effort posting.

                                    Stuff like “OpenSSL 1.1.1x released” is worthless, I’m on the mailing list for that. Same goes if your own blog post if it was written in 30 minutes.

                                    On the other hand, I found @gthm’s latest post about backups to be great, and it sparked a very interesting discussion in comments. The post was submitted by @gthm him/herself.

                                    1. 3

                                      Stuff like “OpenSSL 1.1.1x released” is worthless, I’m on the mailing list for that. Same goes if your own blog post if it was written in 30 minutes.

                                      Yes, exactly. Low-effort blog posts are actually the thing I’m talking about and want to curtail, and my experience is that they’re pretty highly correlated to self-posts.

                                      1. 6

                                        A blanket ban on posting your own articles would be throwing out the baby with the bathwater, IMO. Besides, I don’t have the impression that people who post low-effort blog posts are especially concerned with following community rules/etiquette, so they’d probably get posted anyway.

                                        I think a discussion on how we can curtail low-effort self-promotion would be useful, but I’m not sure if anything further can be done about this outside of some drastic limitations which would limit the normal usage of the site too. I think this is one of those “it sucks, but it’s the best we’ve got” kind of things.

                                        1. 4

                                          and my experience is that they’re pretty highly correlated to self-posts.

                                          Do you have any sort of numbers, or is that merely a feeling? If it’s a feeling: mine is that it only correlates to self posts of a few notorious offenders (who get flagged or banned after a while anyway), most others are quite selective of what they share, and most likely it’s content that is not low-effort, even if it doesn’t interest me personally. A blog post every few weeks is certainly fine by me, and I’d consider it harmful to give those people the feeling they shouldn’t share what they do or think about. Don’t let a few bad apples spoil it for the rest of us. After all, there are other safe-guards in place (up-votes, flagging posts, etc…).

                                    1. 1

                                      Anyone know when the phone will be more “traditionally” available? I would love to get one, but it seems every time I check it’s out of stock or pre-order. Almost seems like vaporware.

                                      1. 2

                                        Check the store on September 17th (or 16th possibly, if you are in the US), that’s when the next batch goes up for pre-order. In my experience, as long as you are in ‘few hours’ window of the release of a batch (maybe up to a day, if you are lucky?) you should be able to order one. If you keep an eye on their subreddit I’m pretty sure there’ll be a post soon after the batch goes online.

                                        “traditionally” available might still be a while, in my estimate.

                                      1. 8

                                        Great article! I have one comment on this:

                                        When I am ready to move a task from In Progress to In Review, I close the subtree with tab and then highlight the collapsed subtree, cut it and paste it under the In Review header.

                                        I suspect you know this already, and the “cut and paste” comment is for the benefit of readers that are not familiar with Org mode, but if not it’s your lucky day! If you set (setq org-refile-use-outline-path 'file) you can hit C-c C-w (org-refile) to move a subtree to a different heading in any of your org-agenda-files, which is a bit more streamlined than cut-and-paste. You don’t even have to have the cursor on the heading!

                                        1. 3

                                          Today I learned! I’ll be sure to try this at work today. Thanks!

                                          1. 1

                                            Sweet, thanks for that. Is there a way to re-file a subtree into a non-root element in an org-file?

                                            1. 1

                                              Indeed! https://orgmode.org/org.html#Refile-and-Copy has the details. You can do M-x customize-group RET org-refile RET for all the configurable settings.

                                              1. 1

                                                Cool, thanks, much appreciated!

                                                1. 1

                                                  Only disappointment so far is that I could not get the refile to a new file working yet. Would help refactoring my noteoliths into zettelservices

                                                  1. 2

                                                    I think for that to work you need to customise org-refile-targets and provide a function to prompt for/generate a filename. From the variable’s documentation:

                                                    This is a list of cons cells. Each cell contains:

                                                    • a specification of the files to be considered, either a list of files, or a symbol whose function or variable value will be used to retrieve a file name or a list of file names. […]
                                            1. 1

                                              I think I’ll get back to working on my generic package manager for files (single-file binaries, config files, other categories of files where its currently cumbersome to manage and update versions and metadata associated with them). Finished a working prototype, but want to re-write some of the parts to remove a few pockets of complexity. We’ll see how that goes, usually I have a harder time motivating myself to do work when it’s about re-writing things I already implemented, instead of solving a new problem…

                                              1. 3

                                                I’ve taken a stance of not running any AGPL software, period, and especially not any software that might be usable via the network (triggering the clauses that make the AGPL unique vs the GPL). My reasoning is simple: I’m lazy. I may patch a piece of software to fit my needs, but by doing so, I may patch in secrets or details about my infrastructure that I don’t want public. I’m not keeping the patch proprietary to turn it a business model or anything that goes against the spirit of the GPL; I just don’t find some patches particularly valuable or appropriate for public consumption. The GPL doesn’t cross this line, because it is easy to not redistribute my modifications in binary form (why would I with these patches?), but it can be potentially disastrous to flirt with these modifications in AGPL software.

                                                If I wasn’t so lazy, I may patch the software “correctly”, introducing configuration options which don’t hardcode pieces of my own infrastructure, but I find this unreasonably burdensome for a so-called free software license, and it slows down proof-of-concept patches that I might otherwise want to deploy to canary instances to check for faults that integration and unit tests may not cover.

                                                1. 2

                                                  I don’t have an opinion on your overall argument. Just: ‘unreasonably burdensome for a so-called free software license’. This does not make sense to me, nobody ever said free software licenses are there to reduce burdens. That most of the permissive licenses actually do that is more a side-effect than anything else, and misses the point, IMHO.

                                                1. 31

                                                  The reason they spread these misconceptions is straightforward: they want to discourage people from using the AGPL, because they cannot productize such software effectively.

                                                  This doesn’t stand up to even a modicum of scrutiny. First of all, it assumes you know the intent of Google here. I don’t think Google’s intentions are that great to be honest, but as a rule of thumb, if you form an argument on knowing the intentions of other humans, it’s probably a bad argument unless you can provide credible evidence of their intent. Secondly, I see no such credible evidence in this article, and the lack of attention paid to how Google handles other licenses in this article is borderline disingenuous. All I see is a casual observation that Google’s policy benefits them systemically, which I would absolutely agree with! But that shouldn’t be a surprise to anyone.

                                                  Why? Because it omits critical context. The AGPL is not the only license that Google bans. They also ban the WTFPL, which is about as permissive as it gets. They ban it because they have conservative legal opinions that conclude it has too much risk to rely on. I think those legal opinions are pretty silly personally, although I am somewhat biased because I’ve released code under the WTFPL only to have one Googler after another email me asking me to change the license because it’s banned at Google.

                                                  My point is that there are other reasonable business explanations for banning licenses. Like that a team of lawyers paid to give their best expert advice on how a judge would rule for a particular license might actually, you know, be really risk averse. Licenses aren’t some black and white matter where things that are true and things that are not are cleanly separated in all cases. There’s oodles of grey area largely because a lot of it actually hasn’t been tested in court. Who would have thought the courts would rule the way they did in Google v. Oracle?

                                                  What’s the cost of being wrong and having Google required to publish all of their source code? Can anyone here, even a Googler, even begin to estimate that cost? If you haven’t thought about that, then you probably haven’t thought deeply enough to criticize the intentions on this particular piece of “propaganda.” Because that’s probably what Google’s lawyers are weighing this against. (And probably an assortment of other such things, like the implications of allowing AGPL but giving each such use enough scrutiny as to be sure that it doesn’t wind up costing them dearly.)

                                                  But by all means, continue punishing companies for making their policies like this public. Because that’s a great idea. (No, it’s not. Despite how annoying I find Google’s policies, I really appreciate having them documented like they are.)

                                                  Disclaimer: I don’t like copyleft, but primarily for philosophical reasons.

                                                  1. 11

                                                    I don’t think Google’s intentions are that great to be honest, but as a rule of thumb, if you form an argument on knowing the intentions of other humans, it’s probably a bad argument unless you can provide credible evidence of their intent.

                                                    As someone who previously worked on the open source team at Google and sat in the office and am friends with these humans, I can say very strongly that those lawyers do not have some sort of hidden agenda. It is also certainly false to assume they are not competent at their job. My read is that they are, as you might expect, very good at their job (noting I am also not a lawyer).

                                                    A common mistake I see many commenters (and news stories etc etc) and I think you head to unintentionally, is to talk about Google as if it is a single anthropomorphic entity with its own thoughts and feelings. This piece does the same. There is not “a Google” that is making amoral decisions for its global benefit . There is an office of humans that try their best and have good intentions.

                                                    The team makes decisions in this order:

                                                    1. Protect the open source ecosystem.
                                                    2. Protect the company.

                                                    “Protect the ecosystem” is hard to believe if you buy into the “amoral entity” argument but is provably true: the easiest way to protect the company is to ban open source contribution (aside from forced copyleft terms) at all, but Google does this a lot under the Apache 2 (permissive) license. The banned licenses, as you note, are those that either do not have enough specificity (like WTFPL) or ones with what the legal team believe are onerous terms. They are good laywers, and so you have to assume they have a pretty strong case for their interpretation. Even if you think they are wrong (as all law is essentially malleable), hashing things out in court to decide what the terms of the license truly mean is a really bad use of time and money.

                                                    1. 13

                                                      There is not “a Google” that is making amoral decisions for its global benefit . There is an office of humans that try their best and have good intentions.

                                                      Yes, there is. The two are not mutually exclusive. A corporation like Google is structured in such a way that the sum of all its humans, all trying their best, serves the interests of the company. It’s not anthropomorphic, but it does have an agenda, and it’s not necessarily that of any of its constituent humans. Whether morality features prominently on that agenda is a legitimate matter for debate.

                                                      I think you’re trying to open a semantic crack in which responsibility can be lost: misdeeds are attributed to Google, but since Google isn’t one person it can’t be guilty of anything. But if companies really aren’t more than the sum of their parts, at least one person at Google must be responsible for each of its transgressions, which I think casts doubt on the claim that they have good intentions.

                                                       

                                                      The team makes decisions in this order:

                                                      1. Protect the open source ecosystem.
                                                      2. Protect the company.

                                                      Maybe that’s true of the open source team. It’d be hard to believe that of Google in general—partly because it’s a coompany and you’d expect it to protect itself first, but more concretely because there’s history. Google has been hegemonizing Android for years. They’re also trying to do the same to the Web, via Chrome. The open source ecosystem gets to use whatever Google puts out, or suffer. I don’t see how that’s healthy.

                                                       

                                                      “Protect the ecosystem” is hard to believe if you buy into the “amoral entity” argument but is provably true: the easiest way to protect the company is to ban open source contribution (aside from forced copyleft terms) at all, but Google does this a lot

                                                      (I note that you don’t have a problem anthropomorphizing Google when it’s doing things you think are good.)

                                                      I’ve yet to see the proof. Publishing open source software doesn’t necessarily speak to any commitment to the wellbeing of the open-source ecosystem, nor does it typically carry any great risk. Let’s take a couple of minutes to think of as many reasons as we can why a company might publish open-source software out of self-interest:

                                                      • The existence of good tooling for markets you dominate (web, mobile) directly benefits you
                                                      • Developers like publishing things, so letting them publish things is a cheap way to keep them happy if it doesn’t hurt you too badly
                                                      • It’s great PR
                                                      • If you have a way to use your open-source thing in a way that nobody else does, the free work other people do on it gives you an advantage

                                                      You might say: so what? Obviously they have businessy motivation to care about open source, but what does it matter if the result is they care about open source? But, as we’ve seen, the moment it benefits them to work flat-out on destroying an open ecosystem, they do that instead.

                                                      1. 3

                                                        But, as we’ve seen, the moment it benefits them to work flat-out on destroying an open ecosystem, they do that instead.

                                                        This could be said of nearly any corporation as well.

                                                        Move from OS sales to cloud services, buy an open-source friendly company, release a good editor that works on the competition, and even inter-op with rhe competition.

                                                        The example may have the best intentions in mind, insofar a corporation can, but could also be a long-con for traction and eventually blast out something that makes the users jump ship to the corporation’s platform.

                                                        Best part of it all is, it could be hedging in case that “something” comes along. There is some win either way and an even bigger win if you can throw the ideals under the bus.

                                                        1. 2

                                                          For sure. It’d be naïve to think Microsoft had become nice. They’ve become smarter, and they’ve become a smaller player comparatively, and in their situation it’s pragmatic to be a good citizen. Google was the same with Android before they won their monopoly.

                                                        2. 2

                                                          (I note that you don’t have a problem anthropomorphizing Google when it’s doing things you think are good.)

                                                          It’s easy to do, mistakes were made, I’m human. Don’t assume malice or misdirection.

                                                          1. 5

                                                            I don’t assume either. I think it’s a natural way to communicate about organisations. But your opening gambit was about how talking about Google in those terms betrayed some error of thought, so I’d hoped that pointing this out might give you pause to reconsider that position. I didn’t mean to cast doubt on your sincerity. Apologies.

                                                            1. 2

                                                              All good 👍

                                                        3. 10

                                                          Right, I mostly agree with what you’re saying! I do think a lot of people make the mistake of referring to any large company as a single entity, and it makes generalizing way too easy. With the WTFPL thing, I experienced that first hand: a bunch of individuals at Google reached out to me because none of them knew what the other was doing. And that’s a totally reasonable thing because no large company is one single mind.

                                                          Now, I don’t want to come off like I think Google is some great thing. The WTFPL thing really left a sour taste in my mouth because it also helped me realize just how powerful Google’s policies are from a systemic point of view. They have all these great open source projects and those in turn use other open source projects and so forth. My libraries got caught up in that, as you might imagine in this day and age where projects regularly have hundreds or thousands of dependencies, and Google had very powerful leverage when it came to me relicensing my project. Because it worked itself back up the chain. “{insert google project here} needs to stop using {foo} because {foo} depends on {burntsushi’s code that uses WTFPL}.” Now foo wants to stop using my code too.

                                                          I’m not saying any of this is particularly wrong, to be honest. I am an individualist at heart so I generally regard this sort of thing as okay from an ethical or legal perspective. But still, emotionally, it was jarring.

                                                          Do I think the lawyers in Google’s open source policy office think about that sort of effect it has on individuals? I don’t really. I don’t think many do. It’s probably a third order effect of any particular decision, and so is very hard to reason about. But from my perspective, the line of policy making on Google connects very directly to its impact on me, as an individual.

                                                          In the grand scheme of things, I think this is not really that big of a deal. I’m not all hot and bothered by it. But I do think it’s a nice counter-balance to put out there at least.

                                                          1. 4

                                                            To play devil’s advocate:

                                                            It appears that seasoned lawyers have deemed the license you use “not specific enough”.

                                                            Isn’t the whole point of a license to fully lay out your intentions in legal terms? If it doesn’t succeed at that, wouldn’t it be better to find another license that does a better job at successfully mapping your intentions to law?

                                                            1. 6

                                                              To be clear, I don’t use the WTFPL any more, even though I think it makes my intent perfectly clear. So in a sense, yes, you’re right and I changed my behavior because of it. I stopped using it in large part because of Google’s influence, although the WTFPL didn’t have a great reputation before Google’s policy became more widely known either. But most people didn’t care until Google’s policy influenced them to care. Because in order for my particular problem to exist, some amount of people made the decision to use my project in the first place.

                                                              I brought up the WTFPL thing for two reasons:

                                                              • To demonstrate an example of a license being banned that isn’t copyleft, to show that Google has other reasons for banning licenses than what is stated in the OP.
                                                              • To demonstrate the impact of Google’s policies on me as an individual.

                                                              I didn’t bring it up with the intent to discuss the particulars of the license though. I’m not a lawyer. I just play one on TV.

                                                              1. 2

                                                                But I think even Google’s influence is just one example of the commercial world interacting with the “libre” world; in this light, Google is just entering earlier and/or investing more heavily than its peers. And it could be argued that’s a good thing, as it puts libre creators more in touch with the real needs of industry. It’s the creator’s choice whether to acknowledge and adapt to that influence, or to bend to it entirely. As I see it, Google can’t make you do anything.

                                                                I do hope that Google carves out exceptions for things like Affero though, since I share Drew’s confusion at Google’s claim of incompatibility. I’m in the same boat, after all; I’m also a user of a niche license (License Zero), the legal wording of which I nevertheless have great confidence in.

                                                                I believe that at some point, companies like Google will have to bend to the will of creators to have control over how their work is licensed. I happen to use License Zero because it seems to provide more control on a case-by-case basis, which I think is key to effecting that shift.

                                                                1. 4

                                                                  I do hope that Google carves out exceptions for things like Affero though, since I share Drew’s confusion at Google’s claim of incompatibility.

                                                                  Large parts of Google work in a monorepo in which anything goes if it furthers the mission. The Google licensing site brings up that example of a hypothetical AGPL PostGIS used by Google Maps. In normal environments that wouldn’t be an issue: your code interfaces to PostGIS through interprocess APIs (which still isn’t linking even with the AGPL) and users interact with your code, but not with PostGIS. In the monorepo concept code can quickly be drawn into the same process if it helps any. Or refactored to be used elsewhere. That “elsewhere” then ends up under AGPL rules which could be a problem from a corporate standpoint.

                                                                  It’s a trade-off between that flexibility in dealing with code and having the ability to use AGPL code, and the organizational decision was apparently to favor the flexibility. It can be possible to have both, but that essentially requires having people (probably lawyers) poring over many, many changes to determine if any cross pollination between license regimes took place. Some companies work that way, but Google certainly does not.

                                                                  I believe the issue with WTFPL is different: because it’s so vague my guess is that the open source legal folks at Google would rather see that license disappear completely to protect open source development at large from the potential fallout of it breaking down eventually, while they probably don’t mind that the AGPL exists. At least that’s the vibe I get from reading the Google licensing site.

                                                                  (Disclosure: I work at Google but neither on open source licensing nor with the monorepo. I also don’t speak for the company.)

                                                                  1. 4

                                                                    As I see it, Google can’t make you do anything.

                                                                    Maybe I didn’t express it clearly enough, but as I was writing my comments, I was painfully aware of the possibility that I would imply that Google was making me do something, and tried hard to use words that didn’t imply that. I used words like “influence” instead.

                                                                    And it could be argued that’s a good thing, as it puts libre creators more in touch with the real needs of industry. It’s the creator’s choice whether to acknowledge and adapt to that influence, or to bend to it entirely.

                                                                    Sure… That’s kind of what I was getting at when I wrote this:

                                                                    I’m not saying any of this is particularly wrong, to be honest. I am an individualist at heart so I generally regard this sort of thing as okay from an ethical or legal perspective. But still, emotionally, it was jarring.

                                                                    Anyway, I basically fall into the camp of “dislike all IP.” I’d rather see it abolished completely, for both practical and ideological reasons. Then things like copyleft can’t exist. But, abolishing IP would change a lot, and it’s hard to say how Google (or any company) would behave in such a world.

                                                                    1. 2

                                                                      Anyway, I basically fall into the camp of “dislike all IP.” I’d rather see it abolished completely, for both practical and ideological reasons.

                                                                      Maybe we should turn Google into a worker coop 😉 Then its employees could change IP policy like you say, the same way they successfully protested the deals w/ China & the US military.

                                                            2. 3

                                                              There is not “a Google” that is making amoral decisions for its global benefit . There is an office of humans that try their best and have good intentions.

                                                              Mike Hoye wrote a short article called “The Shape of the Machine” a couple of months ago that examines the incentives of multiple teams in a large company. Each team is doing something that seems good for the world, but when you look at the company as a whole its actions end up being destructive. The company he’s talking about also happens to be Google, although the lesson could apply to any large organization.

                                                              I definitely agree with you that Google has lots of capable, conscientious people who are doing what they think is right. (And to be honest, I haven’t thought about the licensing issue enough to be able to identify whether the same thing is at play here.) I just think it’s good to keep in mind that this by itself is not sufficient for the same to be said for the organization as a whole.

                                                            3. 9

                                                              This is exactly what I came here to say. Basing an argument on your own interpretation of a license is a great way to get into legal trouble. Not only is there the risk that a judge in a court of law may disagree with your interpretation but there is also the risk that you will invite litigation from others that have a different interpretation and disregarding the risk of losing that litigation that litigation has a cost.

                                                              So by using AGPL you incur not only the risk of having the wrong interpretation once it is tested in court but also the risk of an increase in costly litigation over time. This risk is further magnified by your size and how much larger it makes the target on your back.

                                                              1. 12

                                                                Basing an argument on your own interpretation of a license is a great way to get into legal trouble

                                                                The article starts with “I’m not a lawyer; this is for informational purposes only”, and then proceeds to make strong un-nuanced claims about the license and even proceeds to claim that Google’s lawyers are incompetent buffoons and/or lying about their interpretation. Saying you’re not an expert and then pretending you are in the very next sentence is pretty hilarious. It’s abundantly clear this article is to support the author’s politics, rather than examine legal details.

                                                                1. 6

                                                                  I’m not a lawyer; this is for informational purposes only

                                                                  I believe that Americans write that type of disclaimer because it is illegal over there to practice law without a license, and articles about software licenses can easily wander into dangerous territory. So based on that, I think it’s unfair to hold that up as a point against the article.

                                                                  Disclaimer: I’m not a lawyer; this is for informational purposes only.

                                                                  1. 1

                                                                    I started to call that tactic ‘joe-roganizing’. He does the same: “I don’t know anything about this.”, Then, in the next sentence: ‘[very strong opinion] - everyone who disagrees is surely stupid….’

                                                                2. 9

                                                                  I worked at a startup where we had a massive compliance burden (yay FDA!) and so had even fewer resources than usual. One of my jobs as engineering lead there was to go and audit the tools and source that we were using and set guidelines around what licenses were acceptable because we could not afford the lawyer time if there were any issues.

                                                                  If the AGPL had been tested in court, I think companies would be a bit more chill about it, but I reckon that nobody wants to bankroll a legal exploration that could turn out very much not in their favor.

                                                                  One of the annoying things too about licensing, especially with networked systems and cloud stuff, is that the old reliable licenses everybody basically understands (mostly) like BSD and MIT and GPL and LGPL were made in a (better) world where users ran the software on machines they owned instead of interacting with services elsewhere. We still haven’t really identified an ontology for how to treat licensing for composed services on a network, versus how to handle services that provide aggregate statistics for internal use but not for end users, versus dumbly storing user data, versus transforming user data for user consumption.

                                                                  1. 4

                                                                    What’s the cost of being wrong and having Google required to publish all of their source code?

                                                                    That’s not how the AGPL works.

                                                                    The AGPL does not force you to distribute anything.

                                                                    If they’re “wrong”, they are in breach of contract. That’s it. They can then remedy that breach either by ceasing use of that software or by distributing their changes, or even by coming to some alternative agreement with the copyright holders of the AGPL’d software in question.

                                                                    1. 2

                                                                      This seems like a nit-pick. The point of my question was to provoke thought in the reader about the costs of violating the license. What are those costs? Can you say with certainty that they will be small? I’m pretty sure you’d need to be a lawyer to fully understand the extent here, which was my way of saying, “give deference where it’s due.”

                                                                      I personally think your comment is trying to minimize what the potential costs could be, but this isn’t theoretical. Oracle v. Google is a real world copyright case that has been going on for years and has almost certainly been extremely costly. I don’t see any reason why an AGPL violation couldn’t end up in the same situation.

                                                                      1. 4

                                                                        It’s an actual misconception that many people have, and I don’t think it’s good to perpetuate it.

                                                                        1. 2

                                                                          I guess that’s fair, but it seems like splitting hairs to me. Even you said “distributing their changes” as a possible remedy, and there’s a fine line between that and “publish all of their source code.” It really depends on how the law and license is interpreted, and nobody knows how it will be. So lawyers guess and they guess conservatively.

                                                                          1. 0

                                                                            The easiest way not to perpetuate it is to not use the AGPL.

                                                                      2. 3

                                                                        Thanks for saying this. I don’t work at Google, but I know many people who work at it and other large companies and have talked with them about license policy, and the article just reeks of ignorance as to how corporate lawyers work; even for relatively small companies.

                                                                        There’s no ideology here, there’s just lawyers doing what they were hired to do: use an abundance of caution to give the company as ironclad a position as possible.


                                                                        Hell, forget WTFPL, I’ve been waved off considering triple licensing of (approved) licenses by Googlers as “the lawyers would never go for this”. The lawyers are going to go for well understood, battle tested licenses where the failure cases aren’t catastrophic.


                                                                        Besides that it seems like the article misunderstands what constitutes a “derivative work”, if the article’s definition of “derivative work” (i.e., the code must be modified, not simply “used as a dependency”) was the one used by the *GPL licenses, then there would be no need for LGPL to exist.

                                                                        1. 1

                                                                          but as a rule of thumb, if you form an argument on knowing the intentions of other humans, it’s probably a bad argument

                                                                          This is not true.

                                                                          Firstly, the rule for another person and the rule for CORPORATIONS are completely different. Corporations do not operate like people do. When corporations are small, they sort of do, but as they grow larger then they become more corporations like.

                                                                          Secondly, it is impossible to know the intentions of other humans. So by this argument, no argument is ever good.

                                                                          We might give people the benefit of the doubt, because people are mostly good. They are ruled by an ethical system, built into their brain, to socialise and cooperate. Corporations do not have this internal system. Their motivational system is entirely profit based, and therefore you cannot treat them like people.

                                                                          If you have been alive long enough and paid attention to what corporations do, and especially google, the idea that they consider AGPL hostile, and wish to limit its influence and expansion, is highly plausible. How will they limit its influence? They could ban it completely, and then publish a document detailing why they think it’s bad. That’s highly plausible.

                                                                          Is risk-averse lawyering a factor? Most likely yes. But risk-averse lawyer adds to the hostility argument. Having received the advice from lawyers to not use AGPL, leadership would easily conclude that a limit to AGPL spread would give them the best chance of getting free software and have their way.

                                                                          Additionally, your steelman argument does not explain why google publishes that they do not like AGPL. They could keep it entirely internal. Why do you think they would do that? Free legal advice to competing startups?

                                                                          1. 3

                                                                            Firstly, the rule for another person and the rule for CORPORATIONS are completely different. Corporations do not operate like people do. When corporations are small, they sort of do, but as they grow larger then they become more corporations like.

                                                                            That makes sense in a certain light, sure. But I don’t see what it has to do with my point.

                                                                            Secondly, it is impossible to know the intentions of other humans. So by this argument, no argument is ever good.

                                                                            I don’t really agree. It might be true in the strictest philosophical sense, but that needn’t be our standard here. Intent is clearly something that we as a society have judged to be knowable to an extent, at least beyond some reasonable doubt. Just look at the criteria for being convicted of murder. It requires demonstrating something about the intent of someone else.

                                                                            Why do you think they would do that?

                                                                            When was the last time you saw any company publish legal advice generated by internal review?

                                                                            If you have been alive long enough and paid attention to what corporations do, and especially google, the idea that they consider AGPL hostile, and wish to limit its influence and expansion, is highly plausible. How will they limit its influence? They could ban it completely, and then publish a document detailing why they think it’s bad. That’s highly plausible.

                                                                            I think you’ve really missed my point. If the OP were an article discussing the plausibility of one of any number of reasons why Google published an anti-AGPL policy, then I would almost certainly retract my comment. But that’s not what it was. It’s a one sided turd without any consideration of alternative perspectives or explanations at all.

                                                                        1. 5

                                                                          I like python, I even like javascript but java is just “easy”. Strong, static typing saves me so much time and mental effort it’s phenomenal. I tried becoming more of frontend focused “full stack” developer for a few years but I found I worked far harder, expended more mental effort with dynamic languages than I ever did with java. Of course javascript isn’t python but I experienced that same thing with a python application as well. I tend to work more on extending and maintaining existing applications than green field development. Even then if one takes the time to learn Spring, it’s really really good to.

                                                                          I’m at the point in my life where “easy” counts for a lot. And as the fine article says, java is pretty nice nowadays. The tooling is exceptional. I’m glad for it.

                                                                          1. 1

                                                                            I heard Python can do static types with an extension or something. Found this that shows how. Also claims PyCharm IDE catches the type errors for you. Some info might be out of date since it’s from 2017.

                                                                            Maybe try some of that tooling with Python to see if it gets better.

                                                                            1. 3

                                                                              Yes, type checking with mypy is quite good nowadays, and provides quite a bit of peace of mind. It’s not perfect, and it’s nowhere near the level of what static types in Java give you, but it’s a good middle-ground, and it improves the Python dev experience noticeably. You loose some of the readability of Python (if you are not used to type hints), but I think it’s worth it, esp. for libraries.

                                                                          1. 3

                                                                            I’m a happy user of Ansible, and despite having hundreds of VMs to take care of in the past, I’ve never felt any pain about it. To keep the desired state, Ansible scripts where running hourly with Jenkins, and it worked well.

                                                                            I cannot compare with Chef or Puppet that I’ve never used, anybody have more experience on the limitations mentioned?

                                                                            1. 3

                                                                              Yeah, the recommendations and justifications in that article seem a bit random and biased. Ignoring Ansible and it’s ease of use and relatively quick learning curve (compared to Puppet esp), and requiring a pull model when the premise is to find ‘good enough’ tools doesn’t make a whole lot of sense to me (as do non of those generic recommendation/advice articles like this one in the first place).

                                                                              The bit about Kubernetes and using config management instead is silly.

                                                                            1. 1

                                                                              I’ve been working on and off on a client/server version of this, with a persistent daemon process taking care of the database. I feel that would be a better design, as the work performed by the client after each key press would be really minimal. That daemon could also eventually be used for automating other parts of the CLI experience.

                                                                              1. 1

                                                                                I did consider this, but I didn’t want an entire process running on my computer for the sole purpose of serving me cd suggestions. Moreover, given how fast zoxide already is, I doubt that a client-server model would be a massive improvement.

                                                                                1. 2

                                                                                  You’re right, I’m sure it’s fast enough. The main point would be to add more features in addition to jumping: a central daemon that talks to all your shells simultaneously could certainly help for many other things.

                                                                                2. 1

                                                                                  Yeah, I’ve thought about this as well. Just couldn’t be bothered to figure out all the possible security implications, and never wanted it bad enough. But it could be the sort of generic project that opens up a lot of possibilities we don’t even see yet… Also, who doesn’t love over-engineering stuff ? :-)

                                                                                  1. 1

                                                                                    Right now I was thinking client/server on the same machine, using a UNIX socket for communication, so the security implications aren’t big. But maybe you’re right, and we could imagine a cloud-based version!

                                                                                    I’ll try to ping you if I ever put something on GitHub :)

                                                                                    1. 2

                                                                                      Yeah, please do. I shudder to think what the reaction of some folks on HN or here or reddit would be if you announce ‘I made a client-server version of cd’! :-)

                                                                                      1. 1

                                                                                        Lol, I’m old enough not to care anymore :)

                                                                                1. 4

                                                                                  It mentions it’s inspired by z and z.lua. There’s also jump or autojump, or twenty more if you search a bit. “It’s in Rust” is the big helper here, I suppose. Jumping to directories is never something I’ve found noticably slow even with z, but there’s always an audience trying to eek out the best performance.

                                                                                  1. 10

                                                                                    zoxide’s README says it’s mainly trying to speed up displaying the prompt, as opposed to speeding up the actual jumping. All z-like tools delay the prompt a bit because they run code to track the current directory before each prompt.

                                                                                    1. 2

                                                                                      Ahh, that makes more sense! I used to run some pre-prompt display stuff to get git statuses added into my prompt and that definitely slowed things down. Now I just run a ‘git status’ when I actually want that info and things are snappier. Every little bit adds up on those things.

                                                                                      1. 4

                                                                                        If you’d still like to do that (and use zsh), check out powerlevel10k. You can have your cake and eat it too…

                                                                                  1. 6

                                                                                    Is this solving a problem that doesn’t exist?

                                                                                    zsh’s autocomplete scrolls through only those previously used commands that match what has been typed so far. So I typically enter cd ./s and then the up arrow takes me through my most recently visited project folders, ./src/project_1, ./src/project_2

                                                                                    Even better, it works on any command, not just cd.

                                                                                    1. 5

                                                                                      History prefix search is a useful feature, but it can’t fully replace tools like z and zoxide. Say your two most recent projects are ~/projects/src/rails/ and ~/projects/src/lobsters/. If you want to jump to lobsters from within ~/projects/, you can type z lob or potentially even z l, both of which are faster to type than cd ./s<up><up>.

                                                                                      What’s more, you can always use the same z lob and z rails to jump to those folders from anywhere, whereas the cd command would have to be adjusted if you were within ~ or within rails/. The cd command also requires a different amount of <up> presses depending on which folder you visited most recently, while the z commands can be done from muscle memory.

                                                                                      1. 2

                                                                                        This usually does not work if your current directory is not the same as the one when you typed the command, except you use absolute paths. Or maybe I misunderstand what you exactly do.

                                                                                        Anyway, try it out, I wouldn’t want to miss this kind of instant directory change. Usually, it only takes 4 key presses ‘z [space] [two letters of target dir name]’ and you’re in the target dir - no need to reach for arrow keys or ctrl.

                                                                                        So, yes, it does solve a problem that does exists. Maybe it’s not a problem for you. Or maybe you don’t know yet you have that problem, but will once you tried out this (or z, or autojump, or any of the other solutions – I reckon the fact that there are so many alternative implementations should be a pretty good indicator that a non-trivial amount of people think this is a problem…).

                                                                                      1. 8

                                                                                        The build time and research arguments are valid and definitely good to keep in mind. However, you can make smaller alpine images by following similar patterns. The Dockerfiles in this article don’t use –no-cache with apk and they leave around the development files after the build is done.

                                                                                        The following image should build something similar, though, as mentioned, it takes a while to build. I’ll update this post with the final image size when it’s done.

                                                                                        FROM python:3.8-alpine
                                                                                        
                                                                                        ENV PYTHONUNBUFFERED=1 \
                                                                                          PYTHONDONTWRITEBYTECODE=1 \
                                                                                          PYTHONHASHSEED=random \
                                                                                          PIP_NO_CACHE_DIR=off \
                                                                                          PIP_DISABLE_PIP_VERSION_CHECK=on
                                                                                        
                                                                                        RUN apk add --no-cache freetype libpng openblas
                                                                                        
                                                                                        RUN apk add --no-cache --virtual .build-deps gcc build-base freetype-dev libpng-dev openblas-dev \
                                                                                            && pip install matplotlib pandas \
                                                                                            && apk del --no-cache .build-deps
                                                                                        

                                                                                        EDIT: strangely, the build wasn’t also made slowed by untaring matplotlib-3.1.2.tar.gz which is a 40MB file with lots of small files. That’s not to say the build was fast, but it’s worth noting.

                                                                                        In any sense, the final build size as reported by docker image ls was 469MB.

                                                                                        1. 3

                                                                                          This does result in a smaller image, but it means every time you change your Python or APK dependencies you need to reinstall both, without relying on Docker layer caching.

                                                                                          The image size honestly isn’t a big deal, but the build time is brutal.

                                                                                          1. 3

                                                                                            So, I finally got back the final sizes and it actually surprised me. 469MB for the alpine version I posted. Much better than your 851MB, but also larger than python:3.8-slim. Maybe it’s leaving around the source files somewhere? Have you made sure that your python:3.8-slim version can actually run code using matplotlib or pandas? I’d assume that they’re missing the actual libraries needed to run the code (essentially the non-dev versions of what you had to install with alpine).

                                                                                            At this point, I’m not really willing to take more time to investigate this - you’ve sold me. I’m planning on moving all my images to the slim variant if I can.

                                                                                            1. 3

                                                                                              I’ve wasted a lot of time on this as well, and I can only recommend you do that. So much less time spent in Dockerfiles, and more in dot-py-ones…

                                                                                              I ended up with a Debian-based base image with the build dependencies installed and often used dependencies in a pre-created virtual-env, an Ansible-based toolchain to be able to quickly install any additional system and virtualenv python-package dependencies (which will most likely be overkill for most), and a script that is called as last step in every child-Dockerfile and which cleans up most of the junk that is only used at build time. You could probably also do a lot of that with a multi-stage Docker build.

                                                                                              Anyway, that makes for fairly quick build times, and quite small images. The base image might not be that small, but since it doesn’t chage that often, and all your other images depend on it and can re-use the cached layer, I only have to pay that cost once (per deployed project), and when it updates…