I’ve made a ton of progress since I wrote this post on Sunday. Most crucially, I’ve settled on a way of bundling Python that looks like it might work well: I’m now including a full Python 3.9 (using the standalone Python project) inside the macOS application bundle, then on first startup of the app I use that to create a virtual environment in ~/.datasette-app/venv into which I then install the Python web application.
Crucially, this let me build an “install plugin” mine item which can install additional code into that virtual environment.
More on this Twitter thread: https://twitter.com/simonw/status/1432871047691333634
Next step: figure out how to sign it all!
As one of your downstream maintainers in nixpkgs, I’m fascinated and horrified in equal parts by how much bundling you’ve done. We undo some bundling when possible, like Python-specific package management, so nixpkgs has to separately package each plugin. (Edit: And I just packaged another one.)
But your attention to detail pays off; the bundles work very well. I just used datasette-publish-fly to publish a database on Fly. Within the bundling mechanism of Docker and pip, everything worked as documented, even on NixOS.
I’m obligated somewhat to point out that nixpkgs has a very nice mechanism for bundling declarative multi-language packages into single outputs, from dockerTools to nix-bundle. It would be a lot of effort to rewrite what you’ve already built, but it might be worth examining anyway.
This is great - I hadn’t seen the nix efforts before! Thanks so much for working on this.
My goal with the Datasette desktop app stuff I’m working on is pretty much exclusively to solve the “I use macOS or windows and I want to use this software without having to learn any alternative form of packaging - or even what that word means” - the bundle I’m producing is pretty enormous (with both a full Python and a full Electron/Node/Chromium) so I fully expect more advanced users to pick alternative installation methods!
I should figure out how to use Nix myself and get that added to the documentation at https://docs.datasette.io/en/stable/installation.html