I’m a little confused by what it does. It gives a Rust wrapper library for embedding CPython and also gives a single static binary for distributing CPython (wrapped in it’s Rust library). So the primary benefits are easier interop with Rust programs and easier distribution because it’s a single binary to get “CPython”.
Do I have that right?
This is approximately my understanding. The related PyOxidizer project seems to be like PyInstaller except that its built on Rust infrastructure and has some other tradeoffs (in-memory loading of packages vs. unpacking to temp directory?). I think I could use another blog post, etc., to better explain when/why I’d reach for this toolset.
In my understanding PyOxidizer is an “umbrella” project featuring many of what you’ve described, plus probably much more.
However, strictly speaking of pyoxy, what the article describes it seems to serve the following use-case:
Basically if before I wasn’t too comfortable writing operations scripts in Python, as opposed to bash, because I never knew which turn the Python on my distribution would take, now I can have the pyoxy stored alongside my scripts, and be sure they’ll keep on running as long as I need them, without needlessly breaking due to Python upgrades.
Besides having the Python runtime (with pyoxy run-python), it also allows the user to run pyoxy run-yaml, where the yaml stems from the fact that the user can provide a custom YAML file to instruct the interpreter how it should be set-up. For example I usually use #!/usr/bin/env -S python2.7 -u -O -O -B -E -S -s -R in my scripts, and this always leaves a mess in ps and htop; with the new feature I would use something like #!/usr/bin/env pyoxy run-yaml (see the article), and the process tree would be much cleaner, and also I would have more control over the interpreter setup.
#!/usr/bin/env -S python2.7 -u -O -O -B -E -S -s -R
#!/usr/bin/env pyoxy run-yaml
Fully statically linked Linux binaries are possible. PyOxidizer supports it. But you can’t load compiled extension modules (.so files) from static binaries. Since I was initially going for a drop-in replacement for “python, I didn’t want to sacrifice ecosystem compatibility.
The glibc based builds I believe conform to Python’s manylinux2014 platform level and are about as portable as you can get before going fully static. https://pyoxidizer.readthedocs.io/en/latest/pyoxy_installing.html#linux
You could perhaps provide two kinds of builds, at least on Linux, one dynamic as the current one, and one static for those that need maximum portability but are OK with the limitation of running only Python native code.
All in all, great work!
As a pyenv fan, a single binary multi platform python distribution seems awesome. Is it statically linked?
Speaking of pyenv, would be cool if the python “payload” could be feed separately to the pyoxy binary. Then, if I wanted multiple python versions (historically, a pain in the ass to get on Linux and Mac), I could have one pyoxy binary and multiple python “bundles”.
I guess I could also just get pyoxys with different python versions embedded, but that feels less “neat” for some reason?
Anyway, cool stuff.
It seems this use-case (having pyoxy as separate of the user’s bundled Python code) you are describing is on the roadmap.
However even as it is right now I think it is able to run a zipapp (i.e. a Zip file that contains a __main__.py file in the root of the archive): pyoxy run-python ./app.zip
pyoxy run-python ./app.zip
Regarding the static linking, the Linux binary is not statically linked, however it needs nothing more than Linux’s glibc, thus you can easily copy-paste it between any “recent” Linux distribution without issues:
>> ldd /tmp/pyoxy
libdl.so.2 => /lib64/libdl.so.2 (0x00007f68e1177000)
libm.so.6 => /lib64/libm.so.6 (0x00007f68e108f000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f68e108a000)
librt.so.1 => /lib64/librt.so.1 (0x00007f68e1083000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f68e107e000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f68e105d000)
libc.so.6 => /lib64/libc.so.6 (0x00007f68e0e2e000)
I think it is possible to have it statically linked (at least on Linux), but given the above I think it’s enough for a first release. :)