Does hermes require $HERMES_STORE be consistent across machines to take advantage of caching?
Nix lets you change where the store is located, but nobody ever does it because you lose the enormous community cache at cache.nixos.org. Tons of those binaries have hard-coded paths to their dependencies with the /nix/store/... prefix, which affects their hashes.
I was wondering about this as well. Both Nix and Hermes advertise installation in addition to a system package manager. This especially comes in handy if you’re on a system where you don’t have root access, but then you can’t create a store at the standard location and thus have to build everything from source. This often takes more time than just building the stuff you need manually and linking to system libraries.
I suppose absolute paths (usually into /usr/lib, /usr/share and so on) are very common. I believe AppImages enforce binary-relative paths, which might work here as well, but would mean lots of extra work with packaging. Detecting absolute paths is easy, but patching them out is not.
This often takes more time than just building the stuff you need manually and linking to system libraries.
In the medium term I want to make it easy for someone to get access to a remote build on extremely powerful build machine, currently google is offering 96 cores for cheap at spot prices. These could potentially help for such situations.
For me the most expensive hermes package (gcc) builds in about 4 minutes on my desktop. It is definitely an annoyance at times I want to solve.
I also want to setup a way to export hermes packages as appimages that can work at any path.
Its not fundamentally tied to a linux kernel, but for now that is the only OS with a package set, and the sandboxing mechanisms would need to be altered/rewritten for different platforms.
That being said, I was previously a freebsd user and openbsd user so have quite a lot of interest in making it work, though perhaps not the resources.
More or less, right now seed.hpkg contains a statically linked gcc + a statically linked busybox, everything starts from those two. Hermes is able to build the seed again via core.hpkg and seed_out.hpkg.
Cool! keep in mind its only existed for ~ 3 months, so definitely don’t expect everything Nix or Guix can do yet. Everything in the the quickstart should work, we are still working out how to best write packages as there is a lot of design space to explore.
The builder function for a package is marshalled and passed to hermes-builder, which is run underneath an instance of linux-namespace-container. When the thunk above runs, it issues setuid to a build user and enters a chroot from within the linux namespace, before executing the build function.
Currently package builds never get direct network access, as it gets put into an empty network namespace, and instead proxys all ‘fetch’ requests via a unix socket.
There is a ticket open for kvm based build isolation (probably using qemu), and also a ticket for refactoring this code.
It should be made clear during the collection of package definitions, there is no isolation, this is something I have been thinking about quite a lot, and what to do about it.
And do you overlay the filesystem in case of the install process misbehaving, like for instance ignoring the $PREFIX? For instance, building Janet with make PREFIX=... may install stuff outside of the prefix, for instance man pages if you happen to have $MANPATH in your env, or a temp file in /etc/ld.so.cache~ for some reason. That’s one of the big thing that’s annoying when building: the build process might not be clean and might generate artifacts outside of the “agreed sandbox” (ie. PREFIX), and some of them might be transient (ie. the linker cache above), so building may have side effects on your system.
Currently with ‘multi user’ mode builds they are performed within a chroot without access to the host system. This will be extended to ‘single user’ package stores. I plan to extend the sandboxing further using virtual machines eventually.
Ooh, Janet’s looking better and better these days. It might just become my go-to extension and prototyping language. Hermes, as a project, also seems like a great way to explore it further.
Does
hermesrequire$HERMES_STOREbe consistent across machines to take advantage of caching?Nix lets you change where the store is located, but nobody ever does it because you lose the enormous community cache at
cache.nixos.org. Tons of those binaries have hard-coded paths to their dependencies with the/nix/store/...prefix, which affects their hashes.This is one limitation that is shared with Nixos. For now I am building all software myself as there is not that much of it.
I was wondering about this as well. Both Nix and Hermes advertise installation in addition to a system package manager. This especially comes in handy if you’re on a system where you don’t have root access, but then you can’t create a store at the standard location and thus have to build everything from source. This often takes more time than just building the stuff you need manually and linking to system libraries.
I suppose absolute paths (usually into
/usr/lib,/usr/shareand so on) are very common. I believe AppImages enforce binary-relative paths, which might work here as well, but would mean lots of extra work with packaging. Detecting absolute paths is easy, but patching them out is not.In the medium term I want to make it easy for someone to get access to a remote build on extremely powerful build machine, currently google is offering 96 cores for cheap at spot prices. These could potentially help for such situations.
For me the most expensive hermes package (gcc) builds in about 4 minutes on my desktop. It is definitely an annoyance at times I want to solve.
I also want to setup a way to export hermes packages as appimages that can work at any path.
I have only glanced at it, and interested in trying it out. Maybe this weekend.
Is it tied to GNU/Linux, or can one use it on BSDs as well ?
Thanks!
Its not fundamentally tied to a linux kernel, but for now that is the only OS with a package set, and the sandboxing mechanisms would need to be altered/rewritten for different platforms.
That being said, I was previously a freebsd user and openbsd user so have quite a lot of interest in making it work, though perhaps not the resources.
Thanks, that’s reassuring. I guess core.hpkg is
hpkgs’stdenv. So on a new platform, need to make sure that bootstraps ?More or less, right now seed.hpkg contains a statically linked gcc + a statically linked busybox, everything starts from those two. Hermes is able to build the seed again via core.hpkg and seed_out.hpkg.
Yes! There are a lot of good things in there, but IMHO the no-background-daemon feature is the killer feature.
I’ll definitely give it a try :) .
Cool! keep in mind its only existed for ~ 3 months, so definitely don’t expect everything Nix or Guix can do yet. Everything in the the quickstart should work, we are still working out how to best write packages as there is a lot of design space to explore.
Could you explain how and what sandboxing happens. It looks like the meat of it happens here, or am I missing something?
Forgive the probably ugliest part of the code in the whole project, but:
https://github.com/andrewchambers/hermes/blob/d5f5c387126134708ce3a7587b054e4b1e60f248/src/pkgstore.janet#L556
The builder function for a package is marshalled and passed to
hermes-builder, which is run underneath an instance of linux-namespace-container. When the thunk above runs, it issues setuid to a build user and enters a chroot from within the linux namespace, before executing the build function.You can see the invocation here:
https://github.com/andrewchambers/hermes/blob/d5f5c387126134708ce3a7587b054e4b1e60f248/src/pkgstore.janet#L583
Currently package builds never get direct network access, as it gets put into an empty network namespace, and instead proxys all ‘fetch’ requests via a unix socket.
There is a ticket open for kvm based build isolation (probably using qemu), and also a ticket for refactoring this code.
It should be made clear during the collection of package definitions, there is no isolation, this is something I have been thinking about quite a lot, and what to do about it.
Interesting. Thank you :) .
And do you overlay the filesystem in case of the install process misbehaving, like for instance ignoring the
$PREFIX? For instance, building Janet withmake PREFIX=...may install stuff outside of the prefix, for instance man pages if you happen to have$MANPATHin your env, or a temp file in/etc/ld.so.cache~for some reason. That’s one of the big thing that’s annoying when building: the build process might not be clean and might generate artifacts outside of the “agreed sandbox” (ie. PREFIX), and some of them might be transient (ie. the linker cache above), so building may have side effects on your system.Currently with ‘multi user’ mode builds they are performed within a chroot without access to the host system. This will be extended to ‘single user’ package stores. I plan to extend the sandboxing further using virtual machines eventually.
You have made significant development. I am sure it will also help the Janet community. Thank you!
How does it compare with pkgsrc, or spack?
pkgsrc: https://www.pkgsrc.org
spack: https://spack.io
Ooh, Janet’s looking better and better these days. It might just become my go-to extension and prototyping language. Hermes, as a project, also seems like a great way to explore it further.
Facebook also recently introduced Hermes for React Native: https://reactnative.dev/docs/hermes
Oh well, I hope they don’t take the andrewchambers part next.
It’d be nice to have a little more clue what Hermes is before clicking.
I will make sure to include more info next time I post something.