TIL about the Landlock LSM. This intersects a little with something I hacked up a while ago to use strace to determine actual build deps. I’d played with using cgroups and eBPF to try to make something more efficient (strace has significant overhead) but never got something too compelling. Something always needed root which harms the developer experience of “just run this command”.
I wonder if I can put Landlock in a mode that logs violations rather than blocks them…
Hah. I had that same idea once, of extracting build graphs using strace. I whipped up a tool here https://github.com/jart/cosmopolitan/blob/master/tool/build/pstrace.c which turns strace output into a pythonic data structure. I got pretty far but ended up losing interest in the problem, because ptrace() on Linux is very hard. It’d be great if we could do something like that to distill some order from giant hairballs like Yocto Project. But honestly I’d prefer to just scrap it and start over. Life’s too short.
Hacked to pieces. First thing I did was sit down and delete all 200,000 lines of Vax/Os2/Dos/Amiga/etc. code from GNU Make. Thus giving me a pleasant relaxing space to think and innovate. Had I not done this, it would have been too frustrating and I’d’ve likely given up before completing my mission of bringing sandboxing to Make. Now that the deed is done, folks can use my fork. That’ll help motivate GNU to upstream what we did, so that everyone can benefit.
I remember trying to get gdb building on Solaris ~20yrs ago and having to dig through incomprehensible infinite ifdefs. Makes one appreciate the value of autoconf.. sort of…
I don’t know if you’ve seen it; with LD_PRELOAD it’s possible to load a dynamic library into a binary and override glibc calls with your own version. This approach is much faster than strace but doesn’t work on all binaries (eg: statically-linked ones). See fsatrace and the Shake build system.
That’s a very smart thing to point out. It’d be perfect for my pledge.com command (https://justine.lol/pledge/) since that means we’d no longer need to implicitly pledge("exec"). I love this, since it feels like exploiting something many folks (e.g. the Musl Libc author) view as a security weakness, and instead applying it to the noble virtuous task of giving us better security. I think I might start working on that. If I manage to get it working, then I might consider doing it for Musl and static binaries too, possibly by using Intel Xed to rewrite the binaries. It’s harder to compromise an executable this way, but it’s still possible to do with the proper motivation.
Does macOS support arbitrary sandboxing or just what Apple uses for their own lockdown stuff? Presumably there’s been something for Chromium to use but Landlock seems pretty groundbreaking.
Setting up new user namespaces is quite expensive. Somewhere in the Nix repo, Eelco did some benchmarks and has numbers in the order of 100ms per build sandbox. I’m pretty sure the Landlock approach of blacklisting syscalls for a process is much cheaper.
Using Landlock is probably not super useful to Nix because it’s less portable. Also, Nix has similar overheads as Bazel in terms of copying files, hashing them, … which makes it inherently slower. If the 100ms overhead is per package, it doesn’t amount to a lot compared to if it’s for each .c file.
IMO both approaches are complementary. Use Nix to pull third-party dependencies and declare your build environment. And then use a lightweight sandbox for in-repo incremental rebuilds.
This got me excited, but man sandbox-exec tells me this :(
DESCRIPTION
The sandbox-exec command is DEPRECATED. Developers who wish to
sandbox an app should instead adopt the App Sandbox feature
described in the App Sandbox Design Guide. The sandbox-exec
Due to the fact that all the complexity of sandboxing is now being abstracted by the Linux Kernel, all that I needed to do was add about 200 lines of code to the GNU Make codebase. No root, no mounts, no chroot, no cgroups, and especially no Docker required! All you have to do is issue a system call that tells the kernel which paths should be accessible.
What’s ironic is that Nix does configure a chroot, mounts, etc. in just 200 lines of code:
But Nix requires root for sandbox builds (end-user gets this usually via nix-daemon) and it does seem like Landlock is a bit more simple, so I think I see some clear benefits.
I just pulled the ninja source last night and was refamiliarizing myself with it. It didn’t (after 10 mins of looking around) like it would be all that easy. I’m not sure that ninja rules actually have everything you need to know to sandbox well. I may look at Gn too to see if the logic can be added there.
I’m toying with extending Subprocess with some methods to instruct what to unveil. From where it’s ran in graph.cc, Edge is accessible and I hopeEdge::CollectInputs will give the right paths…
There is a sandbox-exec tool that does what it says, apparently. Sources say it’s been labeled as deprecated but still works and is used by a lot of system services.
TIL about .EXTRA_PREREQS. I mean landlock make sounds great too and I’ll play around with it, but I can put .EXTRA_PREQS into production and start stripping out $(filter-out ...) clutter in Makefiles today!
I really hope to see this get upstreamed someday. I currently depend a lot on remake, but sandboxing would be a great tool to have too. When all the fun tools are their own forks it makes it really hard to use both feature sets!
TIL about the Landlock LSM. This intersects a little with something I hacked up a while ago to use strace to determine actual build deps. I’d played with using cgroups and eBPF to try to make something more efficient (strace has significant overhead) but never got something too compelling. Something always needed root which harms the developer experience of “just run this command”.
I wonder if I can put Landlock in a mode that logs violations rather than blocks them…
Hah. I had that same idea once, of extracting build graphs using strace. I whipped up a tool here https://github.com/jart/cosmopolitan/blob/master/tool/build/pstrace.c which turns strace output into a pythonic data structure. I got pretty far but ended up losing interest in the problem, because ptrace() on Linux is very hard. It’d be great if we could do something like that to distill some order from giant hairballs like Yocto Project. But honestly I’d prefer to just scrap it and start over. Life’s too short.
“life’s too short” said the lady hacking on gnu make…
I attempted using ptrace before giving up and just parsing strace output. That turned out to be the right layer for a pragmatic hack, for me at least.
Hacked to pieces. First thing I did was sit down and delete all 200,000 lines of Vax/Os2/Dos/Amiga/etc. code from GNU Make. Thus giving me a pleasant relaxing space to think and innovate. Had I not done this, it would have been too frustrating and I’d’ve likely given up before completing my mission of bringing sandboxing to Make. Now that the deed is done, folks can use my fork. That’ll help motivate GNU to upstream what we did, so that everyone can benefit.
shudder
I remember trying to get gdb building on Solaris ~20yrs ago and having to dig through incomprehensible infinite ifdefs. Makes one appreciate the value of autoconf.. sort of…
Very cool.
I don’t know if you’ve seen it; with LD_PRELOAD it’s possible to load a dynamic library into a binary and override glibc calls with your own version. This approach is much faster than strace but doesn’t work on all binaries (eg: statically-linked ones). See fsatrace and the Shake build system.
That’s a very smart thing to point out. It’d be perfect for my pledge.com command (https://justine.lol/pledge/) since that means we’d no longer need to implicitly
pledge("exec"). I love this, since it feels like exploiting something many folks (e.g. the Musl Libc author) view as a security weakness, and instead applying it to the noble virtuous task of giving us better security. I think I might start working on that. If I manage to get it working, then I might consider doing it for Musl and static binaries too, possibly by using Intel Xed to rewrite the binaries. It’s harder to compromise an executable this way, but it’s still possible to do with the proper motivation.The build for the day job has some load bearing Go programs in it so that steered me away from the libc interception approach.
Very cool! I hope someone will port the sandboxing code to macOS.
Does macOS support arbitrary sandboxing or just what Apple uses for their own lockdown stuff? Presumably there’s been something for Chromium to use but Landlock seems pretty groundbreaking.
macOS has
sandbox-exec:https://jmmv.dev/2019/11/macos-sandbox-exec.html
And it’s is used in Nix. Nix runs builds within user namespaces on Linux and this is abstracted behind the
sandboxconfiguration option.I’m not sure what Landlock provides above namespacing for things like sandboxes but I’d love to understand.
Setting up new user namespaces is quite expensive. Somewhere in the Nix repo, Eelco did some benchmarks and has numbers in the order of 100ms per build sandbox. I’m pretty sure the Landlock approach of blacklisting syscalls for a process is much cheaper.
Using Landlock is probably not super useful to Nix because it’s less portable. Also, Nix has similar overheads as Bazel in terms of copying files, hashing them, … which makes it inherently slower. If the 100ms overhead is per package, it doesn’t amount to a lot compared to if it’s for each .c file.
IMO both approaches are complementary. Use Nix to pull third-party dependencies and declare your build environment. And then use a lightweight sandbox for in-repo incremental rebuilds.
This got me excited, but
man sandbox-exectells me this :(They say that, but tools like Bazel and Chrome use the tool. I don’t think they can afford to break it.
Just listen to yourself…
What do you mean?
Apple haven’t shown a huge amount of enthusiasm for supporting APIs for alternative browsers to run on their platforms…
We’re talking about macOS, not iOS. And about build tools, not browsers.
Chrome is a browser, fyi.
Sorry, thought they meant Bazel was building Chrome. In any case, I don’t think Apple is going to deliberately break Chrome.
I think the answer to my question is here:
What’s ironic is that Nix does configure a chroot, mounts, etc. in just 200 lines of code:
https://github.com/NixOS/nix/blob/280543933507839201547f831280faac614d0514/src/libstore/build/local-derivation-goal.cc#L767-L952
But Nix requires root for sandbox builds (end-user gets this usually via nix-daemon) and it does seem like Landlock is a bit more simple, so I think I see some clear benefits.
Swift Package Manager uses sandbox-exec too. Its implementation is also ~150 lines of code.
I’m inspired to see if I can do something similar but for ninja.
I just pulled the ninja source last night and was refamiliarizing myself with it. It didn’t (after 10 mins of looking around) like it would be all that easy. I’m not sure that ninja rules actually have everything you need to know to sandbox well. I may look at Gn too to see if the logic can be added there.
I’m toying with extending
Subprocesswith some methods to instruct what to unveil. From where it’s ran ingraph.cc,Edgeis accessible and I hopeEdge::CollectInputswill give the right paths…There is a sandbox-exec tool that does what it says, apparently. Sources say it’s been labeled as deprecated but still works and is used by a lot of system services.
Binding a global hotkey is deprecated - it’s a Carbon API, deprecated for a decade - but it’s widely used by utilities.
TIL about
.EXTRA_PREREQS. I mean landlock make sounds great too and I’ll play around with it, but I can put.EXTRA_PREQSinto production and start stripping out$(filter-out ...)clutter in Makefiles today!I really hope to see this get upstreamed someday. I currently depend a lot on
remake, but sandboxing would be a great tool to have too. When all the fun tools are their own forks it makes it really hard to use both feature sets!