The key benefit of capabilities is that they make it easy to respect the principle of intentionality: if you are going to perform a privileged action, you must explicitly authorise it with some rights that were explicitly granted to you. This makes it easy to audit code and see if it is conforming to the principle of least privilege (were any rights granted to it that it shouldn’t have?) and implicitly enforces least privilege (you can’t exercise rights that you weren’t given).
MarkM’s thesis is great, but you’re not in a mood to dive straight into 200 pages of text, there’s lots of related stuff to warm up with https://github.com/dckc/awesome-ocap
Neat article that helped give some formal structure to what I’ve been doing. When I write user facing code, I’ve always tried to use my type system to catch risks like this: a username isn’t just a username, it’s a TaintedUsername and you then have to explicitly go through some guarded path to get untainted data. This always fit nicely with the ports and adapters model - your core logic only accept clean things, your interfaces only handle tainted things.
Of course, there were huge limitations here. if the library I grafted in would give you root if the user passed in {#root-shell-please!}, I had to know to look for that string in my cleaning function, to aggressively whitelist only certain inputs, or how to configure my library with less magic. So the unknown unknowns were always scary.
Now I’m going to read the linked thesis and discover this problem was solved decades ago ;)
Parse, don’t validate is even closer to what you’re describing. Capabilities are about securing access to “the outside world” (filesystem, network, …) while you’re talking about data in the application. These things are complementary and similar in some ways but not exactly the same thing…
Isn’t user generated content always tainted? What does cleaning it mean? If you put a username into your DB you protected against SQL injection, but that string is still tainted and unsafe to stick in an HTML attribute, for example.
If I’m correctly guessing what you’re saying, here’s some advice: don’t think of a username with special characters in it as “tainted”. It’s not tainted. It’s a regular string, and its possible values are all possible strings (possibly with a length limit or whatnot).
However, don’t confuse strings with SQL code snippets or HTML snippets. “ ’ “ (single quote) is a valid string, and a valid HTML snippet, but not a valid SQL snippet. “&” is a valid string, and a valid SQL id snippet, but not a valid HTML snippet.
If you write:
"<p>Hello, <b>" + username + "</b>!</p>"
that’s akin to a type error. The type of HTML should be different than the type of strings. A “better” way to write this would be
Notice the implied type signature of HtmlText: it takes a string and turns it into an HTML. Now this is verbose enough that you can see why it’s common to represent HTML as text. Which is an OK thing to do, as long as you realize it, and realize that when you jam a username into HTML, you need to do the type conversion from string to HTML by escaping special characters in the username.
Instead, there is exactly one Network object ever in existence, and it is passed in to the program at one location, perhaps as an argument to main. (Actually, if the operating system was capability-safe and Java was cooperating with it, then main would be given a Network if and only if the executable was given network access.)
This is actually a huge point. From what I know about capabilities, they require OS support. If a library could just use the Network object without being granted permission, then having the more granular types doesn’t solve anything.
No, it only requires language design and library support. By “main”, they mean a top-level entrypoint for a program, and this entrypoint happens to be language-specific, not part of the OS API. The language runtime has all of the raw power given by the OS, and must wrap it into objects like the “Network” object discussed in the post. This process is known as taming.
To give a precise example from my Monte language, a relative of E, here are two entrypoint declarations. The first one has network access; specifically, it has a IPv4 outgoing TCP dialer.
def main(argv, => makeTCP4ClientEndpoint) as DeepFrozen:
And this second one does not:
def main(argv) as DeepFrozen:
In practice, this is very useful for quickly estimating a program’s isolation from other programs. For example, the entrypoint for the Monte compiler indicates usage of system timers, the filesystem, and stdio; but not any network access. OS support could make this more robust against bugs in the runtime, but the OS itself also could have bugs in its sandboxing; it’s not necessarily better.
You can have capabilities in the OS, and capabilities in the language, and they’re complementary. At the language level, you state what capabilities main expects, and control how they flow through the program, which lets you reason about what your dependencies (and their transitive dependencies!) can access, relative to the rest of your program. At the OS level, you control what capabilities each process is given.
If a library could just use the Network object without being granted permission
Is the library (log4j / jndi) simply being called (possibly transitively) from the application’s main function, then all you need is capabilities in the language, as my post describes. The result is that as an application author it is extremely visible to you whether log4j might access the network. (Though with the number of people seeming not to get it, I’m worried I might have described it poorly.)
Are you (amw-zero) thinking of the case where log4j or jndi is running in a separate process, communicating with IPCs? That came up on the orange site, and it makes the story a lot more complicated. In that case I think you might need capabilities in both the language and the OS to get things under control.
You didn’t describe things poorly; capabilities are a tough concept to understand. I wrote a short story about what it feels like to have this conversation repeatedly.
If it were written for capabilities, it might always ask for network access (possibly much more than it really needs) because part of log4j’s purpose of for you to be able to write logs to all kinds of places like syslog or Cassandra or many other things. See loads listed here: https://logging.apache.org/log4j/2.x/manual/appenders.html
I think a hypothetical log4monte with a good design would ask for few/no capabilities in the main body of the library but let you pass in an “Appender” object which asked for the specific capabilities required to talk to just the right backend.
Of course a hypothetical log4monte with a really bad design could ask for massive network capabilities always, and filesystem too, in order to make the formatting feature work like log4j’s and be able to do things like look up user data from remote servers and local files during message formatting.
Maybe an interesting exercise would be to try making Monte equivalents of popular libraries like this to see if they can be made both ergonomic and feature rich. :)
(Aside: totally weird open question as to whether you can make them configurable with dependency injection controlled by some XML file that is read and then used to invoke classes through reflection… but actually maybe the ability to do that is an anti-feature.)
One of the features of E that we didn’t include in Monte, URI expressions, is very similar to the magical invocation of ambient authority in log4j. A URI expression is supposed to be safe and tamed somehow, but I’m not sure that I see how that could possibly be safe enough.
We also considered something like your “Appender”; the Strategy pattern would make it easy to change logging backends. This would make sense for a fancy logging module, but there’s a couple design issues still. The first one is that we try not to pass around objects with incomplete responsibilities; we don’t actually want something with sufficient power to interpret URI expressions, because such objects can be misused. Compare two ways to attenuate the core of such power:
def overpowered(x, log) { return log("http", x) }
def attenuate(log):
return def HTTPonly(x) { return log("http", x) }
def attenuated(x, log) { return log(x) }
# Now change overpowered(x, cap) to attenuated(x, attenuate(cap))
# And once that's done, we can lambda-lift calls to attenuate()
# At the very top, the attenuated cap is a Strategy
The second issue is that logging should be discouraged in production. To split it into subissues, a first problem is that we want logging to happen immediately, instead of being queued up as a netwoking or stdio effect; otherwise, it’s not useful for debugging. The E/Monte builtin println function does exactly this, and prints to stderr immediately and impurely.
And this leads nicely to the other reason to avoid logging in production: there are better ways to debug at scale, so debugging with println should not be deployed widely. Instead, I recommend metrics and monitoring agents. Monte does have a literate module implementing a Prometheus client, and I’ve linked to a relevant header. The .processMetrics/1 method takes a very powerful capability and immediately forgets about most of it, wrapping it so that we can safely export metrics from Prometheus without exposing the entire runtime to a potentially-untrusted HTTP application.
FWIW I’m just calling it that because that’s what log4j calls it. :)
And yes, this is an instance of the strategy pattern, though they didn’t use that word in the name. (Rightly so IMO.)
The second issue is that logging should be discouraged in production
I’m not interested in discussing that here because it’s not germane to this specific discussion.
URI expressions, is very similar to the magical invocation of ambient authority in log4j. A URI expression is supposed to be safe and tamed somehow, but I’m not sure that I see how that could possibly be safe enough.
Looking at the documentation, I agree these do seem incredibly overpowered. I think they’d be fine in code or config files but I can’t see how you could ever use variable ones in application code safely. I mean, there’s one that loads and runs arbitrary class files!
Probably worth noting that the underlying Java feature that class interacts with, the security manager, is slated for removal in an upcoming Java release. AccessController and related classes are marked as deprecated in Java 17.
It’s also not present in the Android version of Java at all. The SecurityManager is interesting but it depends on stack introspection to understand whether you’re allowed to do things and has such a large dependency footprint that almost any JVM bug can be turned into a bypass (my favourite one is that it depends on string comparison to identify rights and so a single bit flip that allows you to modify an immutable String can let you privilege elevate). You also basically have to disable reflection with the SecurityManager because you can use reflection to call the private methods that are protected by it (the usual idiom is that you expose a public method that does a security manager check and then calls the private method).
The key benefit of capabilities is that they make it easy to respect the principle of intentionality: if you are going to perform a privileged action, you must explicitly authorise it with some rights that were explicitly granted to you. This makes it easy to audit code and see if it is conforming to the principle of least privilege (were any rights granted to it that it shouldn’t have?) and implicitly enforces least privilege (you can’t exercise rights that you weren’t given).
MarkM’s thesis is great, but you’re not in a mood to dive straight into 200 pages of text, there’s lots of related stuff to warm up with https://github.com/dckc/awesome-ocap
Neat article that helped give some formal structure to what I’ve been doing. When I write user facing code, I’ve always tried to use my type system to catch risks like this: a username isn’t just a username, it’s a
TaintedUsernameand you then have to explicitly go through some guarded path to get untainted data. This always fit nicely with the ports and adapters model - your core logic only accept clean things, your interfaces only handle tainted things.Of course, there were huge limitations here. if the library I grafted in would give you root if the user passed in
{#root-shell-please!}, I had to know to look for that string in my cleaning function, to aggressively whitelist only certain inputs, or how to configure my library with less magic. So the unknown unknowns were always scary.Now I’m going to read the linked thesis and discover this problem was solved decades ago ;)
Parse, don’t validate is even closer to what you’re describing. Capabilities are about securing access to “the outside world” (filesystem, network, …) while you’re talking about data in the application. These things are complementary and similar in some ways but not exactly the same thing…
Isn’t user generated content always tainted? What does cleaning it mean? If you put a username into your DB you protected against SQL injection, but that string is still tainted and unsafe to stick in an HTML attribute, for example.
If I’m correctly guessing what you’re saying, here’s some advice: don’t think of a username with special characters in it as “tainted”. It’s not tainted. It’s a regular string, and its possible values are all possible strings (possibly with a length limit or whatnot).
However, don’t confuse strings with SQL code snippets or HTML snippets. “ ’ “ (single quote) is a valid string, and a valid HTML snippet, but not a valid SQL snippet. “&” is a valid string, and a valid SQL id snippet, but not a valid HTML snippet.
If you write:
that’s akin to a type error. The type of HTML should be different than the type of strings. A “better” way to write this would be
Notice the implied type signature of HtmlText: it takes a string and turns it into an HTML. Now this is verbose enough that you can see why it’s common to represent HTML as text. Which is an OK thing to do, as long as you realize it, and realize that when you jam a username into HTML, you need to do the type conversion from string to HTML by escaping special characters in the username.
To add to the suggested reading list: http://habitatchronicles.com/2017/05/what-are-capabilities/
This is actually a huge point. From what I know about capabilities, they require OS support. If a library could just use the Network object without being granted permission, then having the more granular types doesn’t solve anything.
No, it only requires language design and library support. By “main”, they mean a top-level entrypoint for a program, and this entrypoint happens to be language-specific, not part of the OS API. The language runtime has all of the raw power given by the OS, and must wrap it into objects like the “Network” object discussed in the post. This process is known as taming.
To give a precise example from my Monte language, a relative of E, here are two entrypoint declarations. The first one has network access; specifically, it has a IPv4 outgoing TCP dialer.
And this second one does not:
In practice, this is very useful for quickly estimating a program’s isolation from other programs. For example, the entrypoint for the Monte compiler indicates usage of system timers, the filesystem, and stdio; but not any network access. OS support could make this more robust against bugs in the runtime, but the OS itself also could have bugs in its sandboxing; it’s not necessarily better.
You can have capabilities in the OS, and capabilities in the language, and they’re complementary. At the language level, you state what capabilities
mainexpects, and control how they flow through the program, which lets you reason about what your dependencies (and their transitive dependencies!) can access, relative to the rest of your program. At the OS level, you control what capabilities each process is given.Is the library (log4j / jndi) simply being called (possibly transitively) from the application’s
mainfunction, then all you need is capabilities in the language, as my post describes. The result is that as an application author it is extremely visible to you whether log4j might access the network. (Though with the number of people seeming not to get it, I’m worried I might have described it poorly.)Are you (amw-zero) thinking of the case where
log4jorjndiis running in a separate process, communicating with IPCs? That came up on the orange site, and it makes the story a lot more complicated. In that case I think you might need capabilities in both the language and the OS to get things under control.You didn’t describe things poorly; capabilities are a tough concept to understand. I wrote a short story about what it feels like to have this conversation repeatedly.
Made me smile, thank you.
If it were written for capabilities, it might always ask for network access (possibly much more than it really needs) because part of log4j’s purpose of for you to be able to write logs to all kinds of places like syslog or Cassandra or many other things. See loads listed here: https://logging.apache.org/log4j/2.x/manual/appenders.html
I think a hypothetical log4monte with a good design would ask for few/no capabilities in the main body of the library but let you pass in an “Appender” object which asked for the specific capabilities required to talk to just the right backend.
Of course a hypothetical log4monte with a really bad design could ask for massive network capabilities always, and filesystem too, in order to make the formatting feature work like log4j’s and be able to do things like look up user data from remote servers and local files during message formatting.
Maybe an interesting exercise would be to try making Monte equivalents of popular libraries like this to see if they can be made both ergonomic and feature rich. :)
(Aside: totally weird open question as to whether you can make them configurable with dependency injection controlled by some XML file that is read and then used to invoke classes through reflection… but actually maybe the ability to do that is an anti-feature.)
One of the features of E that we didn’t include in Monte, URI expressions, is very similar to the magical invocation of ambient authority in log4j. A URI expression is supposed to be safe and tamed somehow, but I’m not sure that I see how that could possibly be safe enough.
We also considered something like your “Appender”; the Strategy pattern would make it easy to change logging backends. This would make sense for a fancy logging module, but there’s a couple design issues still. The first one is that we try not to pass around objects with incomplete responsibilities; we don’t actually want something with sufficient power to interpret URI expressions, because such objects can be misused. Compare two ways to attenuate the core of such power:
The second issue is that logging should be discouraged in production. To split it into subissues, a first problem is that we want logging to happen immediately, instead of being queued up as a netwoking or stdio effect; otherwise, it’s not useful for debugging. The E/Monte builtin
printlnfunction does exactly this, and prints to stderr immediately and impurely.And this leads nicely to the other reason to avoid logging in production: there are better ways to debug at scale, so debugging with
printlnshould not be deployed widely. Instead, I recommend metrics and monitoring agents. Monte does have a literate module implementing a Prometheus client, and I’ve linked to a relevant header. The.processMetrics/1method takes a very powerful capability and immediately forgets about most of it, wrapping it so that we can safely export metrics from Prometheus without exposing the entire runtime to a potentially-untrusted HTTP application.FWIW I’m just calling it that because that’s what log4j calls it. :) And yes, this is an instance of the strategy pattern, though they didn’t use that word in the name. (Rightly so IMO.)
I’m not interested in discussing that here because it’s not germane to this specific discussion.
Looking at the documentation, I agree these do seem incredibly overpowered. I think they’d be fine in code or config files but I can’t see how you could ever use variable ones in application code safely. I mean, there’s one that loads and runs arbitrary class files!
https://docs.oracle.com/javase/8/docs/api/java/security/AccessController.html exists… and would be a possible alternative to capabilities.
I’ve never seen it used - unsure if the right abstractions are there to build on top of it.
Probably worth noting that the underlying Java feature that class interacts with, the security manager, is slated for removal in an upcoming Java release.
AccessControllerand related classes are marked as deprecated in Java 17.It’s also not present in the Android version of Java at all. The
SecurityManageris interesting but it depends on stack introspection to understand whether you’re allowed to do things and has such a large dependency footprint that almost any JVM bug can be turned into a bypass (my favourite one is that it depends on string comparison to identify rights and so a single bit flip that allows you to modify an immutableStringcan let you privilege elevate). You also basically have to disable reflection with theSecurityManagerbecause you can use reflection to call the private methods that are protected by it (the usual idiom is that you expose a public method that does a security manager check and then calls the private method).LOL - so many nopes in that then :D