When your configuration has become complicated enough that you need a DSL it starts to look a lot less like configuration and much more like a scripting interface to your application.
For scripting your app I agree no static format or generic special purpose configuration language with logic will really fit your needs. You’ll need to provide some sort of scriptable DSL. However the vast majority of configuration is not and should not be this kind of configuration. For those a static format is perfect since it limits the impulse to grow into a DSL. Languages like DHall, Cue, JSonnet, and others which can generate those formats are a good fit when used as a way to generate the config files.
When your configuration has become complicated enough that you need a DSL it starts to look a lot less like configuration and much more like a scripting interface to your application.
Extremely this.
Configuration isn’t precisely defined. But in the general context, any definition that can’t be satisfied by flat key-value pairs, expressed by e.g. command-line flags, isn’t a useful model.
Funny that we are in 2021 and the world haven’t gotten this right. What more evidence do we need. The article provides the classical example that disproves its point.
Webserver virtual host configuration. Of the two most used webservers in the world, one became the go to example on how a poorly designed configuration system becomes a unescapable nightmare (Apache) and the other has a configuration system that ‘degraded’ to its own platform with multinational companies running their whole products on top of it (nginx).
Now that I think about it, is the article sarcastic?
nginx is sort of in support of his point. It’s a custom format for configuring a host. It’s syntax is better than apaches but it is still very much a custom config format. So I guess it’s one for one?
My point is that, by being a proper configuration by his standards, it ends up being not a configuration but an application platform itself.
If has support for lua scripting. Some of the biggest websites in the world are, by his classification, not webapps, but rather anwebserver with its configuration. You wouldn’t call Alibaba “just a well configured static website”, it is very much a web application.
Objectively, if you support scripting, you are not bulding a configuration system but maybe a programming language/application platform.
JavaScript started sort of like a solution to configure dynamic behaviour of an HTML document. It was only a matter of time before it became widely used as what it is: a programmable environment on which you can build applications.
There were whole websites dedicated to Apache configuration back in the day. The official documentation had essentially a book on it, with the introductory articles presenting themselves as your first baby steps on a long journey of learning Apache configuration. A skill comparable to, say, learning a programming language.
Then lighttpd, nginx and a couple.ofmothrs came. And Apache reign was over in two years or so.I don’t have the data to support it. Just anedoctical. From what I remember pretty much everyone thought nginx configuration is easier. I myself do find it easier than Apache’s.
I used Apache for maybe 10 years. I was a heavy user, I even interviewed to do some Apache training in maybe 2002/3 (I particularly remember that all the stuff I swotted up on about how the weird-ass SSL config options interacted with VirtualHost came up in the call and I was silently high-giving myself), but something else came up, and in retrospect, I’m very very glad. At the time I thought the config was OK, it was a bit random and all over the shop, and it had plenty rough edges, but it was free, what the hell. I wasn’t going to be using IIS so, whatever. Then I discovered nginx back in maybe 2007 and it was night and day. The performance and scale difference on the same hardware was from a different planet, but the configuration was like a breath of fresh air when you hadn’t realized you were suffocating. To say Apache’s was “patchy” was an understatement. There was only enough semblance of consistency and coherence that you thought that you could make sense of it when you needed to work something out or learn some new part of it; in reality you just had to learn all the differently-shaped bits or keep going back to the docs the whole time. It was immediately obvious on reading nginx config for even 0.6 that they were worlds apart. The fall-through stuff with named locations was just … “oh, ok, yeah”. Even though nginx has its own peculiarities, I’ve literally never looked back, apart from when I occasionally have to dig into an Apache config file on some ancient box somewhere and I kind of shiver and enjoy it in a weird, grave-digging kind of way. Haha, you had to do this … and that … and … oh, wow. Wow I’m happy I don’t have to deal with this any more.
I agree. It’s part of the reason why I made one of the output formats of my personal experiment in this space https://github.com/zaphar/ucg to be command line flags. I think it was the first output format I did actually.
I don’t really agree. There are a few things that are common to configuration files, specifically those things related to composition. This is what the universal configuration language does well: it handles all of the details of providing a default, a set of overrides, splitting configurations across multiple files, and so on. It does not give you the schema for your configuration file (though I’ve written some additional tooling that can do this) or handle, for example, what specific strings mean.
I don’t believe the article really substantiated this claim: “But no universal language like Dhall can be a genuinely good configuration language for a program with complex configuration needs.”
It’s giving examples like Apache and PF of hard problems, but I’ve got a generic rebuttal for that - they’re text formats which can be parsed. If they can be parsed, they can be represented by a tree. From there, we know that most of those trees can be reasonably typed. It’s partially proven by puppet, chef, salt, etc. having modules for serialising those configurations from their native k-v configuration stores.
Although maybe we disagree on a more fundamental level, because I already disagree with this claim from earlier post:
Using a general purpose language is adding power while subtracting clarity; now, in order to understand what the configuration is going to do you need to mentally execute the configuration program in your head, on top of understanding the effects of the configuration options themselves.
The first time you look at configuration, you have no clarity regardless of the language. We need tools which allow immediate and easy preview of the result - and that is true whether you use dhall or DSL. Your config is “executed” too after all.
I basically agree, but a generic config language is a great default, as most configuration doesn’t need to be complex. For complex rules I agree a custom config language is probably the best answer, but for many applications, complex configuration is not needed and something generic is totally acceptable. If all you really need is some key/value pairs, then pick a randomly sane config language and move on with life. If your config gets complex, then go reach for writing a custom thing. You almost never want to start with a complex, custom config language.
Most of what I write, I also need to store some data, so I just have a config table in sqlite that keeps key/value pairs for me, no separate config. Bonus, configs are now instantly tied to the data you are messing with. downside, it’s not by default text editable with vim/emacs, but you can import export something like toml very easily.
In the same way that, for example, data for a customer invoicing system is relational. You define keys and relations for the domain entities, and attributes or reference relations for the details.
I don’t think that business domain information is comparable to application configuration information. Application configuration isn’t a set of domain entities with relations to each other; it’s at most hierarchical/namespaced metadata, mapped to components in your dependency graph. Or? What am I missing?
I’m not going to rise to the challenge from the article in order to completely re-model Apache vhost as relations, but to consider that example:
A virtual host is an entity in the domain of Apache configurations. It carries 1:1-arity relations to attributes such as DocumentRoot and ServerName. It defines a 1:many-arity relation to ServerAlias and a 1:zero-or-one-arity relation to mod_proxy attributes.
Deliberately modeling these data instead of using a text-based configuration can avoid caveats such as “IPv6 addresses must be specified in square brackets because the optional port number could not be determined otherwise.”
I totally agree that mapping the data to objects in the application dependency graph is a key requirement of configuration systems. If “ORM” is a dirty word to you, you may not like how I would propose addressing that. : )
I guess my response would be that Apache “config” is extraordinarily complex, fails the test I describe here, and isn’t representative of the needs of the overwhelming majority of software.
When your configuration has become complicated enough that you need a DSL it starts to look a lot less like configuration and much more like a scripting interface to your application.
For scripting your app I agree no static format or generic special purpose configuration language with logic will really fit your needs. You’ll need to provide some sort of scriptable DSL. However the vast majority of configuration is not and should not be this kind of configuration. For those a static format is perfect since it limits the impulse to grow into a DSL. Languages like DHall, Cue, JSonnet, and others which can generate those formats are a good fit when used as a way to generate the config files.
Extremely this.
Configuration isn’t precisely defined. But in the general context, any definition that can’t be satisfied by flat key-value pairs, expressed by e.g. command-line flags, isn’t a useful model.
Funny that we are in 2021 and the world haven’t gotten this right. What more evidence do we need. The article provides the classical example that disproves its point.
Webserver virtual host configuration. Of the two most used webservers in the world, one became the go to example on how a poorly designed configuration system becomes a unescapable nightmare (Apache) and the other has a configuration system that ‘degraded’ to its own platform with multinational companies running their whole products on top of it (nginx).
Now that I think about it, is the article sarcastic?
nginx is sort of in support of his point. It’s a custom format for configuring a host. It’s syntax is better than apaches but it is still very much a custom config format. So I guess it’s one for one?
My point is that, by being a proper configuration by his standards, it ends up being not a configuration but an application platform itself.
If has support for lua scripting. Some of the biggest websites in the world are, by his classification, not webapps, but rather anwebserver with its configuration. You wouldn’t call Alibaba “just a well configured static website”, it is very much a web application. Objectively, if you support scripting, you are not bulding a configuration system but maybe a programming language/application platform.
JavaScript started sort of like a solution to configure dynamic behaviour of an HTML document. It was only a matter of time before it became widely used as what it is: a programmable environment on which you can build applications.
Is this generally-accepted wisdom? Apache’s configuration takes a little getting used to, but I don’t really have a problem with it.
There were whole websites dedicated to Apache configuration back in the day. The official documentation had essentially a book on it, with the introductory articles presenting themselves as your first baby steps on a long journey of learning Apache configuration. A skill comparable to, say, learning a programming language.
Then lighttpd, nginx and a couple.ofmothrs came. And Apache reign was over in two years or so.I don’t have the data to support it. Just anedoctical. From what I remember pretty much everyone thought nginx configuration is easier. I myself do find it easier than Apache’s.
I used Apache for maybe 10 years. I was a heavy user, I even interviewed to do some Apache training in maybe 2002/3 (I particularly remember that all the stuff I swotted up on about how the weird-ass SSL config options interacted with
VirtualHostcame up in the call and I was silently high-giving myself), but something else came up, and in retrospect, I’m very very glad. At the time I thought the config was OK, it was a bit random and all over the shop, and it had plenty rough edges, but it was free, what the hell. I wasn’t going to be using IIS so, whatever. Then I discovered nginx back in maybe 2007 and it was night and day. The performance and scale difference on the same hardware was from a different planet, but the configuration was like a breath of fresh air when you hadn’t realized you were suffocating. To say Apache’s was “patchy” was an understatement. There was only enough semblance of consistency and coherence that you thought that you could make sense of it when you needed to work something out or learn some new part of it; in reality you just had to learn all the differently-shaped bits or keep going back to the docs the whole time. It was immediately obvious on reading nginx config for even 0.6 that they were worlds apart. The fall-through stuff with named locations was just … “oh, ok, yeah”. Even though nginx has its own peculiarities, I’ve literally never looked back, apart from when I occasionally have to dig into an Apache config file on some ancient box somewhere and I kind of shiver and enjoy it in a weird, grave-digging kind of way. Haha, you had to do this … and that … and … oh, wow. Wow I’m happy I don’t have to deal with this any more.Compared to cabby or traefik, looks awfully complicated and ugly.
I agree. It’s part of the reason why I made one of the output formats of my personal experiment in this space https://github.com/zaphar/ucg to be command line flags. I think it was the first output format I did actually.
I don’t really agree. There are a few things that are common to configuration files, specifically those things related to composition. This is what the universal configuration language does well: it handles all of the details of providing a default, a set of overrides, splitting configurations across multiple files, and so on. It does not give you the schema for your configuration file (though I’ve written some additional tooling that can do this) or handle, for example, what specific strings mean.
I came here to mention libucl, too. All my applications use libucl for their config files.
I don’t believe the article really substantiated this claim: “But no universal language like Dhall can be a genuinely good configuration language for a program with complex configuration needs.”
It’s giving examples like Apache and PF of hard problems, but I’ve got a generic rebuttal for that - they’re text formats which can be parsed. If they can be parsed, they can be represented by a tree. From there, we know that most of those trees can be reasonably typed. It’s partially proven by puppet, chef, salt, etc. having modules for serialising those configurations from their native k-v configuration stores.
Although maybe we disagree on a more fundamental level, because I already disagree with this claim from earlier post:
The first time you look at configuration, you have no clarity regardless of the language. We need tools which allow immediate and easy preview of the result - and that is true whether you use dhall or DSL. Your config is “executed” too after all.
I basically agree, but a generic config language is a great default, as most configuration doesn’t need to be complex. For complex rules I agree a custom config language is probably the best answer, but for many applications, complex configuration is not needed and something generic is totally acceptable. If all you really need is some key/value pairs, then pick a randomly sane config language and move on with life. If your config gets complex, then go reach for writing a custom thing. You almost never want to start with a complex, custom config language.
Most of what I write, I also need to store some data, so I just have a
configtable in sqlite that keeps key/value pairs for me, no separate config. Bonus, configs are now instantly tied to the data you are messing with. downside, it’s not by default text editable with vim/emacs, but you can import export something like toml very easily.I’ll drop this here again; your application configuration should be a relational database: https://lobste.rs/s/sr2lhm/pitfalls_using_env_for_configuration#c_k0d4l8
This doesn’t engage with any of the issues outlined in the article.
counterpoint: that language is SQL
this is relational data modeling
correct, it is also expressed in the (relational) data model
or you use a rich, relational data model with SQL for query and transformation
How is configuration relational?
In the same way that, for example, data for a customer invoicing system is relational. You define keys and relations for the domain entities, and attributes or reference relations for the details.
I don’t think that business domain information is comparable to application configuration information. Application configuration isn’t a set of domain entities with relations to each other; it’s at most hierarchical/namespaced metadata, mapped to components in your dependency graph. Or? What am I missing?
I’m not going to rise to the challenge from the article in order to completely re-model Apache vhost as relations, but to consider that example:
A virtual host is an entity in the domain of Apache configurations. It carries 1:1-arity relations to attributes such as
DocumentRootandServerName. It defines a 1:many-arity relation toServerAliasand a 1:zero-or-one-arity relation tomod_proxyattributes.Deliberately modeling these data instead of using a text-based configuration can avoid caveats such as “IPv6 addresses must be specified in square brackets because the optional port number could not be determined otherwise.”
I totally agree that mapping the data to objects in the application dependency graph is a key requirement of configuration systems. If “ORM” is a dirty word to you, you may not like how I would propose addressing that. : )
I guess my response would be that Apache “config” is extraordinarily complex, fails the test I describe here, and isn’t representative of the needs of the overwhelming majority of software.