Crustaceans, how do you do parameterized configuration?
- You have config files where some parts are parameterized - they depend on the particular instance being configured (e.g. IDs, API keys, rules)
- These config files need to be tracked (VCS) and automatically applied (CI) to multiple instances of the thing being configured.
My current solution:
bash, and [the thing that applies a config]. (i.e. a bash script that takes CLI args, sets env variables using those args, uses
envsubst to replace $VARIABLES by values in the config file, and applies the resulting config)
Why the current solution sucks:
- The config and the parameters are both structured data (some combination of maps, lists and primitive types), not arbitrary text. Generating / preserving that structure is tedious and error-prone when using pure bash to set the parameters.
- Dependency/aggregation is cumbersome. If you have a (parameterized) configuration for component A stored in repo git://A, and configuration for component B stored in repo git://B, and you need to apply both configurations with some shared values, you need to explicitly check out each repo at a given branch/tag/commit and then run the respective shell script, providing the parameters as CLI arguments. The CLI syntax could either vary or is duplicated between A and B.
- There is no standard way of tracking which specific config was applied - the bash script applying a given config can do whatever it wants. No record of the actual values of the environment variables used by envsubst is kept.
My attempt at an improvement:
- Use go-templates to parameterize the config files and JSON (or YAML or TOML) to define the parameter values. (https://github.com/sgreben/render)
- Record the provided parameter values
Why the improvement still sucks:
- No notion of dependencies/packages/interfaces, this still needs to be done “manually”
- Too flexible: there are too many ways of achieving the same thing using the template tool.
I’m not satisfied. Maybe someone else has figured this out already? Hence, the question - how do you deal with this?