Due to structural typing the error messages with Dhall were very verbose and sometimes confusing. It’s a very impressive tool nevertheless.
My team and I considered using it but ended up building a Kubernetes configuration kit for ReasonML/OCaml - https://github.com/rizo/rekube. Technically it’s just a embedded DSL for the Kubernetes API, almost everything else is provided by ReasonML.
Any reason why you choose ReasonML specifically? Was it something your team has been comfortable with already and/or were there technical reasons which stood out?
My team was already familiar with ReasonML from working on other parts of the system in ReasonML. But familiarity is not the only reason why we picked ReasonML. It’s a language with approachable syntax, excellent type-system and good IDE integration.
Rekube itself is based on an ocaml-swagger library which makes the generation of Kubernetes bindings trivial.
Tanka uses jsonnet which is in the language category of “data templating languages” as is Dhall. While they both target the structured data generation usecase, they have significantly different heritages. Jsonnet is a mostly intact descendant (with some small but significant improvements) of GCL, which is a Google internal configuration language notably used to configure Borg. I haven’t played with Dhall yet, but the syntax looks a lot like Haskell.
The Dhall Kubernetes package specifies record types for every object available by default in Kubernetes.
I glanced at Dhall & Nix for this purpose, as this could’ve saved me a lot of pain as a casual Kubernetes user. But on the flip-side, I didn’t go through with it after I realized that I did not write 99% of my manifests myself, and instead copy pasted & configured what’s available on Github. The way I use it, I’d need a massive ecosystem around Dhall+k8s with reusable bits around most things you want to deploy on kubernetes.
If I find myself doing this full-time in the future, and writing most of my manifests myself (without worrying of catching up with third party software), the safety and abstraction level this provides sounds like a big win.
I didn’t go through with it after I realized that I did not write 99% of my manifests myself, and instead copy pasted & configured what’s available on Github.
Would it work for you to convert YAML manifests from Github to JSON and importing them through json-to-dall? Haven’t tried that myself yet, but doing something similar with cuelang at the moment. I guess the drawback is that you would probably have to annotate types by yourself?
rekube mentioned in this thread sounds even more up my alley actually, as I’m a heavy Reason user. And looking at it’s repo it seems to have such a tool, so it is certainly possible! Maybe it’s a bit easier with rekube because how good OCaml is at inferring types?
I’ve spent the last two weeks or so to look into jsonnet, dhall and cue specifically for Kubernetes configurations.
jsonnet seems to be most widely used and has a python-esque syntax which might be more familiar to others in your team. On the negative side, it doesn’t really have a strong type system and seems to rely a lot on layers of handwritten helper functions. Tanka seems to inherit them from ksonnet which is (mostly?) discontinued. With qbec.io, tanka.dev, ksonnet.io and more, there are quite a few different projects based on jsonnet.
dhall is quite haskell-like regarding its syntax, has a strong type system, the tooling can be a bit slow at times, but seems to be very well thought-out and the documentation is top-notch. Type definitions for Kubernetes are auto-generated from Kubernetes OpenAPI spec which is quite cool: https://github.com/dhall-lang/dhall-kubernetes/tree/master/dhall-kubernetes-generator
cuelang seems to be the youngest of those projects, isn’t really finished yet but IMHO quite promising and with a strong theoretical underpinning. Cool features include, among others: auto-generated type definitions from any golang types which use encoding/json, constraints as in logic programming for configurations (e.g. a type for ip addresses in a private range and many more. Tooling and concepts were quite unusual compared to what I knew before, so it took me some time before I became productive in cue, but it’s been worth the effort so far. The most throughout tour through its features is probably the Kubernetes tutorial
I’ve tried to use Dhall for kubernetes before and ran into problems because it was taking hours to just import the kubernetes config files. There’s an issue for it here. I really want it to work though, I currently manage a bunch of services using Helm which is very painful - IMO due to the lack of abstraction it provides and Dhall looks like it would solve that.
For completeness’ sake I also recommend to take a look at kubenix.
Kubenix is written in Nix and uses the modules system to add a layer of schema validation. It generates the schema from the Kubernetes swagger spec so you know it will always be complete and up to date.
One advantage of using Nix is that you can use the same language to generate both the container images and the kubernetes configuration. Which allows to do things like run end-to-end tests.
Due to structural typing the error messages with Dhall were very verbose and sometimes confusing. It’s a very impressive tool nevertheless.
My team and I considered using it but ended up building a Kubernetes configuration kit for ReasonML/OCaml - https://github.com/rizo/rekube. Technically it’s just a embedded DSL for the Kubernetes API, almost everything else is provided by ReasonML.
Interesting as well, thanks!
Any reason why you choose ReasonML specifically? Was it something your team has been comfortable with already and/or were there technical reasons which stood out?
My team was already familiar with ReasonML from working on other parts of the system in ReasonML. But familiarity is not the only reason why we picked ReasonML. It’s a language with approachable syntax, excellent type-system and good IDE integration.
Rekube itself is based on an ocaml-swagger library which makes the generation of Kubernetes bindings trivial.
Do you know about Grafana Tanka? It seems to solve the same problem (with some extras).
Tanka uses jsonnet which is in the language category of “data templating languages” as is Dhall. While they both target the structured data generation usecase, they have significantly different heritages. Jsonnet is a mostly intact descendant (with some small but significant improvements) of GCL, which is a Google internal configuration language notably used to configure Borg. I haven’t played with Dhall yet, but the syntax looks a lot like Haskell.
Various data templating languages:
Another, also a descendant of GCL at least in spirit: https://github.com/zaphar/ucg/
I became aware of it after I posted this article, it doesn’t look like it solves the type-safety problem the way Dhall does.
This is very cool!
I glanced at Dhall & Nix for this purpose, as this could’ve saved me a lot of pain as a casual Kubernetes user. But on the flip-side, I didn’t go through with it after I realized that I did not write 99% of my manifests myself, and instead copy pasted & configured what’s available on Github. The way I use it, I’d need a massive ecosystem around Dhall+k8s with reusable bits around most things you want to deploy on kubernetes.
If I find myself doing this full-time in the future, and writing most of my manifests myself (without worrying of catching up with third party software), the safety and abstraction level this provides sounds like a big win.
Would it work for you to convert YAML manifests from Github to JSON and importing them through
json-to-dall? Haven’t tried that myself yet, but doing something similar with cuelang at the moment. I guess the drawback is that you would probably have to annotate types by yourself?rekubementioned in this thread sounds even more up my alley actually, as I’m a heavy Reason user. And looking at it’s repo it seems to have such a tool, so it is certainly possible! Maybe it’s a bit easier withrekubebecause how good OCaml is at inferring types?Very interesting article, thanks!
I’ve spent the last two weeks or so to look into
jsonnet,dhallandcuespecifically for Kubernetes configurations.jsonnetseems to be most widely used and has a python-esque syntax which might be more familiar to others in your team. On the negative side, it doesn’t really have a strong type system and seems to rely a lot on layers of handwritten helper functions. Tanka seems to inherit them from ksonnet which is (mostly?) discontinued. With qbec.io, tanka.dev, ksonnet.io and more, there are quite a few different projects based on jsonnet.dhallis quite haskell-like regarding its syntax, has a strong type system, the tooling can be a bit slow at times, but seems to be very well thought-out and the documentation is top-notch. Type definitions for Kubernetes are auto-generated from Kubernetes OpenAPI spec which is quite cool: https://github.com/dhall-lang/dhall-kubernetes/tree/master/dhall-kubernetes-generatorcuelangseems to be the youngest of those projects, isn’t really finished yet but IMHO quite promising and with a strong theoretical underpinning. Cool features include, among others: auto-generated type definitions from any golang types which use encoding/json, constraints as in logic programming for configurations (e.g. a type for ip addresses in a private range and many more. Tooling and concepts were quite unusual compared to what I knew before, so it took me some time before I became productive in cue, but it’s been worth the effort so far. The most throughout tour through its features is probably the Kubernetes tutorialI’ve tried to use Dhall for kubernetes before and ran into problems because it was taking hours to just import the kubernetes config files. There’s an issue for it here. I really want it to work though, I currently manage a bunch of services using Helm which is very painful - IMO due to the lack of abstraction it provides and Dhall looks like it would solve that.
I put my own instance of it here. It may be faster depending on how your latency is to my server in Montreal.
YAML has node anchors which are sometimes referred to as variables
I looked into that, but node anchors don’t provide me with type-safety like I wanted.
For completeness’ sake I also recommend to take a look at kubenix.
Kubenix is written in Nix and uses the modules system to add a layer of schema validation. It generates the schema from the Kubernetes swagger spec so you know it will always be complete and up to date.
One advantage of using Nix is that you can use the same language to generate both the container images and the kubernetes configuration. Which allows to do things like run end-to-end tests.
Do you know of any guides on how to get started with Nix to define container images?
How familiar are you with nix already?
There are some examples in the manual but that might not be enough: https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools If you learn best by example, there are a few example there: https://github.com/nix-community/docker-nixpkgs
And to learn about Nix itself, I heard that this is a pretty good source: https://github.com/tazjin/nix-1p