I don’t understand this at all. It’s titled “the fundamental difference between Terraform and Kubernetes” but then goes on to explain that they’re identical.
I think the fundamental difference they’re getting at is K8s continually evaluates reality vs desired and adjusts it as required. Terraform only does so when you run an apply. (Not sure that’s a fundamental difference, but that’s their terminology. 🤷🏻♂️)
I guess if you ran while :; do terraform apply --auto-approve; done you could equate it to being the same as K8s.
This is the most terrifying shell script fragment I’ve seen in recent memory. 😱
Yes, in theory your TF code should be audited and have guard rails around destroying critical resources etc. etc. but…IME it’s usually at least partly the human in the loop that prevents really horrible, data-destroying changes from happening.
Kubernetes, on the other hand, makes it (intentionally) difficult to create services that are “precious” and stateful, so you’re somewhat less likely to fire that particular footgun.
…and lest it seem I’m dumping on TF and talking up K8s: I think Kubernetes gives you all kinds of other ways to complicate and break your own infrastructure. It just usually takes more than a single-line change anywhere in your dependency graph to potentially schedule a stateful node for deletion (and likely re-creation just afterward, but that doesn’t protect you from local state being lost).
They’re also equally good at helping you create a bunch of “objects” for which your cloud provider can charge you a substantial monthly fee, which I think helps to explain their mutual popularity amongst the AWS/GCP/Azure/etc. set.
Beyond that, Terraform also understands dependencies between services while k8s continuously tries to reconcile (so if you create some resource that depends on another, Terraform will wait for the first to be created before creating the second, while Kubernetes will immediately try creating both and the second will just fail until the first is created).
They’re quite different. Here are some things you can do with Terraform and Kubernetes that aren’t symmetric, showing that they are distinct:
we can use the Kubernetes provider for Terraform to declare objects for a Kubernetes cluster, and apply Terraform multiple times against multiple Kubernetes clusters
we can apply Terraform from inside a Kubernetes Pod, regardless of which Terraform providers we are using
we can leave a Kubernetes cluster running for a week, constantly changing state, without interrupting coworkers (while Terraform requires a lock in order to apply state changes)
Oh, I’m not claiming that they are identical. I just think the OP is: it’s of the form “the fundamental difference between things A and B is that A does X, while B does X”.
I am satisfied by caius’s sibling comment that it’s trying to talk about the bandwidth difference: K8s’s feedback loop is very fast, while TF’s isn’t. But I don’t regard that difference as particularly fundamental, and I don’t think the ones you’ve identified strike me as fundamental either. So I’m asking: what is?
I don’t use either tool regularly, and I don’t have great mental models of them, but just in case it helps clarify, I think what I’m looking for might be ownership. K8s materializes the “resources” it manages inside itself, while TF prods external systems into changing state.
I don’t understand in which sense they are even remotely similar. It is like saying: the fundamental difference between Linux and Microsoft PowerPoint.
Terraform does let you do some imperative things, because it uses something closer to a programming language by default, but saying it is somehow less declarative than kubernetes is kinda wrong.
You can argue that kubernetes also offers a declarative interface to imperative commands: creating containers, updating etcd, managing volumes. In any declarative system, there something, at some point, that will execute imperative commands.
What I would say is a more significant difference between terraform and kubernetes is that terraform has to deal with the fact that there will be unmanaged resources in the cloud environments it manages. This is pretty much impossible with kubernetes.
Thank you for this article. Although you focus on Terraform vs. Kubernetes, you mentioned CloudFormation in the first sentence as identical to Terraform. However, I wonder if there are some differences between them that are worth noting.
For example, my understanding of CloudFormation is that it is a managed service with server-side state and built-in automated rollbacks , whereas Terraform does not have these features by default but can achieve them with some additional configuration or tools. In particular, I’m interested in how Terraform handles rollbacks when something goes wrong during provisioning. I learned that Terraform labels the resources as “tainted” and tries to recreate them in the next execution plan, but it does not support rolling back to a previous state file. I wonder how this compares to CloudFormation’s rollback mechanism, which restores the stack to its original state on failure.
As you’ve said Kubernetes is fully declarative. However, I also learned that Kubernetes does support automated rollbacks for some resources like deployments, which can revert to a previous version if something goes wrong. I think this is a useful feature for managing infrastructure changes with minimal downtime and risk.
As per Going faster with continuous delivery, I feel like features like rollback are central to evaluating deployment engines. How do you compare these different approaches to managing infrastructure changes? Thank you.
I don’t understand this at all. It’s titled “the fundamental difference between Terraform and Kubernetes” but then goes on to explain that they’re identical.
I think the fundamental difference they’re getting at is K8s continually evaluates reality vs desired and adjusts it as required. Terraform only does so when you run an apply. (Not sure that’s a fundamental difference, but that’s their terminology. 🤷🏻♂️)
I guess if you ran
while :; do terraform apply --auto-approve; doneyou could equate it to being the same as K8s.This is the most terrifying shell script fragment I’ve seen in recent memory. 😱
Yes, in theory your TF code should be audited and have guard rails around destroying critical resources etc. etc. but…IME it’s usually at least partly the human in the loop that prevents really horrible, data-destroying changes from happening.
Kubernetes, on the other hand, makes it (intentionally) difficult to create services that are “precious” and stateful, so you’re somewhat less likely to fire that particular footgun.
…and lest it seem I’m dumping on TF and talking up K8s: I think Kubernetes gives you all kinds of other ways to complicate and break your own infrastructure. It just usually takes more than a single-line change anywhere in your dependency graph to potentially schedule a stateful node for deletion (and likely re-creation just afterward, but that doesn’t protect you from local state being lost).
They’re also equally good at helping you create a bunch of “objects” for which your cloud provider can charge you a substantial monthly fee, which I think helps to explain their mutual popularity amongst the AWS/GCP/Azure/etc. set.
Just noticed the typo in it, stops someone copy/pasting it I guess.
I’m not convinced sticking TF in a while loop is a good choice, but that’s how my scaled out mental model of the two tools differs.
Beyond that, Terraform also understands dependencies between services while k8s continuously tries to reconcile (so if you create some resource that depends on another, Terraform will wait for the first to be created before creating the second, while Kubernetes will immediately try creating both and the second will just fail until the first is created).
They’re quite different. Here are some things you can do with Terraform and Kubernetes that aren’t symmetric, showing that they are distinct:
Oh, I’m not claiming that they are identical. I just think the OP is: it’s of the form “the fundamental difference between things A and B is that A does X, while B does X”.
I am satisfied by caius’s sibling comment that it’s trying to talk about the bandwidth difference: K8s’s feedback loop is very fast, while TF’s isn’t. But I don’t regard that difference as particularly fundamental, and I don’t think the ones you’ve identified strike me as fundamental either. So I’m asking: what is?
I don’t use either tool regularly, and I don’t have great mental models of them, but just in case it helps clarify, I think what I’m looking for might be ownership. K8s materializes the “resources” it manages inside itself, while TF prods external systems into changing state.
You could conceivably write a Terraform Custom Resource Definition for Kubernetes and make a controller that continuously applies it. 🙃
I don’t understand in which sense they are even remotely similar. It is like saying: the fundamental difference between Linux and Microsoft PowerPoint.
The fundamental difference is that one is a CLI and one is a daemon. They have many differences that are not fundamental.
Terraform does let you do some imperative things, because it uses something closer to a programming language by default, but saying it is somehow less declarative than kubernetes is kinda wrong.
You can argue that kubernetes also offers a declarative interface to imperative commands: creating containers, updating etcd, managing volumes. In any declarative system, there something, at some point, that will execute imperative commands.
What I would say is a more significant difference between terraform and kubernetes is that terraform has to deal with the fact that there will be unmanaged resources in the cloud environments it manages. This is pretty much impossible with kubernetes.
Thank you for this article. Although you focus on Terraform vs. Kubernetes, you mentioned CloudFormation in the first sentence as identical to Terraform. However, I wonder if there are some differences between them that are worth noting.
For example, my understanding of CloudFormation is that it is a managed service with server-side state and built-in automated rollbacks , whereas Terraform does not have these features by default but can achieve them with some additional configuration or tools. In particular, I’m interested in how Terraform handles rollbacks when something goes wrong during provisioning. I learned that Terraform labels the resources as “tainted” and tries to recreate them in the next execution plan, but it does not support rolling back to a previous state file. I wonder how this compares to CloudFormation’s rollback mechanism, which restores the stack to its original state on failure.
As you’ve said Kubernetes is fully declarative. However, I also learned that Kubernetes does support automated rollbacks for some resources like deployments, which can revert to a previous version if something goes wrong. I think this is a useful feature for managing infrastructure changes with minimal downtime and risk.
As per Going faster with continuous delivery, I feel like features like rollback are central to evaluating deployment engines. How do you compare these different approaches to managing infrastructure changes? Thank you.
Nice, brief explanation. I had this conversation with someone just last week, shame I couldn’t point them to this post.