this should be tagged “rant”, beause there is nothing productive in there, just good old DHH-style self-contradicting “it was better AND worse in the nineties”.
I (too?) believe it exists because UNIX never moved past being a multi-user operating system for a solitary minicomputer. Users and groups are vestigial organs. Berkeley sockets was a hack. And Plan9 failed.
(author here) Thanks for mentioning it :) I had collected some more links about fundamental design problems with Kubernetes, but I haven’t had time to put them on the blog. I’ll drop a couple good recent links here:
Kubernetes was originally intended to act as a container orchestration platform for stateless workloads, not stateful apps.
An important thing that seems to be lost in the hype is that Google’s Borg (which Kubernetes began as a clone of) was ALSO “intended to act as a container orchestration platform for stateless workloads, not stateful apps”. (I used it for several years across many teams starting ~2007, and tried to write something of my own many years ago.)
That’s because Borg was built for Google search in ~2003-2004. This app requires mind-boggling amounts of CPU/memory/storage/networking, but is very “stateless”. That is, the state was basically tiers of enormous immutable files that were inverting the whole web. There is no online state; there is no notion of consistency and thus no notion of transactions.
This was a world before Gmail or Google Docs existed as public products (Maps and YouTube too).
In the years after Borg was first deployed, the difficulty of running regular non-search, stateful apps on it became well known. The Broccoli man video went viral in Google because so many engineers (including myself) had experienced that pain:
(Although my memory is that it made fun of what I think of as the M requirements x N applications problem [1], not necessarily stateful apps, but both were problems. The status quo for stateful apps was to write your own application-specific distributed database, like GMail, or try to latch on to Bigtable, which was a non-relational DB with no transactions and limited querying ability. It was designed for search.)
I think not until Spanner was there a true solution for fine-grained and consistent state, in applications like Gmail or Google Docs. (FWIW Google ads was the early app other than search, and it the ran MySQL clusters for a long time – like completely into the tens / hundreds of billions of dollars range.)
Anyway, my point is that overall there is a huge problem to be solved, as Kubernetes didn’t solve it. It basically took all the Broccoli Man problems we had over 10 years ago and exported them to the rest of the world. It was NOT designed to run your applications well – hence all the layers of additional stuff you need to run the most basic workloads like a web app with a database.
The problem is to create abstractions that let you run distributed applications on your own hardware, not the cloud. All cloud providers and self-hosters like Hey need to solve this problem! This is similar to Unix letting you bring up say a typesetting package on any machine to create documents, without writing assembly code for that machine.
(Maybe it’s worth rabble rousing a bit more, as I noticed a few months ago that a nascent Kubernetes / systemd “competitor” called Aurae cited my “narrow waist” posts as inspiration:
Mild nit, because it really has no bearing on your well-written comment:
BigTable does have transactions, they’re just single row, where a row is an arbitrary collection of columns (KV pairs) grouped into column families. This property was/is leveraged pervasively to build transactional applications on BigTable. Sadly (or, perhaps blessedly) some of these systems never had papers published, and avoided opensource projects making their mistakes.
It probably doesn’t reveal too much to say that most applications you’re familiar with that you might guess are using Spanner today predated the wide deployment of Spanner.
I wonder as well!
But it’s not more closed than other runtimes in the end (a “traditional” language isn’t better at interop per se).
Also if the protocol used for distribution is open, competing runtimes could participate to the cluster.
Reinventing a whole ecosystem (again, applicable to non-distributed runtimes as well) is what’s risky.
I’ve been playing with Plan 9 recently and love it. I’m leaning towards the conclusion, though, that I wouldn’t give up Common Lisp programming to switch.
I’m not planning to be unemployed for that long ;-P
Seriously, beyond the time issue, there’s also the skill and experience. I’m still planning to (somehow) get mDNS working & integrated with 9front (whether they accept the patch or not is another matter …). But a SBCL port I think would be aiming a bit too high.
I’ve always liked Ansible for small-to-medium sized projects; it seems to strike a nice balance between simplicity and power. It doesn’t solve the provisioning problem but IME most shops very rarely need to provision new hardware or services, so doing it manually thought AWS, DO or whatever usually works well enough. Also, since it’s basically just an SSH for-loop on steroids it’s pretty easy to debug—just repeat the command manually!
Also, since it doesn’t pretend to be declarative you can easily do things like run a database migration, something that seems to be treated in the Cloudformation / K8s world as something weird and esoteric that require a bunch of esoteric hacks just to get working.
Am I the only one? I only ever hear about Ansible once in a blue moon, but I don’t know of any alternatives that work better for my use case. Or are there better alternatives I don’t know about?
I’ve tried Ansible only a little bit, but my experience with it is definitely not positive. For example, you say it doesn’t pretend to be declarative, but it kinda does, and you have to work your way around the bits where the declarativeness doesn’t work. For example, if you add a cron entry or a user or something like it in your script (or maybe you’re looping over a list and create the things in the list), when you take out one entry, that entry doesn’t get deleted. So you have to adjust your script to delete it conditionally (because the host may or may not have executed the previous version of your Ansible recipe). Also, you have to take care to make sure your recipe is idempotent, which is not always obvious.
And of course you may end up with different systems which have slightly different state. For example, Ansible supports multiple operating systems, but the subtle differences will still show up when you’re writing recipes. And what if users have manually made some changes one a subset of hosts that your Ansible recipe will run on?
At work, we’ve had a colleague write some Ansible tasks, which were basically unmaintainable and we had to scrap the entire thing and rewrite a different solution from scratch - but that might just have been the rest of the team’s unfamiliarity with the tool at the time.
Those are mostly fine. That’s just a normal three tier app, plus they outsourced authentication to a service (Cognito) and they outsourced syslog to a service (CloudWatch).
One thing that does suck is the usability of Application Gateway (this is the thing for invoking lambdas from HTTP requests) but eh oh well.
It seems somewhat ironic that the Rails-based application I look after (an instance of Redmine) is perhaps the most tedious application to build, install, operate, and most especially try desperately to scale, that I have the misfortune of operating. It’s true that there is some complexity in the Java ecosystem, but at least they had threads and you could turn maybe one or two knobs to begin the process of scaling up something running inside Glassfish or Tomcat or even Jetty.
As someone who came to Rails in 2006 from the J2EE world, I can assure you that Rails 2006 was far simpler than J2EE 2006. I literally left the WebSphere team because I couldn’t stand how awful IBM’s tech was by comparison. It took weeks for me to build in Java what took days in Rails.
Obviously this is a point in time comparison from a webapp developer’s perspective; 2023 might be different but I doubt it.
It’s more “natural” than “ironic”. Like most brilliantly conceived projects that succeed, the young rails project was far simpler, and then grew bloated and complex over time.
DHH only remembers the young rails, of course. We all have blind spots.
Rails was never simple. The early Rails was very hacky (some of it okay, some of it… was binding_of_caller) and was always a high-concept framework. It argued well for its complexity, as its cultural impact far beyond the Ruby language shows, but it was never simple. Note that at the time early Rails was released, working without an ORM and in single-file PHP apps without a framework for small things was still a regular thing. Then you threw it on a VPS and were ready. RoR always required knowledge of CGI/Apache (or other HTTP servers) configuration. Rails managed a lot of the complexity by a really well-chosen set of defaults.
It may seem like a useless distinction, but I would describe early Rails as minimal, e.g. it only supported mysql, but I would attribute that to the problem that no other database driver of reasonable quality was available, not simplicity thinking.
Thank you for the clarification. I never personally built anything with the early Rails versions, so my opinions are based on listening to other people talk about how simple and clean it was.
It’s funny, while I agree with his rant on k8s, I was really hoping it was going to be a “Rails – we’ve become what we hated” post, because it’s equally true that the death-star merchants of complexity have infiltrated Rails itself, and has been for some time.
Ruby has threads, too. Rails is thread-safe since some time ago. It doesn’t mean that everything in any Rails app is thread-safe, though (I don’t think that’s the case in Java either). So your mileage can vary with Redmine. Puma (Ruby web server often used with Rails apps) does a good job using threads with minimal setup. You have all the knobs to tweak Puma, or Ruby VM.
I’m not saying it’s easy to deploy/run/scale a Rails app. I’m just pointing out that it’s possible and probably not as hard as many imagine it to be.
In Rails things are made “simple” by implicitly applying all kinds of configuration and conventions, preferably ones that are not documented, or that only have simple example of the happy path. This works well if you start from scratch and want that happy path. It fails miserably in all other situations.
I still shudder when I think of the legacy Rails application that I had to maintain for a year. Granted, my predecessors would have made a mess on every platform (they had no development experience), but my biggest frustration was the time spent just figuring out why hell the the framework was doing this or that given the input it got.
If DHH thinks that is “simple” and J2EE is “not simple”, then I am going to buy a J2EE book tomorrow.
So, J2EE wasn’t simple partly because the platform didn’t support the things you needed in order to get by, and everyone wound up using Spring to get around it. Java EE 6, I could argue for, because they kind of learned some of their lessons and built enabling technologies that formerly were provided by Spring, which devolved into its own web of complexity. But their web story, JSF, was never simple because it was trying to abstract over more possibilities than HTML (like WAP and XUL and stuff) that never took off, so you wound up with a web abstraction framework that was more complex than the underlying technology.
“We’re trying to run an application on some hardware in a reasonably efficient manner, where we can easily redeploy it when it changes. “
Why not deploy on bare metal then ? Kubernetes is ….complex, no sane person will ever deny this. The obvious question before using it is….do you really need it ? Kubernetes exists to solve a very complex problem that most people simply don’t have it.
Kubernetes exists to solve a very complex problem that most people simply don’t have it.
That pretty much sums up the blog post – avoid complex “enterprisey” solutions if your needs are simple. It seems that Kubernetes is the answer to everyone’s problems.
Okay, I see why folks who sell services would gain from software being complex. More complexity, more work, more contracts. But what about buyers and users? I mean as a user I want simple, cheap stuff. My boss wants me to deliver results faster (and keep delivering them longer if they’re smart). And if I write something I then sell the maintenance of, it’s nice to reduce maintenance down to almost zero because the product just works.
Something pulls our industry towards unreasonable complexity. I’m not sure what that is to be honest. I guess a combination of short term thinking (causing us to use the first library or framework that vaguely looks like it may solve our problem), not knowing any better, or maybe seeking a challenge (learning a new technique or tool). Perhaps a lack of pride in simple stuff?
I believe the first step though is to recognise the problem. Fairly often in my career have I seen complexity that in hindsight turned out to be unneeded. More surprising perhaps is that many times other people don’t see it that way. They don’t have this feeling that we could do better, they can’t change their ways, they don’t question anything. I tend to get in trouble in that kind of environment, but the fact remains: many programmers are utterly blind to the mountain of unneeded complexity they have caused.
Perhaps this blindness is the real cause. Perhaps that’s why we get grumpy programmers moaning about things being better on 80’s personal computers with barely 64K of RAM. Because the only way to make programs work in those was to make it simple enough to fit. Now anything fits, so we gradually stopped caring, and then we gradually stopped seeing.
I think you underestimate psychology with these thing. People use Kubernetes and cloud computing, etc. because Google does, because it’s hip, because people pay ridiculous amounts of money for SREs and consultants (including myself) to solve problems they face with it. They also like it because blame can be easier shifted to AWS, Azure and GCP. Either their status says it’s down, there is an open ticket, the instance again stalls coming up, quota in that region can’t be increased right now (for a year at GCP). You can do this because you can easily call it “industry standard”, cause everyone is using it. Just the same as it’s normal that windows updates come in the way or the whole dev and marketing team is off, because their MacBooks are updating.
There is a silent agreement that if Google, Amazon, Microsoft or Apple has or causes problems it’s a higher power intervening.
If it’s your own system, a smaller provider, some less known software that is by a non-giant company it’s the Sysadmin’s fault.
So it’s your decision. Do you want to make good money, with a job where the demand just keeps growing, where you actually just manage outsourced hosting and essentially write contracts/specs in form e of terraform files or do you want to have a simple system that mostly just works, where you can automate yourself out of where you don’t have to change everything each version, have less demand, earn less money, but are woken up at 4am because some request somewhere failed and be blamed for physics, third parties, etc.
Oh and if you need another instance or more sroragry resources you order it no questions asked, but if you need another server for HA people are shocked.
Personally I made the decision from profiting by helping companies with their usually out of place Kubernetes/Cloud setups. (Not the person to decide they use it nor in the position to let them know it’s unwise)
For a personal setup or private company I’d never do that though.
There’s some good stuff coming out of this:
Things are standardized (see minio using S3)
When I started, if I had demanded stateless processes devs would probably have demanded me being fired (yet another thing only Amazon and Google can do)
If there’s something like terrafy in use you can more easily get an overview than using some old outdated diagram
I know there’s people complaining about not wanting to learn something new, but I think the majority of people complaining about complexity of software in their free time actually know what they talk about. At least that’s what it want to believe.
Something pulls our industry towards unreasonable complexity. I’m not sure what that is to be honest.
Also, don’t forget that, essentially, complexity is the norm. The first solution to a problem is usually jury-rigged and obtuse. It takes time (and, often, real-world experience) to pare down to the simple solution. Just because something is simple does not mean that it is obvious.
Just because something is simple does not mean that it is obvious.
Tell me about it. That’s probably a big driver indeed. I forgot to here, but I do often say that the simplest solution is rarely the most obvious. That’s bad enough, but the alarming thing here is that many people don’t believe me when I say it. Perhaps that’s part of the general blindness: they apply obvious solutions, they thing they’re simple because they’re obvious, and the eventual monstrosities they produce feel inevitable.
Something pulls our industry towards unreasonable complexity
If less complex solutions were economically viable we’d see more of them. The empirical deduction is either that complex systems are in fact required for solving business problems, but we might not be seeing why (an example of Chesterson’s fence) or that complex software solutions are not that big a problem in the grand scale of things.
While I agree with the article overall RoR isn’t the first thing I think about when I think about non-complex. Isn’t that why Sinatra, etc. were created? (wasn’t there in early times though)
I think complexity can succeed. However, it usually takes ages and usually means unnecessary complexity is being shaved off. This can happen by rewrites/clones with complexities being removed or by a long path. I think classical DBMSs went that route. And web frameworks did though in a very round about way though. I think containers/cloud computing/etc. might be in the middle of taking that route. The whole RPC paradigm resurrection is in their nth iteration of maybe becoming too complex.
Oh and programming languages (including their ecosystems), web and javascript frameworks are a great example of where complexity is being accumulated only to be replaced by something initially simpler, only to become complex again, because every feature and every use case, and every pattern and trend must be followed, to be “on par” only that at some point you have to approach every project, library, helper, version, differently and based on time of creation, when it starts to be considered that ugly/old thing, that people don’t dare to touch or learn anymore, not even if they fix stuff, also because their is too much old stuff (libraries, tutorials, etc.) that nobody wants to waste their time digging through when there is that hot new thing that promises to make everything simple.
And that’s where Kubernetes comes in. The solution to everything, well, if you can set it up, well you can use some managed kubernetes, well, if you use helm, well, if you keep upgrading (and sometimes have to rewrite quite a bit), if you use istio, but you wanna use operators, which totally isn’t reinventing the wheel for each software out there, …
But then I’m in camp “use what works for you”, just don’t lie to yourself on what does. And especially if software changes, you can change opinions. It’s not that black and white, and not falling for every ad or rant is a skill and shows expertise.
This is a curiously out-of-date sentiment from DHH. I’d expect this, by the title, to be a critique of WASI’s development rather than a complain about the deployed container ecosystem which most industry members know is complex to the point where almost every successful org using it succeeds because of which parts they don’t use.
Given this, the article feels more like a January 2020 article rather than a January 2023 article.
Are folks generally aware that there’s a massive change incoming in this space because of WASM+WASI? Relevant to these complaints: the goals include reclaiming cold-start performance, reducing I/O overhead (a longstanding issue with many container implementations), direct modeling of how components compose (as opposed to an implicit model), ease of scheduling, lower overhead, and language agnosticism with only a minimal performance cost. All of these will substantially change the requirements that systems like Kubernetes address, and in my opinion they will do so for the better.
Were I addressing Kubernetes problems and then publicly complaining about them, I think I’d be talking about how I am running towards this next generation design rather than trying to suggest the bad old days of VM shuffling or fleet-managed and statefully modified machines was my goal.
this should be tagged “rant”, beause there is nothing productive in there, just good old DHH-style self-contradicting “it was better AND worse in the nineties”.
you can suggest tags by clicking on the “suggest” link under the post
thanks! didn’t know that :)
Kubernetes is our generation’s Multics.
I (too?) believe it exists because UNIX never moved past being a multi-user operating system for a solitary minicomputer. Users and groups are vestigial organs. Berkeley sockets was a hack. And Plan9 failed.
Erlang is the Galapagos. 😉
(author here) Thanks for mentioning it :) I had collected some more links about fundamental design problems with Kubernetes, but I haven’t had time to put them on the blog. I’ll drop a couple good recent links here:
Kubernetes Was Never Designed for Batch Jobs - https://lobste.rs/s/z8gxe6/kubernetes_was_never_designed_for_batch - 4 months ago on lobste.rs
Kubernetes StatefulSets are Broken - https://www.plural.sh/blog/kubernetes-statefulsets-are-broken/
https://news.ycombinator.com/item?id=32439255 - 5 months ago
The tagline is VERY right:
An important thing that seems to be lost in the hype is that Google’s Borg (which Kubernetes began as a clone of) was ALSO “intended to act as a container orchestration platform for stateless workloads, not stateful apps”. (I used it for several years across many teams starting ~2007, and tried to write something of my own many years ago.)
That’s because Borg was built for Google search in ~2003-2004. This app requires mind-boggling amounts of CPU/memory/storage/networking, but is very “stateless”. That is, the state was basically tiers of enormous immutable files that were inverting the whole web. There is no online state; there is no notion of consistency and thus no notion of transactions.
This was a world before Gmail or Google Docs existed as public products (Maps and YouTube too).
In the years after Borg was first deployed, the difficulty of running regular non-search, stateful apps on it became well known. The Broccoli man video went viral in Google because so many engineers (including myself) had experienced that pain:
https://www.youtube.com/watch?v=3t6L-FlfeaI
(Although my memory is that it made fun of what I think of as the M requirements x N applications problem [1], not necessarily stateful apps, but both were problems. The status quo for stateful apps was to write your own application-specific distributed database, like GMail, or try to latch on to Bigtable, which was a non-relational DB with no transactions and limited querying ability. It was designed for search.)
I think not until Spanner was there a true solution for fine-grained and consistent state, in applications like Gmail or Google Docs. (FWIW Google ads was the early app other than search, and it the ran MySQL clusters for a long time – like completely into the tens / hundreds of billions of dollars range.)
Anyway, my point is that overall there is a huge problem to be solved, as Kubernetes didn’t solve it. It basically took all the Broccoli Man problems we had over 10 years ago and exported them to the rest of the world. It was NOT designed to run your applications well – hence all the layers of additional stuff you need to run the most basic workloads like a web app with a database.
The problem is to create abstractions that let you run distributed applications on your own hardware, not the cloud. All cloud providers and self-hosters like Hey need to solve this problem! This is similar to Unix letting you bring up say a typesetting package on any machine to create documents, without writing assembly code for that machine.
(Maybe it’s worth rabble rousing a bit more, as I noticed a few months ago that a nascent Kubernetes / systemd “competitor” called Aurae cited my “narrow waist” posts as inspiration:
https://medium.com/@kris-nova/why-fix-kubernetes-and-systemd-782840e50104
https://lobste.rs/s/dwui6j/why_fix_kubernetes_systemd – followup: I didn’t manage to crawl out from under a big pile of work, which is good because we got the garbage collector working :) I’m still looking for people who want to be paid to write this kind of code under our second NLnet grant – contact me if interested.
[1] Good references on this architectural problem:
https://www.oilshell.org/blog/2022/02/diagrams.html#these-abstractions-can-be-designed
https://lobste.rs/s/mjo19d/unix_microservice_platforms
Mild nit, because it really has no bearing on your well-written comment:
BigTable does have transactions, they’re just single row, where a row is an arbitrary collection of columns (KV pairs) grouped into column families. This property was/is leveraged pervasively to build transactional applications on BigTable. Sadly (or, perhaps blessedly) some of these systems never had papers published, and avoided opensource projects making their mistakes.
It probably doesn’t reveal too much to say that most applications you’re familiar with that you might guess are using Spanner today predated the wide deployment of Spanner.
and https://www.unison.cloud/ is another planet :)
Indeed. I wonder if a single language distributed runtime can succeed?
I wonder as well! But it’s not more closed than other runtimes in the end (a “traditional” language isn’t better at interop per se). Also if the protocol used for distribution is open, competing runtimes could participate to the cluster. Reinventing a whole ecosystem (again, applicable to non-distributed runtimes as well) is what’s risky.
I’ve been playing with Plan 9 recently and love it. I’m leaning towards the conclusion, though, that I wouldn’t give up Common Lisp programming to switch.
Why not both? All you gotta do is port SBCL …
I’m not planning to be unemployed for that long ;-P
Seriously, beyond the time issue, there’s also the skill and experience. I’m still planning to (somehow) get mDNS working & integrated with 9front (whether they accept the patch or not is another matter …). But a SBCL port I think would be aiming a bit too high.
In case you think DHH is exaggerating:
Been in industry almost 20 years. The solo tech stack diagram terrifies me. I don’t see how that is productive at all for side projects.
It’s not a side project, it’s for his business.
I was wondering why someone would make so terrifying for new programmers and why it was so widely shared. Now it all makes sense!
I’ve always liked Ansible for small-to-medium sized projects; it seems to strike a nice balance between simplicity and power. It doesn’t solve the provisioning problem but IME most shops very rarely need to provision new hardware or services, so doing it manually thought AWS, DO or whatever usually works well enough. Also, since it’s basically just an SSH for-loop on steroids it’s pretty easy to debug—just repeat the command manually!
Also, since it doesn’t pretend to be declarative you can easily do things like run a database migration, something that seems to be treated in the Cloudformation / K8s world as something weird and esoteric that require a bunch of esoteric hacks just to get working.
Am I the only one? I only ever hear about Ansible once in a blue moon, but I don’t know of any alternatives that work better for my use case. Or are there better alternatives I don’t know about?
I’ve tried Ansible only a little bit, but my experience with it is definitely not positive. For example, you say it doesn’t pretend to be declarative, but it kinda does, and you have to work your way around the bits where the declarativeness doesn’t work. For example, if you add a cron entry or a user or something like it in your script (or maybe you’re looping over a list and create the things in the list), when you take out one entry, that entry doesn’t get deleted. So you have to adjust your script to delete it conditionally (because the host may or may not have executed the previous version of your Ansible recipe). Also, you have to take care to make sure your recipe is idempotent, which is not always obvious.
And of course you may end up with different systems which have slightly different state. For example, Ansible supports multiple operating systems, but the subtle differences will still show up when you’re writing recipes. And what if users have manually made some changes one a subset of hosts that your Ansible recipe will run on?
At work, we’ve had a colleague write some Ansible tasks, which were basically unmaintainable and we had to scrap the entire thing and rewrite a different solution from scratch - but that might just have been the rest of the team’s unfamiliarity with the tool at the time.
Those are mostly fine. That’s just a normal three tier app, plus they outsourced authentication to a service (Cognito) and they outsourced syslog to a service (CloudWatch).
One thing that does suck is the usability of Application Gateway (this is the thing for invoking lambdas from HTTP requests) but eh oh well.
(Edit: deleted an incorrect paragraph.)
It seems somewhat ironic that the Rails-based application I look after (an instance of Redmine) is perhaps the most tedious application to build, install, operate, and most especially try desperately to scale, that I have the misfortune of operating. It’s true that there is some complexity in the Java ecosystem, but at least they had threads and you could turn maybe one or two knobs to begin the process of scaling up something running inside Glassfish or Tomcat or even Jetty.
As someone who came to Rails in 2006 from the J2EE world, I can assure you that Rails 2006 was far simpler than J2EE 2006. I literally left the WebSphere team because I couldn’t stand how awful IBM’s tech was by comparison. It took weeks for me to build in Java what took days in Rails.
Obviously this is a point in time comparison from a webapp developer’s perspective; 2023 might be different but I doubt it.
It’s more “natural” than “ironic”. Like most brilliantly conceived projects that succeed, the young rails project was far simpler, and then grew bloated and complex over time.
DHH only remembers the young rails, of course. We all have blind spots.
Rails was never simple. The early Rails was very hacky (some of it okay, some of it… was binding_of_caller) and was always a high-concept framework. It argued well for its complexity, as its cultural impact far beyond the Ruby language shows, but it was never simple. Note that at the time early Rails was released, working without an ORM and in single-file PHP apps without a framework for small things was still a regular thing. Then you threw it on a VPS and were ready. RoR always required knowledge of CGI/Apache (or other HTTP servers) configuration. Rails managed a lot of the complexity by a really well-chosen set of defaults.
It may seem like a useless distinction, but I would describe early Rails as minimal, e.g. it only supported mysql, but I would attribute that to the problem that no other database driver of reasonable quality was available, not simplicity thinking.
Thank you for the clarification. I never personally built anything with the early Rails versions, so my opinions are based on listening to other people talk about how simple and clean it was.
Happy to!
It’s funny, while I agree with his rant on k8s, I was really hoping it was going to be a “Rails – we’ve become what we hated” post, because it’s equally true that the death-star merchants of complexity have infiltrated Rails itself, and has been for some time.
Ruby has threads, too. Rails is thread-safe since some time ago. It doesn’t mean that everything in any Rails app is thread-safe, though (I don’t think that’s the case in Java either). So your mileage can vary with Redmine. Puma (Ruby web server often used with Rails apps) does a good job using threads with minimal setup. You have all the knobs to tweak Puma, or Ruby VM.
I’m not saying it’s easy to deploy/run/scale a Rails app. I’m just pointing out that it’s possible and probably not as hard as many imagine it to be.
In Rails things are made “simple” by implicitly applying all kinds of configuration and conventions, preferably ones that are not documented, or that only have simple example of the happy path. This works well if you start from scratch and want that happy path. It fails miserably in all other situations.
I still shudder when I think of the legacy Rails application that I had to maintain for a year. Granted, my predecessors would have made a mess on every platform (they had no development experience), but my biggest frustration was the time spent just figuring out why hell the the framework was doing this or that given the input it got.
If DHH thinks that is “simple” and J2EE is “not simple”, then I am going to buy a J2EE book tomorrow.
So, J2EE wasn’t simple partly because the platform didn’t support the things you needed in order to get by, and everyone wound up using Spring to get around it. Java EE 6, I could argue for, because they kind of learned some of their lessons and built enabling technologies that formerly were provided by Spring, which devolved into its own web of complexity. But their web story, JSF, was never simple because it was trying to abstract over more possibilities than HTML (like WAP and XUL and stuff) that never took off, so you wound up with a web abstraction framework that was more complex than the underlying technology.
“We’re trying to run an application on some hardware in a reasonably efficient manner, where we can easily redeploy it when it changes. “
Why not deploy on bare metal then ? Kubernetes is ….complex, no sane person will ever deny this. The obvious question before using it is….do you really need it ? Kubernetes exists to solve a very complex problem that most people simply don’t have it.
That pretty much sums up the blog post – avoid complex “enterprisey” solutions if your needs are simple. It seems that Kubernetes is the answer to everyone’s problems.
Or maybe the problem to everyone’s answers…?
Developers want complexity. I’m convinced of it.
They say they don’t. But suggest taking it away from them and see how they react.js
Okay, I see why folks who sell services would gain from software being complex. More complexity, more work, more contracts. But what about buyers and users? I mean as a user I want simple, cheap stuff. My boss wants me to deliver results faster (and keep delivering them longer if they’re smart). And if I write something I then sell the maintenance of, it’s nice to reduce maintenance down to almost zero because the product just works.
Something pulls our industry towards unreasonable complexity. I’m not sure what that is to be honest. I guess a combination of short term thinking (causing us to use the first library or framework that vaguely looks like it may solve our problem), not knowing any better, or maybe seeking a challenge (learning a new technique or tool). Perhaps a lack of pride in simple stuff?
I believe the first step though is to recognise the problem. Fairly often in my career have I seen complexity that in hindsight turned out to be unneeded. More surprising perhaps is that many times other people don’t see it that way. They don’t have this feeling that we could do better, they can’t change their ways, they don’t question anything. I tend to get in trouble in that kind of environment, but the fact remains: many programmers are utterly blind to the mountain of unneeded complexity they have caused.
Perhaps this blindness is the real cause. Perhaps that’s why we get grumpy programmers moaning about things being better on 80’s personal computers with barely 64K of RAM. Because the only way to make programs work in those was to make it simple enough to fit. Now anything fits, so we gradually stopped caring, and then we gradually stopped seeing.
I think you underestimate psychology with these thing. People use Kubernetes and cloud computing, etc. because Google does, because it’s hip, because people pay ridiculous amounts of money for SREs and consultants (including myself) to solve problems they face with it. They also like it because blame can be easier shifted to AWS, Azure and GCP. Either their status says it’s down, there is an open ticket, the instance again stalls coming up, quota in that region can’t be increased right now (for a year at GCP). You can do this because you can easily call it “industry standard”, cause everyone is using it. Just the same as it’s normal that windows updates come in the way or the whole dev and marketing team is off, because their MacBooks are updating.
There is a silent agreement that if Google, Amazon, Microsoft or Apple has or causes problems it’s a higher power intervening.
If it’s your own system, a smaller provider, some less known software that is by a non-giant company it’s the Sysadmin’s fault.
So it’s your decision. Do you want to make good money, with a job where the demand just keeps growing, where you actually just manage outsourced hosting and essentially write contracts/specs in form e of terraform files or do you want to have a simple system that mostly just works, where you can automate yourself out of where you don’t have to change everything each version, have less demand, earn less money, but are woken up at 4am because some request somewhere failed and be blamed for physics, third parties, etc.
Oh and if you need another instance or more sroragry resources you order it no questions asked, but if you need another server for HA people are shocked.
Personally I made the decision from profiting by helping companies with their usually out of place Kubernetes/Cloud setups. (Not the person to decide they use it nor in the position to let them know it’s unwise)
For a personal setup or private company I’d never do that though.
There’s some good stuff coming out of this:
I know there’s people complaining about not wanting to learn something new, but I think the majority of people complaining about complexity of software in their free time actually know what they talk about. At least that’s what it want to believe.
Also, don’t forget that, essentially, complexity is the norm. The first solution to a problem is usually jury-rigged and obtuse. It takes time (and, often, real-world experience) to pare down to the simple solution. Just because something is simple does not mean that it is obvious.
This. Complexity is the norm because it’s way easier to accomplish than simplicity.
Tell me about it. That’s probably a big driver indeed. I forgot to here, but I do often say that the simplest solution is rarely the most obvious. That’s bad enough, but the alarming thing here is that many people don’t believe me when I say it. Perhaps that’s part of the general blindness: they apply obvious solutions, they thing they’re simple because they’re obvious, and the eventual monstrosities they produce feel inevitable.
If less complex solutions were economically viable we’d see more of them. The empirical deduction is either that complex systems are in fact required for solving business problems, but we might not be seeing why (an example of Chesterson’s fence) or that complex software solutions are not that big a problem in the grand scale of things.
This is a lot of text for that basically says “I could not get Kubernetes to work”
Are you saying he created a Death Star of complexity for a simple sentence?
While I agree with the article overall RoR isn’t the first thing I think about when I think about non-complex. Isn’t that why Sinatra, etc. were created? (wasn’t there in early times though)
I think complexity can succeed. However, it usually takes ages and usually means unnecessary complexity is being shaved off. This can happen by rewrites/clones with complexities being removed or by a long path. I think classical DBMSs went that route. And web frameworks did though in a very round about way though. I think containers/cloud computing/etc. might be in the middle of taking that route. The whole RPC paradigm resurrection is in their nth iteration of maybe becoming too complex.
Oh and programming languages (including their ecosystems), web and javascript frameworks are a great example of where complexity is being accumulated only to be replaced by something initially simpler, only to become complex again, because every feature and every use case, and every pattern and trend must be followed, to be “on par” only that at some point you have to approach every project, library, helper, version, differently and based on time of creation, when it starts to be considered that ugly/old thing, that people don’t dare to touch or learn anymore, not even if they fix stuff, also because their is too much old stuff (libraries, tutorials, etc.) that nobody wants to waste their time digging through when there is that hot new thing that promises to make everything simple.
And that’s where Kubernetes comes in. The solution to everything, well, if you can set it up, well you can use some managed kubernetes, well, if you use helm, well, if you keep upgrading (and sometimes have to rewrite quite a bit), if you use istio, but you wanna use operators, which totally isn’t reinventing the wheel for each software out there, …
But then I’m in camp “use what works for you”, just don’t lie to yourself on what does. And especially if software changes, you can change opinions. It’s not that black and white, and not falling for every ad or rant is a skill and shows expertise.
This is a curiously out-of-date sentiment from DHH. I’d expect this, by the title, to be a critique of WASI’s development rather than a complain about the deployed container ecosystem which most industry members know is complex to the point where almost every successful org using it succeeds because of which parts they don’t use. Given this, the article feels more like a January 2020 article rather than a January 2023 article.
Are folks generally aware that there’s a massive change incoming in this space because of WASM+WASI? Relevant to these complaints: the goals include reclaiming cold-start performance, reducing I/O overhead (a longstanding issue with many container implementations), direct modeling of how components compose (as opposed to an implicit model), ease of scheduling, lower overhead, and language agnosticism with only a minimal performance cost. All of these will substantially change the requirements that systems like Kubernetes address, and in my opinion they will do so for the better.
Were I addressing Kubernetes problems and then publicly complaining about them, I think I’d be talking about how I am running towards this next generation design rather than trying to suggest the bad old days of VM shuffling or fleet-managed and statefully modified machines was my goal.