While I like the trend, calling it “no PaaS required” when you have to set up your own servers and manage SSH keys is a bit of a stretch. It’s a big jump in operational complexity from a Heroku-like PaaS which doesn’t need any of that.
Doesn’t Heroku require you to set up SSH keys as well? Or set up your own “applications” (servers) before you can push to them?
Most VPS providers I’ve used let you save public SSH keys to your account, including at the point of creating the server, to (automatically) place them into the servers you create.
I don’t think it’s that big a jump, if at all. I’m not a big fan of Rails, but this criticism seems misplaced.
Doesn’t Heroku require you to set up SSH keys as well?
Not necessarily. Almost every PaaS has integrations for your forge to auto deploy. Having to set up an SSH key and pushing from local is the “classic” way of deploying to heroku, but unless you’re on a very small team you’re not doing that in practice.
Or set up your own “applications” (servers) before you can push to them?
What? Are you saying creating an application on a PaaS is functionally the same as setting up and maintaining a server?
Most VPS providers I’ve used let you save public SSH keys to your account,
As I said before, this works when you’re a solo dev or a very small team. As soon as you’re doing basic things like automating deployments or need fallbacks you need to do a lot more to just manage your SSH keys than you would with a PaaS.
Not necessarily. Almost every PaaS has integrations for your forge to auto deploy. Having to set up an SSH key and pushing from local is the “classic” way of deploying to heroku, but unless you’re on a very small team you’re not doing that in practice.
I mean, that’s just pushing the issue back a bit. Presumably, you had to set up your SSH keys on your forge. Sure, there’s some repetition, but unless you’re a huge company to the point that enough developers churn, I wouldn’t exactly call it a big jump in operational complexity, but when you grow like that, everything becomes more complex.
What? Are you saying creating an application on a PaaS is functionally the same as setting up and maintaining a server?
Well, no, but that’s what it sounds like they’re trying to solve. From what the article makes it sound like, you only need to create a server in your VPS provider’s web UI, get your SSH key onto it (which is what I was saying is not that much harder than a PaaS), and Rails 8 will basically do the server setup (and presumably, some basic level of maintenance) for you.
As I said before, this works when you’re a solo dev or a very small team. As soon as you’re doing basic things like automating deployments or need fallbacks you need to do a lot more to just manage your SSH keys than you would with a PaaS.
Just to clarify, you mentioned that before in a previous paragraph within the same comment, but not in an earlier comment in this conversation/thread.
So, as I said earlier in this same comment :P, everything becomes a lot more complex when you grow. They probably feel perfectly fine about releasing stuff that’ll work for the majority of their users, which won’t be large companies. Presumably, as time goes on, they’ll refine it for larger users.
Anyway, at really large scale, like GitHub, having dedicated ops instead of PaaS goes back to being worth it, so while they won’t use this, they’re probably not even using PaaS, anyway.
I guess PaaS is for a certain size band (medium businesses, or profitable or long runway small businesses), and this solution aims to serve a chunk of that band.
As soon as you’re doing basic things like automating deployments or need fallbacks you need to do a lot more to just manage your SSH keys than you would with a PaaS.
That’s fair. When you said, “you have to set up your own servers and manage SSH keys”, I was thinking about managing SSH keys for those “own servers”, not extra stuff like CI, CD, branch deploys, and so on (not sure what you mean by fallbacks), where you need to manage your SSH keys on further machines and create SSH keys for machines instead of people, etc.
I see where you’re coming from now. I guess a better title would be, “No More PaaS Required (For Many)”. But I can see a lot of this stuff being solvable with the same infra/foundation they developed here. I can also see it being made a lot easier for teams to solve themselves using these tools.
I’m just speculating now, though, I was initially responding to your point about managing the production app servers and the SSH keys for those.
Presumably, you had to set up your SSH keys on your forge.
There is a huge difference between dev SSH keys that individual devs have and a deploy SSH key that can be used to deploy to the live, production system. One of them is managed by the dev locally and is cheap to cycle and manage. The other has to be managed by the team or company, is shared and can be time consuming or difficult to cycle. Ask me how I know!
Just to clarify, you mentioned that before in a previous paragraph within the same comment,
My bad - I had to edit the comment and missed this, I’d mentioned team sizes before in a previous version of my reply itself.
From what the article makes it sound like, you only need to create a server in your VPS provider’s web UI,
This is only true in the very very very simple case where you have only one server, and this still ignores other moving parts like the database or a cache service. Also, sure, you can create a server from the web UI, but what about maintaining it? What about regular software updates, security patches, database updates, etc? These are all things PaaS free you up from thinking about. Saying “you can create a VM and you’re all set!” Is only true if you literally only need a fixed number of VMs, are using SQLite, and don’t care about any operational resiliency or even what happens to that VM a week, month or year from now.
And please, please, please don’t think solo devs or small teams don’t need to do that of this. For my hobby projects I mostly use VMs, and very often I spend more time dealing with ops instead of programming. I don’t mean to shill for PaaS services, but I also think people seriously underestimate the ops work it takes to keep a service going, and how much a PaaS can free you up to just code.
There is a huge difference between dev SSH keys that individual devs have and a deploy SSH key that can be used to deploy to the live, production system.
To be clear, I wasn’t suggesting that the same keys would be used, merely that integrating with a forge doesn’t get rid of the requirement to manage SSH keys. For small teams (as you say), where SSH keys are managed manually, including deploy keys and other keys for machines, I don’t see why this couldn’t remove the need for a PaaS.
I don’t think we’re really disagreeing. I’ll say again:
I see where you’re coming from now. I guess a better title would be, “No More PaaS Required (For Many)”.
With Heroku you can add someone to the team and they are able to deploy, get a shell, etc. You can limit permissions as needed or use Single Sign On.
With Kamal you need to get their SSH keys on the server somehow and give them the list of IP addresses (then keep both of those things up to date). You can’t really set granular permissions and you are giving them (effectively root) access to the host, not just the inside of the container.
You could pair Kamal with Tailscale to get something a little closer to the Heroku experience.
Well, no, but that’s what it sounds like they’re trying to solve. From what the article makes it sound like, you only need to create a server in your VPS provider’s web UI, get your SSH key onto it (which is what I was saying is not that much harder than a PaaS), and Rails 8 will basically do the server setup (and presumably, some basic level of maintenance) for you.
What exactly is it doing for you? Is it configuring logging and monitoring somehow? What about network firewalls? Databases (TFA mentions SQLite support but absent some clustering support, that’s an antifeature)? There’s a lot of stuff that PaaSes provide that I have a hard time imagining this software doing properly.
My own mental model is IaaS | PaaS | SaaS. To me, SSH keys and servers is infrastructure. To me, PaaS is a big mystery in the industry right now. I think many are trying to solve it. IDPs, Kubernetes, Backstage and others. One isn’t better than the other and you can run the “aaS types” in parallel. But things at the PaaS level of abstraction isn’t IaaS. When you use k8s you don’t and cannot SSH into your servers. You don’t even call them servers.
Each layer has trade-offs but I find the middle PaaS one especially confusing right now. I think there’s a need for a mid-complexity thing, something like Nomad. So, I think Rails 8’s opinions are pretty interesting given the alternatives.
I think of your PaaS layer as two different layers.
K8s, Nomad, Swarm, etc. are orchestrators. They do provide an abstraction on top of the compute but is still fairly low level.
Tools like Kamal provide the developer facing abstraction: apps, deploys, rollbacks, etc. Kamal is not the best example but you can imagine an internal developer platform with everything Heroku offers on top of the company’s K8s cluster.
I think that split will continue to happen just because of Conways law. And if it does fully hosted PaaS makes less sense for most orgs.
i tried using kamal outside the rails ecosystem and it didn’t feel great. i love the “point-to-ip-and-deploy” approach though and the v8 changelog is great, especially sqlite by default! i wish something like this existed in a type-safer language.
it wasn’t possible to customize the paths of the .env and the .yaml files. the locations probably make sense in a rails app, but my non rails app doesn’t have production secrets in the local .env.
also i had to run kamal as a docker container moumting the docker socket because i didn’t want to deal with ruby. is this really the best way to distribute ruby as executable?
overall kamal felt opinionated, its development is clearly driven by a very specific use case. i couldn’t figure out how to get it to work with an existing typescript monorepo.
As much as I like the general idea, I’m worried about the rosy picture dhh paints. Running own infra is not rocket science, but you need to have a bit more knowledge and the PaaS abstractions are quite useful and save time. I’m worried people will jump on the new trend without understanding the details and get burned in interesting ways and without good enough backup systems. Kind of like early EC2 times when people kept losing production data and killing their products, because they didn’t understand the ephemeral/permanent partitions split.
But I’m glad that considering alternatives to cloud is being a bit normalised. Some posts about the actual tradeoffs would be great to see these days.
The big change seems to be the 3 SOLID things – pubsub, cache, and queue
I do think that will make it easier to run your own infrastructure
But yeah backups are good point, and it’s probably too early to draw any conclusions
But it seems like an interesting experiment, and they also have some “skin in the game”, with https://once.com/
As far as I understand, they are selling some of their old products as “shrink wrap”, moving away from the SaaS model. And to eliminate the devops burden, it’s a Docker image with all storage as sqlite. So no “moving parts” apparently
All it needs is the IP addresses for a set of servers with your SSH key deposited, and you’ll be ready to go into production in under two minutes.
Kamal can do this so easily because Rails already comes with a highly efficient and tuned Dockerfile for turning your application into a production-ready container image out of the box. All you need to bring is your own container registry account
I was only going by their wording, “just like any other PaaS”. They say that, but preface it with, “soo..”, as if to say that makes it pointless because it only matches a PaaS instead of beating it. They never mentioned SSM.
I had to look up SSM, and it seems to be specific to AWS, and not really anything to do with PaaS per se.
I’m confused about how they set up a database, log exfiltration, monitoring, etc for you. TFA mentions something about SQLite instead of Postgres/MySQL, but that seems like a step backwards unless there’s some special clustering support for SQLite (not to mention backups and failover) in there.
I think kamal is occupying some in-between space between dokku and ansible/chef/puppet/etc. Like it’s trying to save you some work in those deployment tools, but not going as far as “push to a specific remote and you’re in prod”. And a little like kamal is trying to help you reduce the # of services you need to manage while dokku is just trying to put a facade on your own infrastructure that makes them as almost as easy to manage as heroku would.
I’m not sure I’ve got the differences between them modeled properly in my own mind yet, but I think that’s close.
I tried out Kamal for a deployment job I had but it wasn’t really well suited for Rust and the configuration was extremely shaky/undocumented/obscure. It definitely was not a 2 minute job.
Also I could not for the life of me figure out how to best install a Ruby devtool on my machine and felt it was just a rather anachronistic concept.
I use asdf for all languages on my dev machine. I think it’s been about 5 years straight I’ve been using asdf and to me it’s great. I rarely think about it. To me, it’s solved. You get a single CLI interface instead of having node version manager and ruby version manager.
You can set shell defaults or activate them temporarily just like the single language managers (nvm etc). You can bind projects with .tool_versions so it switches on cd. You get project isolation on the runtime with asdf and dependency isolation with a package manager. My projects don’t get weird and my dev machine doesn’t get weird. I can list all runtimes for all languages, for when security news happens or if I wonder “do I have Python 8.0 installed?”
So, when I wrote an asdf upgrader shell function to replace a language (hey, replace nodejs 42 with nodejs 43) it works for all languages because asdf is a language manager not a node language manager.
While I like the trend, calling it “no PaaS required” when you have to set up your own servers and manage SSH keys is a bit of a stretch. It’s a big jump in operational complexity from a Heroku-like PaaS which doesn’t need any of that.
Doesn’t Heroku require you to set up SSH keys as well? Or set up your own “applications” (servers) before you can push to them?
Most VPS providers I’ve used let you save public SSH keys to your account, including at the point of creating the server, to (automatically) place them into the servers you create.
I don’t think it’s that big a jump, if at all. I’m not a big fan of Rails, but this criticism seems misplaced.
Not necessarily. Almost every PaaS has integrations for your forge to auto deploy. Having to set up an SSH key and pushing from local is the “classic” way of deploying to heroku, but unless you’re on a very small team you’re not doing that in practice.
What? Are you saying creating an application on a PaaS is functionally the same as setting up and maintaining a server?
As I said before, this works when you’re a solo dev or a very small team. As soon as you’re doing basic things like automating deployments or need fallbacks you need to do a lot more to just manage your SSH keys than you would with a PaaS.
I mean, that’s just pushing the issue back a bit. Presumably, you had to set up your SSH keys on your forge. Sure, there’s some repetition, but unless you’re a huge company to the point that enough developers churn, I wouldn’t exactly call it a big jump in operational complexity, but when you grow like that, everything becomes more complex.
Well, no, but that’s what it sounds like they’re trying to solve. From what the article makes it sound like, you only need to create a server in your VPS provider’s web UI, get your SSH key onto it (which is what I was saying is not that much harder than a PaaS), and Rails 8 will basically do the server setup (and presumably, some basic level of maintenance) for you.
Just to clarify, you mentioned that before in a previous paragraph within the same comment, but not in an earlier comment in this conversation/thread.
So, as I said earlier in this same comment :P, everything becomes a lot more complex when you grow. They probably feel perfectly fine about releasing stuff that’ll work for the majority of their users, which won’t be large companies. Presumably, as time goes on, they’ll refine it for larger users.
Anyway, at really large scale, like GitHub, having dedicated ops instead of PaaS goes back to being worth it, so while they won’t use this, they’re probably not even using PaaS, anyway.
I guess PaaS is for a certain size band (medium businesses, or profitable or long runway small businesses), and this solution aims to serve a chunk of that band.
That’s fair. When you said, “you have to set up your own servers and manage SSH keys”, I was thinking about managing SSH keys for those “own servers”, not extra stuff like CI, CD, branch deploys, and so on (not sure what you mean by fallbacks), where you need to manage your SSH keys on further machines and create SSH keys for machines instead of people, etc.
I see where you’re coming from now. I guess a better title would be, “No More PaaS Required (For Many)”. But I can see a lot of this stuff being solvable with the same infra/foundation they developed here. I can also see it being made a lot easier for teams to solve themselves using these tools.
I’m just speculating now, though, I was initially responding to your point about managing the production app servers and the SSH keys for those.
There is a huge difference between dev SSH keys that individual devs have and a deploy SSH key that can be used to deploy to the live, production system. One of them is managed by the dev locally and is cheap to cycle and manage. The other has to be managed by the team or company, is shared and can be time consuming or difficult to cycle. Ask me how I know!
My bad - I had to edit the comment and missed this, I’d mentioned team sizes before in a previous version of my reply itself.
This is only true in the very very very simple case where you have only one server, and this still ignores other moving parts like the database or a cache service. Also, sure, you can create a server from the web UI, but what about maintaining it? What about regular software updates, security patches, database updates, etc? These are all things PaaS free you up from thinking about. Saying “you can create a VM and you’re all set!” Is only true if you literally only need a fixed number of VMs, are using SQLite, and don’t care about any operational resiliency or even what happens to that VM a week, month or year from now.
And please, please, please don’t think solo devs or small teams don’t need to do that of this. For my hobby projects I mostly use VMs, and very often I spend more time dealing with ops instead of programming. I don’t mean to shill for PaaS services, but I also think people seriously underestimate the ops work it takes to keep a service going, and how much a PaaS can free you up to just code.
To be clear, I wasn’t suggesting that the same keys would be used, merely that integrating with a forge doesn’t get rid of the requirement to manage SSH keys. For small teams (as you say), where SSH keys are managed manually, including deploy keys and other keys for machines, I don’t see why this couldn’t remove the need for a PaaS.
I don’t think we’re really disagreeing. I’ll say again:
With Heroku you can add someone to the team and they are able to deploy, get a shell, etc. You can limit permissions as needed or use Single Sign On.
With Kamal you need to get their SSH keys on the server somehow and give them the list of IP addresses (then keep both of those things up to date). You can’t really set granular permissions and you are giving them (effectively root) access to the host, not just the inside of the container.
You could pair Kamal with Tailscale to get something a little closer to the Heroku experience.
What exactly is it doing for you? Is it configuring logging and monitoring somehow? What about network firewalls? Databases (TFA mentions SQLite support but absent some clustering support, that’s an antifeature)? There’s a lot of stuff that PaaSes provide that I have a hard time imagining this software doing properly.
Classic 37s marketing.
My own mental model is
IaaS | PaaS | SaaS. To me, SSH keys and servers is infrastructure. To me, PaaS is a big mystery in the industry right now. I think many are trying to solve it. IDPs, Kubernetes, Backstage and others. One isn’t better than the other and you can run the “aaS types” in parallel. But things at the PaaS level of abstraction isn’t IaaS. When you use k8s you don’t and cannot SSH into your servers. You don’t even call them servers.Each layer has trade-offs but I find the middle PaaS one especially confusing right now. I think there’s a need for a mid-complexity thing, something like Nomad. So, I think Rails 8’s opinions are pretty interesting given the alternatives.
I think of your PaaS layer as two different layers.
K8s, Nomad, Swarm, etc. are orchestrators. They do provide an abstraction on top of the compute but is still fairly low level.
Tools like Kamal provide the developer facing abstraction: apps, deploys, rollbacks, etc. Kamal is not the best example but you can imagine an internal developer platform with everything Heroku offers on top of the company’s K8s cluster.
I think that split will continue to happen just because of Conways law. And if it does fully hosted PaaS makes less sense for most orgs.
i tried using kamal outside the rails ecosystem and it didn’t feel great. i love the “point-to-ip-and-deploy” approach though and the v8 changelog is great, especially sqlite by default! i wish something like this existed in a type-safer language.
What about Kamal didn’t feel great?
it wasn’t possible to customize the paths of the .env and the .yaml files. the locations probably make sense in a rails app, but my non rails app doesn’t have production secrets in the local .env.
also i had to run kamal as a docker container moumting the docker socket because i didn’t want to deal with ruby. is this really the best way to distribute ruby as executable?
overall kamal felt opinionated, its development is clearly driven by a very specific use case. i couldn’t figure out how to get it to work with an existing typescript monorepo.
https://www.sidekickdeploy.com/ is similar to kamal but less opinionated.
Oh for sure. Yeah, that’s like, these folks whole thing.
As much as I like the general idea, I’m worried about the rosy picture dhh paints. Running own infra is not rocket science, but you need to have a bit more knowledge and the PaaS abstractions are quite useful and save time. I’m worried people will jump on the new trend without understanding the details and get burned in interesting ways and without good enough backup systems. Kind of like early EC2 times when people kept losing production data and killing their products, because they didn’t understand the ephemeral/permanent partitions split.
But I’m glad that considering alternatives to cloud is being a bit normalised. Some posts about the actual tradeoffs would be great to see these days.
The big change seems to be the 3 SOLID things – pubsub, cache, and queue
I do think that will make it easier to run your own infrastructure
But yeah backups are good point, and it’s probably too early to draw any conclusions
But it seems like an interesting experiment, and they also have some “skin in the game”, with https://once.com/
As far as I understand, they are selling some of their old products as “shrink wrap”, moving away from the SaaS model. And to eliminate the devops burden, it’s a Docker image with all storage as sqlite. So no “moving parts” apparently
Why do they want to replace nginx with Thruster? Nginx is really good at what it does and I’ve found it to be reasonably easy to use.
I’m just glad they had the good sense not to write Thruster in Ruby.
Truly. I was relieved when I saw it was written in Go.
It does smell like NIH a bit. There’s lots of different good proxies they could’ve used and basically all of them can do seamless updates.
That… came sooner than I was expecting!
Exciting times for Rails!
soo.. just like any other PaaS then?
Except, you know… without the price markup. As mentioned in the article.
So if they’ve made the cheaper option equivalent to the high markup option, which is what you’re saying, then they’ve succeeded.
I wouldn’t call SSH equivalent to SSM at all tbh. Maybe they’ve made it as operationally simple, but security-wise I’m doubtful.
I was only going by their wording, “just like any other PaaS”. They say that, but preface it with, “soo..”, as if to say that makes it pointless because it only matches a PaaS instead of beating it. They never mentioned SSM.
I had to look up SSM, and it seems to be specific to AWS, and not really anything to do with PaaS per se.
I’m assuming they’re comparing to cloud PaaS offerings.
I’m confused about how they set up a database, log exfiltration, monitoring, etc for you. TFA mentions something about SQLite instead of Postgres/MySQL, but that seems like a step backwards unless there’s some special clustering support for SQLite (not to mention backups and failover) in there.
W.r.t. to SQLite, take a look at https://www.youtube.com/watch?v=wFUy120Fts8&pp=ygUMc3FsaXRlIHJhaWxz or https://fractaledmind.github.io/2023/12/23/rubyconftw/ which is the linked Rails World talk on YT but as a written blog post. I think it’ll answer most of your questions about the database part.
Except it’s your own server! I don’t mind deploying to some form of PaaS, but having stuff like this included is definitely a breath of fresh air.
https://dokku.com/ is also available as a self hosted PaaS and it’s very framework-agnostic.
I think kamal is occupying some in-between space between dokku and ansible/chef/puppet/etc. Like it’s trying to save you some work in those deployment tools, but not going as far as “push to a specific remote and you’re in prod”. And a little like kamal is trying to help you reduce the # of services you need to manage while dokku is just trying to put a facade on your own infrastructure that makes them as almost as easy to manage as heroku would.
I’m not sure I’ve got the differences between them modeled properly in my own mind yet, but I think that’s close.
Now just to make a web server that can do this without chewing through resources like there’s no tomorrow
[Comment removed by author]
I tried out Kamal for a deployment job I had but it wasn’t really well suited for Rust and the configuration was extremely shaky/undocumented/obscure. It definitely was not a 2 minute job.
Also I could not for the life of me figure out how to best install a Ruby devtool on my machine and felt it was just a rather anachronistic concept.
asdf or mise abstract quite a bit of it for you. For Ruby and almost anything else.
I stopped using those now preferring just to stick with nvm/uv/rustup.
fnm is the better nvm.
It never ends!
And fnm pollutes your filesystem with thousands of symlinks over time. So I switched to mise… uv can detect mise-installed python fwiw.
In that case you’re probably going to get along better with
ruby-install+chrubythan other install options.I use asdf for all languages on my dev machine. I think it’s been about 5 years straight I’ve been using
asdfand to me it’s great. I rarely think about it. To me, it’s solved. You get a single CLI interface instead of having node version manager and ruby version manager.You can set shell defaults or activate them temporarily just like the single language managers (nvm etc). You can bind projects with
.tool_versionsso it switches oncd. You get project isolation on the runtime withasdfand dependency isolation with a package manager. My projects don’t get weird and my dev machine doesn’t get weird. I can list all runtimes for all languages, for when security news happens or if I wonder “do I have Python 8.0 installed?”So, when I wrote an asdf upgrader shell function to replace a language (hey, replace nodejs 42 with nodejs 43) it works for all languages because asdf is a language manager not a node language manager.
Heroku tutorial (cause sometimes a PaaS is still kinda nice).
https://devcenter.heroku.com/articles/getting-started-with-rails8
Excellent!