1. 9

Hello dear Lobsters,

I have not setup server deployments outside of building images for Azure VMs for a very long time (since before Docker, basically) and I’ve made a little web api that needs to be deployable and updatable in a simple manner, and this deployment and testing should preferably be possible through Jenkins.

What I do remember from when I did server deployments, is that it needs to be automated, or I will screw something up.

The servers it needs to be running on are Ubuntu 20.04 LTS.

What are the options today - I’ve heard of puppet, chef, ansible, NixOS - but I don’t know enough about either of them to know if they’re fit for what I’m doing. I’m going be to running less than 10 servers, by the way.

  1. 8

    I like using ansible for such simple setups with low deploy rates because it’s far lower complexity than puppet / chef / the major container orchestration systems. I like that it relies on simple ssh to interact with deploy targets. If you find that the up-front effort of specifying containers saves you testing, deployment or general tooling friction overall, I see that ansible has modules for deploying individual containers as well as a “docker compose” module for deploying sets of containers that will be able to communicate with each other based on well-known addresses, although I don’t have personal experience with this module it may be a nice stepping stone before your situation justifies something with higher complexity. In my mind the niche that things like puppet and chef have fulfilled has been shrinking with the popularity of other container orchestration systems, but they are still useful for certain mixtures of prior experience + deploy rate + actual changes being applied, and retain utility for things like machine configuration or managing the container orchestration system itself. I haven’t worked in the container orchestration space for a few years and frankly have enjoyed mostly forgetting it exists, so this info may be a little out of date, so if I’m off about any of this I’d appreciate it if someone corrected me.

    1. 3

      Can also recommend a proxmox + ansible stack with ZFS for backup and a ceph cluster for storage.

    2. 3

      I haven’t dug into it much yet but I’m looking at a “literate devops”-type thing with Emacs, Org mode, and Tramp mode. You run commands, write scripts, and other tasks all within an Org document so you’re able to capture the thoughts and reasoning behind the code blocks in a longer form rather than just having some terse and likely sparse comments. This process leaves you with snippets of code that can be moved to a script once the process is worked out but, more importantly, you’ll have a record of why you chose the paths you did. That will prove to be invaluable 5, 10 years down the road when the script inevitably breaks and you (or someone else) needs to revise it.


      This could of course be accomplished without the Emacs stack but I find the tight integration and fluidity it affords very compelling. The idea of doing everything from within a single environment and not needing to switch around between various terminals, browsers, text editors, IDEs, whatever is very appealing to me.

      1. 3

        Do you like Python and/or YAML? Use Ansible to configure your hosts.

        Do you love Ruby (like me)? Use Chef + Chef Solo for an Ansible-like SSH-based experience (I do my home server this way, and Airbnb ran on chef-solo for a long time)

        Use the aforementioned tools to do the minimum host set-up to manage Docker containers. I imagine you can easily get away with <100 lines of Ansible YAML. You probably don’t need systemd units because Docker can keep containers up itself (it seems wrong, but also, fine).

        I have separate unit test CI (uses GitHub actions or whatever), CI to Docker build/publish to registry (I use the free Docker registry CI for hobby stuff), and CI for the Chef/Ansible thingies (most expensive, uses vagrant).

        1. 3

          Puppet or Chef are great for more complex systems - managing whole companies from servers to switches and everything in between. But for your usecase, I agree that it sounds like either an external managed Container Plattform (EKS, GCP, DO, …) or 1-n virtual machine(s) on your favourite provider (number depends on expected traffic and uptime contraints; want zero downtime deployments?)

          For a small number of different machines, a simple ansible playbook (with or without docker or podman) seems like the right solution

          (source/context: I do SysOps with bare metal as well as cloud providers for a living)

          1. 2

            Yes, this seems like the way to go forward, I thankfully only have a single dependency, so I don’t think I need docker.

          2. 2

            If you can package the app into a (docker or other) container I think this makes testing and deployment on Jenkins & elsewhere a much simpler issue - you just need to install Docker and run the containers.

            Once you’ve a working container it should just be a case of installing docker on a server and running the app as a systemd unit. For this bit the tools you mentioned are all suitable - I’d also like to plug my own pyinfra if you know/prefer Python, Ansible is also pretty quick to get started with.

            Hope this helps a bit! Happy to answer any further questions :)

            1. 1

              Yes, I figured it would be nice to put it all into a Docker container.

              How do I then get the systemd unit file into the system - that’s sort of the missing piece for me right now.

              1. 3

                Some people suggested using Ansible. You can manage containers on your host with the docker_container module (https://docs.ansible.com/ansible/latest/modules/docker_container_module.html). You don’t need to use systemd to manage your containers as the docker daemon is essentially a systemd for containers.

                1. 2

                  Getting systemd to start/stop Docker containers is a bit of a pain. I think the easiest way to go is to just start your container with --restart unless-stopped (or docker update --restart unless-stopped $containername if you’ve already created a container).