1. 38

  2. 14

    As someone who paid a fair bit of attention to the early docker world, and now seeing its commodification am left wondering “what was it”, I think this article does a good job of explaining it. What it doesn’t explain is… I was around at that early redhat time, when it was small, when you could shake Bob Young’s hand at a Linux meetup. Heck, I remember when google was a stanford.edu site… the question in my mind is… why did redhat and google succeed (as corporate entities) and docker not so much? Perhaps it was the locking in of the company name and the core tech? Perhaps the world of 2010-2020 was far more harsh to smaller businesses, perhaps they just overshot by trying to fight their competitors instead of partnering with them. That will probably have to wait for a HBR retrospective, but I’m not 100% psyched that the big incumbents won this.

    1. 13

      Docker lost, as I understand it, because of commoditisation. There’s a bunch of goo in Linux to try to emulate FreeBSD jails / Solaris Zones and Docker provided some tooling for configuring this (now fully subsumed by containerd / runc), for building tarballs (not really something that needs a big software stack), and for describing how different tarballs should be extracted and combined using overlay filesystems (useful, but should not be a large amount of code and now largely replaced by the OCI format and containerd). Their two valuable things were:

      • A proprietary build of a project that they released as open source that provided tooling for building container images.
      • A repository of published container images.

      The first of these is not actually more valuable than the open source version and is now quite crufty and so now has a load of competitors. The second is something that they tried to monetise, leaving them open to competitors who get their money from other things. Any cloud provider has an incentive to provide cheap or free container registries because a load of the people deploying the containers will be spending money to buy cloud resources to run them. Docker didn’t have any equivalent. Running a container registry is now a commodity offering and Docker doesn’t have anything valuable to couple their specific registry to that would make it more attractive.

      1. 9

        I wrote a bit about that here – Docker also failed to compete with Heroku, under its former name dotCloud.


        I don’t think the comparison to Google makes much sense. I mean Google has a totally different business that prints loads of money. If Docker were a subdivision of Google, it could lose money for 20 years and nobody would notice.

        As for Red hat, this article has some interesting experiences:

        Why There Will Never Be Another RedHat: The Economics Of Open Source


        To make matters worse, the more successful an open source project, the more large companies want to co-opt the code base. I experienced this first-hand as CEO at XenSource, where every major software and hardware company leveraged our code base with nearly zero revenue coming back to us. We had made the product so easy to use and so important, that we had out-engineered ourselves.

        (Although I don’t think Docker did much engineering. It wasn’t that capable a product. It could have been 30 to 100 people at Google implementing it, etc. Previous thread: https://lobste.rs/s/kj6vtn/it_s_time_say_goodbye_docker)

        1. 3

          I appreciate the article on RedHat. It has certainly opened my eyes to the troubles with their business model, which I had admired in the past. (I suppose it is still admirable, but now at least I know why there aren’t more companies like it.)

          The back half of the article is strange, though. I’m not sure what I’m supposed to learn about building a new business based around open source by looking at Microsoft, Amazon or Facebook. While they all contribute open source code now, they did not build their businesses by selling proprietary wrappers around open source products as far as I know. And given the enormity of those companies, it seems very hard to tell how feasible it would be to copy that behavior on a small scale. Github seems like a reasonable example of a company monetizing open source, however. It is at least clear that their primary business relies on maintaining git tools. I just wish the article included a few more examples of companies to look up to. Perhaps some lobsters have ideas.

          1. 5

            I just wish the article included a few more examples of companies to look up to

            To a first approximation, there are no companies to look up to.

            1. 2

              I feel like some of the companies acquired by RedHat might be valid examples. I expect that the ones that are still recognizable as products being sold had a working model, but I don’t know what their earnings were like.

            2. 3

              the biggest ones I can think of, not mentioned, are mongo and elastic… redis may go public soon, there are lots of corps around data storage and indexing that to some extent keep their core product free. There might be more. If you look at interesting failures, going back to the early days, LinuxCare was a large service oriented company that had a giant flop, as did VA Linux (over a longer time scale):

              linuxcare https://www.wsj.com/articles/SB955151887677940572

              va linux https://www.channelfutures.com/open-source/open-source-history-the-spectacular-rise-and-fall-of-va-linux

              1. 2

                Appreciate it, thanks.

          2. 8

            same question, I think, could be asked why netflix succeeded but blockbuster failed, both were doing very similar thing. It seems that market success consists of chains / graphs of very small incremental decisions. The closer decisions are to the companies ‘pivot time’, the more impactful they seem to be.

            And, at least in my observation, paying well and listening to well-rounded+experienced and risk-taking folks – who join your endeavor early, pays with huge dividends later on.

            In my subjective view, docker failed to visualize and execute on the overall ecosystem around their core technology. Folks who seem to have that vision (but perhaps, not always the core technology) are the ones at hashicorp. They are not readhat by any means, but any one of their oss+freemium products seem to have good cohesive and ‘efficient’ vision around the ecosystem in this space. (where by ‘efficient’ I mean that they do not make too many expensive and user-base jarring missteps).

            1. 1

              could be asked why netflix succeeded but blockbuster failed, both were doing very similar thing

              I’m not sure I agree. Coincidentally, there’s a YT channel that I follow that did a decent overview on both of them:

            2. 3

              the question in my mind is… why did redhat and google succeed (as corporate entities) and docker not so much?

              Curating a Linux distribution and keeping the security patches flowing seamlessly is hard work, which made Red Hat valuable. Indexing the entire Internet is also clearly a lot of hard work.

              By comparison, what Docker is doing as a runtime environment is just not that difficult to replace.

              1. 1

                I kinda feel like this is the ding ding ding answer… when your project attempts to replicate a project going on inside of a BigCo, you will have a hard time preventing embrace and extend. Or perhaps, if you are doing that, keep your company small, w/ limited debt, because you may find a niche in the future, but you can’t beat the big teams at the enterprise game, let alone a federation of them.

              2. 3

                My opinion on this is that both Google and Redhat are much closer to the cloud and the target market than Docker is/was.

                Also, I thought that Docker was continuously trying to figure out how to make a net income. They had Docker Enterprise before it was sold off, but imo I’m not sure how they were aiming to bring in income. And a startup without income is destined to eventually close up.

                1. 2

                  I think we all know our true desires we are just left to discover them.-

                  Lets not forget, The Docker Timeline:

                  • Started in 2013.
                  • Got open-source recognition.
                  • Got increased public use in 2015/2016.
                  • In 2017. project renamed from Docker to Moby. Mistake 1.
                  • In 2018. started requiring User Registration on DockerHub. Mistake 2.
                  • In 2019. Docker Database has been hacked which exposed user. Mistake 3.
                  • In 2020. Docker finally died and awaits new reborn. Good bye.

                  When I think about it, I’m not even mad. Hail death of Docker.

                2. 7

                  There is some truth here. However, this article loses a bit of history there by calling Docker just an API and ease of use tool on top of containerd. Containerd was explicitly refactored out of the original Docker to make it easier for low level tools (K8s, primarily) to interact with the core bits. Runc was factored out for purposes of satisfying the increasingly vocal (I’m not saying wrong) community who wanted a common runtime spec. These were all changes driven from outside and were never how the original Docker architecture worked.

                  1. 6

                    To briefly summarize, this was an attack on Red Hat’s model of backporting patches to older versions of software (also known as “enterprise support”). Red Hat at that time shipped a version of Docker that was just slightly behind the latest cut, whereas Docker shipped their latest.

                    I have never worked for either Docker or Red Hat, but my understanding was that the conflict was related primarily to Red Hat shipping patches in their build of Docker that were rejected upstream, such as this one about overriding the default registry, this one that adds partial audit logging to syslog, this one allowing hook specification to enable systemd integration, and this one for injecting RHEL licensing secrets to containers. Many distributions (including Amazon Linux, which I work on) backport bugfixes, but fewer distros include wholesale-rejected feature patches in their builds; I can understand an upstream project being concerned about a very different experience for users of those distro-maintained packages when the set of supported features are different. This list was last updated in 2016, but gives a bunch of background on the types of patches Red Hat carried at the time.

                    1. 4

                      Yeah it was pretty clear at the time that RedHat initiated the problem by essentially attempting to strongarm the Docker project into decisions they didn’t want to make. Not to mention that RedHat’s rejection of AUFS in their kernels forced Docker to support RedHat’s implementation of devicemapper as a filesystem driver… which was a complete mess for a long time, and made everyone’s user experience worse.

                    2. 6

                      I’m curious how does dockershim being removed from K8s leads to a conclusion that Docker Inc as a company is dying? As explained by many, Kubernetes team took that step to remove the bloat which was created by Docker in the codebase. But do you think people will go back to stop using docker CLI altogether and write 10 lines of bash script to spin up a new container, network etc? docker run is a UX layer on those containerd commands and I don’t see why people will stop using it just because K8s decided to remove the “dockershim” module. And how any of this has an affect on Docker Inc, that I’m still unable to understand AFAIK docker the CLI is open source and obv doesn’t generate any revenues for Docker Inc (which is what matters when we are talking about a company!)

                      1. 5

                        I think the reason that it points that direction is that there are multiple k8s providers and installers that default to the docker runtime (digitalocean managed k8s uses docker as the runtime, and kubespray defaults to it as well but also supports CRI-O). With dockershim going away where else will docker be used other than developers’ desktops?

                        Personally I bit the bullet was basically forced to switch to podman/buildah due to docker straight up not supporting Fedora 32+ due to the kernel change to cgroups v2. Docker Desktop for Mac/Windows is a nice product for running containers on those OS’ but my guess is that is the only place it will stay relevant. It’s easy enough to have a docker-compatible cli aliased to docker that doesn’t require the daemon on linux etc.

                        Also, with their attempts at monetizing DockerHub it kind of paints a “failing” aura over the company. If they can’t make money off of DockerHub how can they monetize a daemon that runs containers when there are many other equivalent solutions?

                        1. 1

                          multiple k8s providers and installers that default to the docker runtime (digitalocean managed k8s uses docker as the runtime, and kubespray defaults to it as well but also supports CRI-O). With dockershim going away where else will docker be used other than developers’ desktops?

                          SImilarly microk8s, k3s are using containerd since forever.

                          With dockershim going away where else will docker be used other than developers’ desktops?

                          Yep, exactly. It will be used by end developers just the way it is right now. I understand there are more lightweight alternatives for building images (esp something which doesn’t require you to run a local daemon) that are more lucrative. But not everyone runs K8s and I think there’s a large market out there for people running standard installations of their software just with docker/docker-compose :)

                          1. 2

                            I think there’s a large market out there for people running standard installations of their software just with docker/docker-compose

                            This is extremely true. I have many friends/colleagues who use docker-compose every day and there is no replacement for it yet without running some monster of a container orchestration system (compared to compose at least).

                            I guess my main worry is that docker is a company based on products which they are having an extremely hard time monetizing (especially after they spun off their Docker EE division). I don’t see much of a future for Docker (the company) even if loads of developers use it on their desktops.

                            1. 2

                              docker compose was based on an acquihire of the folks that made fig.sh, then very little ever happened feature-wise. Super useful tool and if they’d been able to make it seamless with deployment (which is very hard it seems) the store might’ve been different.

                            1. 1

                              Yep, I appreciate that they finally made it available for Fedora 32 (after having to tweak kernel args), but many of us already switched to alternatives.

                              They still don’t ship repos for Fedora 33 (the current release). After checking the GitHub issue related to supporting Fedora 33 it appears the repo is now live, even though it only contains containerd.

                        2. 4

                          As someone who uses Docker but not k8s, it is kind of puzzling to see the degree to which they have merged into a single barely-differentiated blob in the collective mind of the industry. The article mentions this briefly at the end, but there are plenty of use cases for Docker that have nothing to do with k8s, and some of them aren’t currently well-served by alternatives (e.g., doing local development on Mac or Windows systems).

                          Whether those non-k8s use cases are enough to keep Docker Inc. afloat, I can’t say, though.