What has Docker become?

44 points by tuananh a day ago on lobsters | 33 comments

Corbin | a day ago

This is a good retrospective from a business perspective. I think that there's also something to be said about Free Software in there.

Docker’s strategic shifts tell a story of a company searching for its place in a market it helped create. The containerization technology Docker pioneered became so successful that it became infrastructure - something everyone uses but no one wants to pay for directly.

This is anachronistic and I think that correcting it will reveal something. dotCloud was incorporated in 2008. Linux cgroups were contributed to the kernel from maybe 2003 to 2006, including insights developed inside Google and Silicon Graphics. Google lore is that Google discovered containers about a decade before they were public; giving that minimal credence and shifting it by a couple years, Google may have started playing with containers as early as 1998. (Pushing the point, FreeBSD jails were introduced in 1999.) Docker didn't invent containerization.

From that perspective, Docker was always about monetizing already-free software and already-implemented concepts. Their sales were oriented around a moat defined entirely by their own proprietary APIs. Red Hat may have initially been willing to kickstart Docker in order to get Openshift off of their plate but by 2015 they had started the process of turning proprietary Docker into standardized OCI. In 2018, with the release of Podman, Docker no longer had any value-providing justification for existence and had become purely rent-seeking.

jzb | a day ago

Docker did not invent containerization -- the hosting company I worked for was using Virtuozzo (which sparked OpenVZ) in 2004 -- but it certainly streamlined the use of it to a point where it was a viable thing for just about anybody to use.

IMO the thing that really hurt Docker was their attempt to pivot from developer tools to infrastructure. They partnered with companies like Red Hat, which gave them legitimacy and reach in the enterprise market, and then turned around and started declaring the OS was dead and all would be containers... Docker was also not quite ready to make that leap, IMO they did not understand a lot of the problems faced by enterprise-scale deployments.

They really tanked things with the whole Moby debacle. I was at Dockercon or whatever when it was announced, and the mood in the room was not great. It felt like a big rug pull, before rugpulls were becoming a common occurrence.

weberc2 | 8 hours ago

Honestly Docker as developer tools was always a pretty clunky experience—it never remotely lived up to the promise of “develop in a completely prod-like environment”. It introduced all kinds of workflow and performance inefficiencies to make APIs portable despite that modern programming languages and package managers already solve 95% of those problems better than Docker does anyway—strict platform portability problems were rarely the issue with dev/prod portability in the first place; it was usually about dependencies on services running in prod.

It always felt weird that dev tools was the marketing angle in the first place, rather than as a container orchestrator. Maybe they thought they could make more money that way, but I (knowing nothing about business) would have figured they would stand to make much more money by building the best, most secure, most performant container runtime for enterprise users or building an awesome container orchestration platform or a Linux distribution for booting straight into a container runtime (so that everyone doesn’t have to reinvent the host management wheel and can configure hosts with the same conventions used in their container runtime and orchestration platforms) or any number of things that would presumably be very attractive to enterprise users.

[OP] tuananh | a day ago

very much agree. i think Solomon Hykes can agree with that last part about Red Hat too.

Don't partner with Red Hat. They are competitors even though they're not honest about it. https://news.ycombinator.com/item?id=46734451

koala | a day ago

Docker didn't invent containerization.

Docker did not invent containerization at all. But I believe they had a huge part in inventing "application containers". That is, having containers that run a single process and that don't pretend to be a full Linux system, separating clearly data using volumes, etc.

They also provided a very usable ecosystem for application containers. Layered/cached container image builds might have lots of flaws, but most people really like them.

I'm actually more or less against application containers, but Docker had the talent for bringing a complete new thing that people loved (application containers) and making them easy (not simple).

The problem is that these are not two things that will make a huge company. It can be easily replicated and commoditized. (Plus they were "true" open source, right?)

And on top of that, Docker volunteered to host for free terabytes and terabytes of public container images that consume terabytes and terabytes of bandwidth.

This is one of the best examples that having and executing a brilliant idea is completely different from making money.

Corbin | a day ago

I would argue that chroot() is the beginning of single-process containerzation. Wikipedia somewhat agrees; see their page on OS-level virtualization. In particular, FreeBSD jails predate Docker by nearly a decade, although I'll admit that they "don't pretend to be a full Linux system" in a different way!

Internally to Google, the Midas package manager was part of their containerization strategy; it's a package manager where every package is a container-ready filesystem. Midas was named to the public about a decade ago. I wasn't around for it, but if Google really does invent things about a decade before discussing them with the public then they developed Midas roughly around 2003-2006, right around when they were helping to add cgroups to Linux. This would suggest that Docker never had a technical moat; they were merely reinventing what Google had already done, an instance of GIFEE.

We should be very specific if we want to learn from Docker's business failures. Docker's first viable product was Docker Registry and their first viable service was Docker Hub. This gives them roughly the same shape as npm, another failure worth learning from. Like you say, Docker Hub is expensive to operate and they eventually had to monetize it harshly to stay afloat. Docker Registry is the commoditized complement for public clouds and they were only able to make money as long as there were no viable substitutes for the embraced-and-extended tools that they put onto the desktop, particularly on Mac and Windows. This is the true reason that e.g. nVidia puts so much money into embracing game studios, extending GL, and extinguishing alternatives to CUDA; their hardware isn't good enough to sell itself.

koala | a day ago

That is why I carefully said "they had a huge part" :)

Prior to Docker, I don't think anyone was really using application containers outside chrooted daemons for security. When Docker came out, application containers exploded.

(I wouldn't be surprised if Docker got the idea from a Google employee, or something like that.)

The only people making money with application containers are companies selling Kubernetes (or similar) services; either things like OpenShift licenses, or the margins in AWS EKS or whatever. Or those who reap the benefits of using application containers for themselves.

I don't think this is anything new. I think people making money with open sourced ideas is very rare, while it's relatively common to make money selling services, etc. around open source code. What is surprising is that people still try. (Although I believe most people trying to make money nowadays go with source-available licenses, etc.)

ZicZacBee | a day ago

hongminhee | a day ago

I think this is a fair critique, but maybe a bit harsh? Docker didn't invent containers, true, but they did innovate significantly on how to build and distribute them. Dockerfile and image layering made reproducible builds accessible in a way that prior art (Nix, configuration management tools, etc.) didn't quite achieve.

The commons enclosure attempt is real, and I'm not defending their business model. But reducing Docker's contribution to pure rent-seeking from 2018 onward overlooks that the developer experience they created genuinely changed how we think about packaging software. That innovation deserved some reward, even if their monetization strategy failed.

Maybe the tragedy is that they succeeded too well… Their approach became the standard so quickly that there was nothing left to build a moat around.

option | a day ago

...developer experience they created genuinely changed how we think about packaging software.

Yes and no.

95% of all containers I've ever deployed to production starts with an Alpine or Debian base image. Those are Linux distributions, and it is rare that I don't need at least one or two calls to apk add in my build pipeline.

The OCI image format is tarballs + metadata. It's a lowest-common-denominator approach that works for a broad array of use cases, but it's not particularly efficient, either on disk or on the wire. It's just good enough.

The identity of images depends on hashing json of all things, which is one of the crazier decisions I've seen. An errant bit of whitespace could break this. https://specs.opencontainers.org/image-spec/considerations/#json Nobody notices because nobody really thinks of OCI as an optimal format. It de-dupes stuff on docker pull well enough.

The whole push/pull business is convenient, but still bears the scars of "all image pulls are coming from docker.io by default".

Another bargain-basement choice Docker made for all of us: versions are raw strings. Most people do some sort of name:something-semver combination, but there is no reliable way to determine versions in the general case.

Don't get me wrong. The massive adoption of Docker speaks for itself. There is a kind of virtue in a crazy kludge. I actually remember watching the first PyCon demo at my desk at work. I was wow'd.

But there are many aspects of packaging that Docker punted on or solved in a non-optimal way.

samuelkarp | 18 hours ago

The identity of images depends on hashing json of all things, which is one of the crazier decisions I've seen. An errant bit of whitespace could break this. https://specs.opencontainers.org/image-spec/considerations/#json Nobody notices because nobody really thinks of OCI as an optimal format. It de-dupes stuff on docker pull well enough.

Layers are content-addressed based on the (gzipped) tar. Only the index, manifest, and config are json blobs. The de-duping of layers is what you're seeing that you're calling "well enough".

A better criticism of the layer content-address is that it is the compressed ordered content which is hashed. A change in tar ordering (extracting to disk and then re-tarring) or a change in compression settings (or even gzip implementation, like using pigz to compress) will cause otherwise identical content to be hashed differently and no longer properly de-duplicate.

peter-leonov | a day ago

Docker is so so much more than a restricted chroot. Sad to see their downfall.

weberc2 | 9 hours ago

The containerization primitives certainly existed in the Linux kernel, but assembling them into a container runtime, working out the image format, designing and implementing the Docker daemon APIs, building the container image registry, building the Docker for Desktop stuff for macOS and Windows, etc—none of this was “already free software”.

I disliked a lot of Docker’s design choices, but they clearly did real work and their product influenced the entire container ecosystem. I don’t have any special affinity for Docker Inc and I have a lot of criticisms for many of their design choices and implementations (and I’m glad we are in a post-Docker-Inc world), but it feels wrong to reduce it down to “they were monetizing already free software” simply because the kernel primitives existed. Assembling primitives into something useful was inherently valuable work.

mperham | a day ago

There's just something hilarious about the CEO change from Scott Johnson to Don Johnson, like Silicon Valley white guy MBA execs are just interchangeable Lego minifigs.

benjajaja | a day ago

The article says the former guy's surname was Johnston with a T. Still hilarious, or maybe even more so.

Aks | a day ago

I don't think you're far off.

nickmonad | a day ago

It's sad that Kubernetes is considered the thing that killed Swarm, or the ability of Docker Inc to monetize and grow the "orchestration" side of the equation. Some kind of middle ground between "ssh + systemd" and full-blown Kubernetes seems to be desired. I know there are options in that space now, but Docker was in a unique position to sell that idea early on.

jstoja | a day ago

To me they were also in the fire with Nomad, which advertised itself as simpler orchestrator.

atmosx | a day ago

I believe that swarm was abandoned prematurely. I recall maintaining a swarm cluster with minimal setup as a sole engineer for years. We had to be a team of five to run Kops on AWS.

doctor_eval | 11 hours ago

We used docker swarm in production on a small cluster. It was a nightmare, we would need to reboot workers on the regular because networking would hose itself so badly we couldn’t do anything.

The benefits of orchestration were clear - I went on to use Nomad and now k8s - but, uh, I don’t have fond memories of swarm.

Today I run a Vultr k8s cluster single handed, not sure why you’d need a team of 5!

ashishb | a day ago

The market might have been small.

On one side, you have docker compose and on the other side you have Kubernetes.

atmosx | 14 hours ago

IMO there was/is space for both tools. Swarm can be managed by a small dev team or a part time SRE. I have done the same thing with k8s but needs 2x or 3x more maintenance and we’re taking about the “managed” offerings of k8s vs self-managed Swarm. A part time engineer maintaining a self-managed k8s cluster is a tall order.

Relax | a day ago

I know there are options in that space now

Oooo like what?

nickmonad | a day ago

Nomad, Kamal, Uncloud, Dokku, Coolify, ... many others :)

squarism | a day ago

I've looked at most of those, but eventually bought Dokku Pro. At the same time and recently, Komodo seems the most interesting to me for personal use. It's a difficult space because of complexity/simplicity tradeoff and use cases. Coolify doesn't work in a team flow (yet). Dokku non pro would have to be installed over and over, single instance per team or group. And then how do you protect yourself from bad actors? Quotas?

I think a majority of projects could just use a Heroku which makes me think that Heroku was ahead of its time because we still don't have it again. But then, I never had visibility into the Heroku backend and what it took to run.

schneems | 20 hours ago

Hi. I work for Heroku. Check out CNBs as an alternative to dokku you can now use “pack” https://github.com/heroku/buildpacks/blob/main/docs/ruby/README.md.

Heroku backend and what it took to run.

It's like a ship of thesus, constantly being built and rebuilt. The latest, Fir, uses kubernetes and standard containers with CNB standards to build those containers. (Re)Wrote our own router from scratch in Go.

And everything wired up to everything else. That's really the secret sauce. It's a bunch of systems integration work. It's not "hard" but it's not easy either. A lot of both depth and breadth problems.

That and I get paged so others don't.

ashishb | a day ago

This is another of those examples where charging for a tool (docker desktop) is really hard but charging for compute (kubernetes) is easy.

peter-leonov | a day ago

Yep, they should have continued on the track of unifying infra and go after clouds. Swarm was such a sleek abstraction to build upon.

ashishb | a day ago

Unfortunately, they lost, and not on technical merit.

Once all hyperscalers agreed on Kubernetes, the alternatives had little chance of winning.

Kratacoa | 11 hours ago

Once all hyperscalers agreed on Kubernetes, the alternatives had little chance of winning.

As a VC-funded company that requires explosive growth, or in general?

francedot | 19 hours ago

Containers and sandboxed environments are getting commoditized fast. On macOS, there are now better native options like Apple's Containerization framework (https://opensource.apple.com/projects/containerization/) that run Linux containers without the Docker Desktop overhead. When the OS itself provides lightweight container primitives, the value proposition of a third-party tool shrinks considerably.

david_chisnall | 11 hours ago

Docker Dekstop uses Apple’s virtualisation framework to run a Linux VM, mount shared folders there, and then run Linux containers in them.

Apple’s container framework uses Apple’s virtualisation framework to run a Linux VM for each container. It then mounts shared folders in there and runs a single Linux container in each VM.

Both use the same underlying technologies, but Apple runs one container per VM. There are some benefits to Apple’s approach (easier resource accounting, better security) but I wouldn’t think lower overhead was one of them.

emmanueloga | a day ago

Docker ended up as essential infra without much of an identity, which makes me wonder how Dagger (Hykes’ current project) parallels or diverges from that. My rough model:

  • Docker’s real win was DX.
  • Dagger feels like Docker++: what Dockerfiles were straining toward, but expressed in real languages instead of another DSL.
  • They started with CI pain, and the recent AI direction seems to fall out of the same execution model. The container‑use work from the team is an example of how that foundation lends itself to sandbox‑style agent workflows.