Hi, I'm the creator of Docker and came across this post. It's always fun to see other engineers think about how you designed a system. I haven't been involved in developing Docker in many years, but it's a fun trip down memory lane.
Oh wow, hi! I think my biggest “gripe” with docker is that everyone grabs it as a hammer and views many problems as a nail. Am I way off base with that? What was your original vision for how docker might be used in the wild?
There are many. But there is an alternative universe where I remained CEO instead of transitioning to CTO, and continued pursuing my product vision obsessively instead of compromising on it because of lack of self-confidence and a desire to be loved. I think I would have avoided a lot of the mistakes that the company made - but on the other hand, I would have made a fresh set of mistakes instead, so who knows what the outcome would have been!
Thanks. I needed to hear this. So many things in life can turn out differently if we can overcome our “lack of self-confidence and a desire to be loved”. It’s never too late!
To give a sense of time: Docker launched in 2013, but it was one of many iterations of the same project, which I had been working on continuously since 2008. Over those 5 years, virtually nobody cared or understood what the hell I was working on. I lived in France and had no connection to the tech industry.
So, not only did I not anticipate this kind of success, I did not even have a reference frame for what success might even look like. All I knew was, I had an itch that I had to scratch, and I had nothing better to do :)
I believe Docker was always called Docker, but it grew directly out of the previous generation of the technology used by dotCloud (the company Solomon founded), which was using LXC with AUFS.
I think the breakthrough with Docker was bundling the AUFS-based image layer format with a centralized container image sharing registry. This was revolutionary at the time. But another huge element of success was Solomon and others evangelizing the technology relentlessly at meetups and conferences. I distinctly remember attending a demo in 2012 where the crowd was very excited to see the Docker workflow that we now take for granted, in large part because Solomon was so good at presenting it. At the time, I was working at a cloud PaaS company where we developed a similar technology on top of LXC and BTRFS (but no image registry). Docker really made me think hard about what we were doing, although I didn't end up making any moves.
> the previous generation of the technology used by dotCloud [...], which was using LXC with AUFS
One small note, the earliest versions (2007-2011) were based on vserver then openvz, which predated lxc. Back then you had to patch the kernel to get container primitives. And since virtualization also required patches (eg. Xen), we had to carefully combine both sets of patches, which was poorly supported because nobody else was crazy enough to stack 2 kinds of virtualization!
In those years in the systems group I was working in at a fortune 500, we were very interested in this type of technology. Mostly using Solaris Zones and cgroups and trying to do this ourselves with limited results.
When Docker came out it solved so many issues we were having. Thank you for scratching that itch.
I remember in 2013 the first time someone showed me Docker [im in Mexico]. My first thought is that we already had Vagrant for setting up and running virtual machines through console.
If you were implementing an operating system (free from compatibility concerns), what would you do differently to better support lightweight containers?
I'm curious what made think of it in the first place? What was your thought process that made you think that these Linux primitives give the opportunity to make it a platform for distributing and deploying packaged applications and eliminating a lot of the issue around "it works my machine" argument?
You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)
Its nice having the flexibility for both system containers and application containers. It does feel though that most people forget that nspawn does both, and that it only can do system containers.
Agreed, I briefly forgot it can do both. Thanks for the reminder.
I went down the rabbit hole of Incus and once I realized it requires the full network aspects to be configured, then I figured using KVM, QEMU, Libvirt was more practical. Then stumbled into nspawn and it was basically ready to go.
Although, I haven’t done enough research to know if Incus provides certain useful capabilities.
I'm a huge fan of Incus. VMs, LXC, and OCI containers all under the same Docker-like CLI interface. I run it on a local server and on a dedicated server in a local DC, and connect to both remotely using Nebula.
It includes project-level resource separation, advanced networking, clustering, snapshots, and more. Plus, Stéphane Graber is super responsive on the forums and issue tracker. I bounced off of Proxmox, and can't recommend Incus more for hypervisor-type scenarios.
Do you have any experience with multi-node operation with Incus? I like the simplicity of Proxmox for moving VMs and LXCs around but I haven't settled on a solution for OCI. K3S is an option but it's a big jump. I'm moving towards Fedora CoreOS VMs at the moment for live migration and security isolation.
I run Incus, but I had to uncluster my home lab to allow to have zfs duplicated containers (as cold standby) on another physical instance. It may have changed now, but it was not possible to have two near identical container, one up one down, on the same cluster.
The LXC way of getting an up per container is pretty great - I’m basic bitch with this sort of thing so I assume it’s less of an issue for the experts but really enjoyed the IP per container when I hit it.
This is sometimes easier, but once you have a docker-compose of what you want, it’s hard to beat. You can have running server with all the services and network shares all sorted in minutes.
Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.
The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".
As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.
“Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough”
Developers are a tough market to please, especially when your business revolves around open source software.
Unless you have an option to do private cloud hosting (like all these databases and other platforms do), it’s incredibly difficult to monetize.
In the case of docker, I think it’s in the nature of the product that it’s difficult to monetize. Best you can do is enterprise support contracts, but those aren’t good enough to justify all that VC funding.
I don't think it's necessarily that difficult to monetize, they just ignored a lot of the needs of enterprise users (rootless, not punching holes in the firewall, not fighting with systemd, private registries) for long enough that other projects (e.g. Podman, Quay) filled the void.
None of those things were obstacles to commercialization. None of them were common requests from enterprise customers except private registries, which Docker did build.
Podman has never been commercialized.
Docker's difficulties in commercializing were self-inflicted. Enterprise and open source teams were culturally siloed and as a result the enterprise products were not "native". There was a lack of focus and a "checkbox" approach to building product. There was no singular "dictator" who could enforce a coherent product design and vision, and it showed. etc.
I was a DotCloud user before Docker, and I remember keenly thinking - “oh, is that how they went from hosting only PHP to any language/stack?”
I always assumed it was the same internal tech, now productized. Then of course, it became THE tech, overtaking vagrant and anything else for local development and deployment. The latter was the true money shot. Server runs exactly what you test with locally.
BTW just for giggles opened hub.docker.com with Network Tools opened:
368 requests, 9.62MB, Finish: 4.50s.
For a thing which should be a static cached copy with everything bundled. Sure, it's a progress from the previous iterations when it could hang for 30 seconds before actually loading... but this is still ridiculous.
>>[0] 'Bridge' was and still is an established network term for joining two broadcast domains into one. Why the hell you decided to name your NAT'ed network layer a 'bridge'?
Like they told you 9 months ago, the terminology is exactly right. They're called "bridge" networks because they use Linux `bridge` devices to bridge `tap` adapters of all containers attached to the same Docker network.
There's nothing exotic about this terminology at all and complaining about it betrays your own lack of knowledge.
> PS there is still (AFAIK) no support for a transaction-like image building in Compose. So instead of something like this:
Do you mean Dockerfiles? RUN command has had heredoc support for 5 years now.
> Like they told you 9 months ago, the terminology is exactly right
Oh, well. I guess you ride to your work on wheels, right? Because while you sit in the car/bus/train it's the wheels what actually rides on the road/track, right?
And if you bothered to look at that comment then why do you ignore what 'macvlan' - which is an actual bridge both in terms of a Linux device and the functionality - is not called 'bridge'?
Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
You see, if they just called it as it should, ig:
bridge => hostnatnetwork // or internalbridgenatnetwork or whatever else
macvlan => bridge
then they would be both named correctly and be the technically correct - and 99% of Docker users couldn't tell a difference - because they are not net or sys administrators.
But the last 1% is the people who actually makes the workspaces for these developers where they actually do run their fancy-shmancy containers. And for that 1% it surely pays naming this as they are.
And there is also that dumb decision to silently rewrite 127.0.0.1 to Google DNS without a question. Especially when 98% of Docker users are incapable to diagnose it by themselves.
So it's not only the naming of a network functions.
But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
> RUN command has had heredoc support for 5 years now
So ten years late? Adoption is off the limits, as you can see - that snippet is from the official Docker image.
And if I need to run multiple commands and not just one RUN statement? Like the people are asking for years? And you so knowledgeable person of course do know the reason for that, right?
> And if you bothered to look at that comment then why do you ignore what 'macvlan' - which is an actual bridge both in terms of a Linux device and the functionality - is not called 'bridge'?
Because renaming things that map 1:1 onto the underlying concepts is a terrible idea that makes learning and communication harder by introducing ambiguity and indirection. "bridge" now means multiple things and not only is it needlessly obfuscated, it's wrong (explained later). I _hate_ projects that do this.
> Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
Rhetorical question, maybe it'll help you find a source of misunderstanding: Why do you expect it to join the broadcast domains of your containers and your LAN? And which LAN, exactly?
> No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN. Therefore you incorrectly expect and demand that term "bridge" accurately describe the logical connection between your container and your LAN.
Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
A Docker 'bridge' network is not necessarily routed to LAN either so your suggestion of calling them "hostnatnetwork" is wrong. Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
> But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
No I didn't say that, I think you suffer from the curse of having a lot of previous experience in an adjacent area where you built a strong mental model of how things ought to work, but now you're trying to forcefully apply this same mental model onto Docker.
All of this networking terminology is consistent and applied correctly but for some reason it's not clicking with your previous knowledge and you're trying to fit a square peg through a round hole. Take a step back, investigate.
> So ten years late?
Don't shift the goalposts. You claimed it still doesn't exist.
> No I didn't say that, I think you suffer from the curse of having a lot of previous experience in an adjacent area where you built a strong mental model of how things ought to work, but now you're trying to forcefully apply this same mental model onto Docker.
I think you just proved his point. Docker came onto the scene and hit roadblocks in enterprises where sysadmins lived. These architectural and terminology choices ran head on into well established domains. Yes, docker did change things, but I think his point is it would’ve had more success with paying customers if the authors of docker had a better understanding of established network administration and system administration concepts from the start (though these were the early devops days…). That lack of understanding caused adoption friction in enterprise space, and different choices could’ve prevented that without changing at all the paradigm of what docker is.
The docker mental model isn’t crazy different. Design choices made it unnecessarily less compatible with existing mental models. This probably contributed to the feeling of it being revolutionary, but impeded its adoption with paying customers.
In my experience adoption was primarily driven by developers and sacrificing clarity would hurt adoption. This perfect familiarity was never achievable anyhow. Docker was a completely new paradigm in software development, similar to how modern cars aren't just horse carriages without the horse. It transformed everything from development, testing, deployment, updates.
Experienced sysadmins often tried to approach it with "VMs but with less overhead" mentality, which couldn't possibly work. Coworkers would ask me how they can install SSH daemons in containers to update packages and view logs, or they would refuse to adopt Dockerfiles, sometimes setting up the container manually and exporting the image. They would wonder how to install many different services in 1 container and handle crashes, etc. It was a huge shift.
> Design choices made it unnecessarily less compatible with existing mental models.
Again, I disagree. Whose mental model do you prioritize? How invested are you in making Docker work just like whatever elaborate system those enterprise admins used before?
Let's not forget that by creating new abstractions, you're penalizing users who already possess foundational knowledge of the primitives. If you do this then I'm probably going to get a whiff that you care more about enterprise sales than developer experience (duh) which is a huge red flag for me.
> Because renaming things that map 1:1 onto the underlying concepts is a terrible idea that makes
Sorry, but you do insist on naming 'docker' bridge a bridge because it's underlying tech is a bridge. Make up your mind.
> A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
And noone absolutely ever cared about what it's a bridge down there. It could be using gronklew and maybe 20 people on the planet would care to learn what it is.
You are quite bent on it, but nobody cares how the containers talk between each other.
> Why do you expect it to join the broadcast domains of your containers and your LAN?
I expect a setting named 'bridge' in some $thing to actually implement a bridge.
Just like you pressing an AC button in your car expect a very specific thing to turn on/off and not 'Artifically flavoured windshield wipers served in a Container'.
Why do I want it is on me. Maybe I need it. Maybe I control my network and freely give out addresses to anything. Maybe I don't run on some cheap VPS somewere with only 1 IP I can't choose nor change. Bazillion of options. None matters because this is not about 'why do you want it'.
> And which LAN, exactly?
The path of a least automagik.
Only one ethernet interface? Then connect to that ethernet interface. Bitch in the logs what automagik was used and this can lead to the problems.
Many interfaces? Just shutdown (least pain for newbies) or start without attaching (harder to troubleshoot for newbies). Bitch in the logs in either case.
> Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN
'Your misunderstanding seems to stem what there are some people assume what they are driving in like c-a-a-r. When even 5y.o. knows it's the wheels what drives you around.'
> Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
Yes, exactly. Don't call it 'bridge' ~20 years ago and we don't have this talk here.
> Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
It's not even a property of a bridge network. For someone claming knowledge superiority this is a quite lame statement.
> links
Really.
> previous experience in an adjacent area where
Ah, yes, irrelevant 'adjacent area' of computer networks. I'm out of words.[0]
> All of this networking terminology is consistent and applied correctly
No. Go drive your wheels home.
> Don't shift the goalposts. You claimed it still doesn't exist.
I'm just mentioned RUN as an example. You decided what multiline is the goalpost.
And yes, if even the official Docker images doesn't use RUN heredocs then... there is something wrong with them. Like being implemented too late.
[0] but perfectly explains your previous 'flawed assumption'
> And noone absolutely ever cared about what it's a bridge down there.
Just like nobody ever cared that the container isn't bridged to their LAN in the first place - in fact the isolation is a feature.
But you should care, it's only the most common building block of network isolation for containers/VMs on Linux.
---
I'm not going to get into it further. I tried to explain nicely but you seem hell bent on complaining and not learning anything new. At this point the appropriate response is: RTFM, it's all in there, their docs are great.
If you don't have a clue about how things work under the hood and if you have no interest in learning anything, then stop complaining about imaginary technical inaccuracies.
I think there are some interesting points there, but your rude, arrogant, and aggrieved tone overwhelms them.
If you decide to edit your tone, I'm happy to discuss more here re: filesystem transactionality tradeoffs, since that's something I've struggled with in both directions WRT containers.
I'm only responding in kind to GP - no less but not more.
I think you are the one who built an alternate layering system? On mobile, don't like to search rn.
If yes the basics are in the first comment- just not limited to the one RUN statement ofc. And if I do remember how the layers are made then making it transaction-like would be a great dedup saver eg for adding a static rarely changed artifacts - there could be essentially a ~one layer for how many you want images.
Implementation wouldn't be so difficult - you just stop committing until TRAN END; but the main is too provide a mental model for a not very experiencied dev how this supports his work and makes it more efficient on the storage and execution.
>I'm only responding in kind to GP - no less but not more
bullshit, you've been cunty right out of the gate since your first reply. I'm just surprised anyone else in the thread had as much patience as they've shown here.
you seem exactly like the type of person who is incapable of realizing they're a complete asshole and wonders why people choose to avoid them.
> BTW just for giggles opened hub.docker.com with Network Tools opened: 368 requests, 9.62MB, Finish: 4.50s.
lol What on god's green earth does this have to do with commercializing Docker? Do you really think better lighthouse scores on hub.docker.com was what was holding them back?
Did you miss 'up to 30 secs' in the previous comment? It's not about lighthouse scores. It's the whole feeling of a hodge-podged project which is run by people who are - maybe - passionate but ultimately lacking in the basics.
The ultimate buyers of their product was the system operators and the system owners - not the developers per se. They failed hard everything that space. Having a dog slow the main content site didn't added anything positive.
> As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.
yes, and no. Probably both. Docker would not become such a default standard if it wasnt for all the open sourcing.
Oh gosh, the choice of language is such a context-specific design decision. The first versions of what became Docker were written in Python. The choice of Go was a mix of nerdy excitement ("it's like the best of Python and C!"), and adoption strategy: back then devops tools were segmented by language runtime: ruby, python or Java mostly. Shipping a compiled binary allowed us to be adopted by all tribes.
None of these things can be transposed today, so I guess I can't answer...
For what it's worth, my current project (dagger) is written in Go.
The article's point that Go is a bad fit for some of the low-level interactions with the kernel, is absolutely correct though. We had to do a lot of juggling to get it to work properly. But in the end it was fine. I would say it was the correct choice of language at the time, and it certainly contributed to Docker's early success.
Another explicit goal was to facilitate contribution. We thought Go syntax was "vanilla" enough to attract a broader set of contributors than a more niche, opinionated language like Erlang, Haskell, Clojure etc. I think that also worked out well - Docker was known for attracting many contributors, many of them making their very first open source contributions. We even ran special meetups to help people become contributors, that was fun.
I think that's so true and it was one of the keys to success of the project. I contributed the very first simplistic implementation of referencing linked containers by name. I never would have bothered if the project was written in C or C++ (I'm more than capable, I just don't enjoy torturing myself). Go made it MUCH easier for a fly-by-night contributor such as myself to join the party.
Do you think e.g. students learning docker syntax at a uni is beneficial in 2026 or you think LLM-assisted programming means that Docker will become an abstraction under an abstraction and its setup will be mostly automated by LLMs? I find myself very rarely debugging or handwriting dockerfiles nowadays!
This is a strange question because you could learn Dockerfile syntax in a single night. There are only 18 directives, and the average Dockerfile uses like 6 of them. 90%+ of Dockerfiles is just "run this shell command" and "copy this file into the docker image". The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.
> The only time it flirts with complexity is multi-stage Dockerfiles, but 90%+ of Dockerfiles are a single stage.
And multi stage builds are just a directed graph short hand for having separate intermediate dockerfiles. There’s nothing special about them either.
If I had to pick one thing that is tricky for someone new to docker to understand, it’s how the filesystem state is cached using the previous state, the env, and the raw command text.
Indeed, the one mistake I keep seeing new people make when writing Dockerfiles is writing them in a way that doesn't optimize for caching. For example, they'll copy their code into the container first and then start installing packages and dependencies. It's an easy mistake to make because the end result is still a functional Dockerfile, they're just wasting time and stressing the poor package servers unnecessarily.
I'm gonna be real with you. No one needs to know docker syntax. Most engineers just copy and paste chunks of Dockerfiles around anyways.
Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.
Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.
LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.
edit: this isn't to say the syntax is good or bad (though there are several questionable parts of it) but extremely simple, and concise enough that you can just look up literally any command with great ease. It's surely a large part of its success, for better or for worse.
I think the key thing for a university student is to get the big ideas down: what is a container exactly, how does it work, how does it differ from a VM, what assumptions does it make about what you're running (and how do you run something that doesn't fit that pattern), why do images exist in layers and how are layers implemented, what are the performance benefits/tradeoffs, etc. etc.
When I was learning Docker I can't tell you how many people would describe it to me by analogy to a standardized shipping container, and start going off about shipping container dimensions and supersized cargo ships and trucks and railcars... and finish by smiling beatifically, like they had just explained the whole thing in a masterstroke. Let's... uh... not do that. :-D
In a problem set, the student might learn a bit about dockerfiles or docker-compose.yml... but only, I think, so they can try building and running a container of their own!
If you're going to fully delegate something so trivial as Dockerfiles to slopmachines, why go to university at all? You don't need to learn anything if you delegate everything to the slopmachine.
I'm a big fan of docker and mainly just want to say thanks for bringing it about.
I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...
After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.
It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.
I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)
Yes, I still use Docker. Although my way of using it has changed over time, like many others I assume.
My new project, Dagger, although it doesn't directly compete with Docker, does implement its own container runtime and orchestrator (for CI pipelines), and we recently added compatibility with Dockerfiles and Docker Compose files... So increasingly I find myself using Dagger instead of Docker, in the context of my CI pipelines.
On the other hand, I use docker compose on almost every personal project - like a minecraft server for my kids, home automation, AI hacks, etc. These days I just let my AI agent set it up for me, though :)
Yeah I've really been enjoying using AI to write docker/compose config and related ad hoc scripts. It was all pretty understandable anyway, but now you can just set up whatever you can think of in literally minutes, even if you just want a particular configuration as a one off. It's a great era for devops!
> I use docker compose on almost every personal project
Even if you don't need to? Even if there's no benefit?
I was handed a vibe coded project recently, a radio station playout system. For some reason, it's spread across 8 different Docker containers. One container has just Icecast in it, for example.
It's a private system that runs a single station on a dedicated server that we have full control over, but because of all the containers, it's harder to do common tasks like check a log or edit a config file.
OTOH it's a godsend for setting up a local dev machine or something like that, where you need to run different vendor provided packages. but I'm tired of all the mindless over-engineering (which unfortunately the LLMs trained on).
I did make a bunch of money with Docker, something I am eternally grateful for.
I did not start a band, but I did build a nice house, help my family etc.
I still program everyday, working on my second startup, https://dagger.io (please forgive the plug). We have a very active discord if you want to come say hi.
I work heavily with CI/CD. From a quick glance Dagger looks very promising and I might experiment with it this weekend.
I guess my follow up question is once you’re financially independent, what continues to motivate you?
Is it fun?
Do you feel like you’re creating tools that the industry needs.
I imagine if I ever get rich I’d work on music and small games.
Video games that you buy once and get to play forever, self host, etc. With a full open source commitment within 5 years or so. I’d probably create a game studio in a lower cost of living country though.
If you do try Dagger, please join our Discord and tell us how it goes! There's a very friendly and active community of CI/CD nerds, who will be happy to help.
We actually use the same Discord server for community and private team communications. So we are literally there all the time.
Shortly before this (the weekend after you secured YC funding, IIRC), I['m almost certain I] met you in downtown Oakland at Saagar & Tiffany's /r/bayarea meetup [0] – won't go into further details, other than I was one of their co-conspirators =P
My point of writing this is dualfold:
1) Thank you for sharing your design – I literally just installed my first Docker module (last week) and am floored at all the problems Docker might help me solve (I am not employed in tech)
2) My recollections of you, this decade+ later, was of a humble young man (we're the same age) that didn't know what else to do with his newfound techfame (and money) than keep'on keepin'on.
After watching your Docker Debut video, I'm fairly certain it WAS you: we chatted about dirtbikes (and just what the hell you were going to do now that you "had money" – then such a foreign concept to us both)... if this wasn't you it still seems to echo in your mannerisms/tact/approach. May your humbleness and cleverness prevail!
[0] It might have been at Gustafson's NYE party (I was the guy remorselessly conjuringbitches; "Volcano"); we had a mutual friend at MalwareBytes?
I’d love to hear more about some limitations/tradeoffs of docker that you made at the time that you regret in hindsight. Was there anything that you wish didn’t get baked into the “default norms” for containers, or areas that would benefit from more attention and redesigns?
I’ve been following dagger and it’s been really interesting to see the evolution of the design, and new set of abstractions. There’s also nix which has seen popularity, with their approach of building layered oci containers.
A while back you had this tweet which often gets cited, or at least _i_ personally think about it a lot.
> If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task!
Thanks for your innovation and helping developers! If Docker hasn't been your focus in recent years, what have you mainly been working on and what's been your favorite?
Were you around for the big Moby open sourcing? What went down from your perspective? If I recall, it was a fairly dramatic move that was not driven by the interests of Docker corp.
Yes, I was around for that. It was one of my skunkworks projects, as I had lost formal authority over engineering and product by then, and mostly used my "founder aura" to convince small pockets of people within the company to build cool things with me.
However Moby was a poor choice for a skunkworks project, it involved mostly communications and project management, which requires consistency and scale. Overall it was poorly executed because I failed to build enough support internally ("Solomon's at it again, let him do his thing"). Here I am trying to explain it on Twitter after the keynote, using a picture of a hand-drawn diagram, because nobody in marketing or product was assigned to help. https://x.com/solomonstre/status/855918630915133440
I think the goal of Moby was fundamentally correct: we were trying to replicate the Red Hat model, with well-scoped upstream components (linux -> runc, containerd..) integrated into a commercial product (Docker -> RHEL), with a faster-moving community project in between (Fedora -> Moby).
This was a way for us to clarify that Docker was a product, with a community of users, and Moby was an open source project, with a community of contributors. This clarity was very much needed, because different communities were projecting very different expectations on the brand "Docker", and these expectations conflicted with each other.
In a way, Moby aspired to be what CNCF ended up becoming - the "big tent" where the broader community meets to hammer out the general shape of the container stack. It seems obvious now that a single company can't be the sole authority over such a vast scope. But in the first years of Docker, we really did have that level of influence, because we literally invented the market, and brought the community together in meetups and an ever-growing Dockercon. It's only when Google and Red Hat pushed Kubernetes and CNCF, that our influence over the stack started to wane. Maybe it was for the better, I don't know if we would have been up to the task as designers and engineers. But at least there would have been a consistent design, with a Linus Torvalds figure willing and able to say "no" to stupid and redundant features. For a while I played that role - but then I was replaced by SIGs and committees.
I feel grateful for having seen from the inside one of those rare "hypergrowth" moments, it's a terrifying and exhausting experience but I would do it all over again without hesitation.
I also feel lucky that people are still using my work, that it endures without me, and that it generates (mostly) positive reactions. There was a time where Docker had a lot of haters, which was hard on me. But with time I've noticed that the negativity gets mostly washed away, and what remains is positive interactions like the ones in this whole thread. So now Docker is this cool experience I share with lots of people around the world, and when we meet we get to talk about this shared interest. I love it.
What made you decide to use a URI as the namespace for images? I like it, but I think it's been very misunderstood which has caused people to dislike it.
I was inspired by Go's use of URI as a namespace for user packages, and short names for the standard library. I thought it was an elegant way to allow everyone to host anywhere, and dodge a lot of complicated namespacing issues.
One design principle that was very important to me, was portability and avoiding fragmentation. I wanted every image name to have the same meaning in every engine, so that when writing a Dockerfile, you could have confidence that it would actually behave the same on every docker installation. In this we disagreed with other platforms (like Red Hat) who sought to "embrace and extend" Docker by changing the meaning of short names. We always fought this to avoid fragmentation of the platform. This did not make sense from Red Hat's perspective, since they were the platform - who cared about fragmenting Docker? That led to some negativity which is probably what you're remembering.
At the same time early Docker users wanted the ability to host their own images without being forced to use our central registry. Using Go's system solved both problems at once.
My only regret is that so many users had to run local docker registries on each node of their cluster, then write `FROM localhost:1234/myimage` in all their Dockerfiles - all to avoid unnecessary re-fetching of remote images. My goal was to make image distribution a native feature of the cluster, so that you wouldn't have to choose between portable Dockerfiles and proper image caching. But Kubernetes ended up dominating the cluster layer, and as far as design goes, we were not invited to that table. So as far as I know, the problem was never solved, at least universally for all Docker users.
Thank you for taking the time to reply. Yes, it was the short names and the way people were trying to run local mirrors that I remember.
I personally never use the short names and have really come to like your solution. I host my own registry and I can glance at a build or compose file and immediately know if I’m depending on any external resources that I shouldn’t be.
I originally disliked it, but it’s been my favorite namespace solution for a long time now.
Thank you for inventing Docker, it truly changed the life of many developers.
I was there when dotCloud announced Docker in 2013 and it was truly pragmatic. I had previously burned my hands on LXC, Juju Charms and Vagrant to containerize the apps, but it was all a house of cards. I remember helping a friend ship his whole Vagrant Box to the interview panel for a job!
It’s incredible to think about how much boilerplate Docker removed right at it's launch. Thank you.
I remember going to a demo of Docker in South Carolina before it hit 1.0. Came back and wrote the “Docker is the Heroku Killer” article on my blog and it was the first thing I wrote that made the front page of HN.
They aren't at all the same thing, but at a high level they both reduce the amount of fragile/tedious setup-specific futzing you need to do to get a few basic services up.
I'm happy to answer any questions you may have.
So, not only did I not anticipate this kind of success, I did not even have a reference frame for what success might even look like. All I knew was, I had an itch that I had to scratch, and I had nothing better to do :)
I think the breakthrough with Docker was bundling the AUFS-based image layer format with a centralized container image sharing registry. This was revolutionary at the time. But another huge element of success was Solomon and others evangelizing the technology relentlessly at meetups and conferences. I distinctly remember attending a demo in 2012 where the crowd was very excited to see the Docker workflow that we now take for granted, in large part because Solomon was so good at presenting it. At the time, I was working at a cloud PaaS company where we developed a similar technology on top of LXC and BTRFS (but no image registry). Docker really made me think hard about what we were doing, although I didn't end up making any moves.
https://jpetazzo.github.io/2017/02/24/from-dotcloud-to-docke...
> the previous generation of the technology used by dotCloud [...], which was using LXC with AUFS
One small note, the earliest versions (2007-2011) were based on vserver then openvz, which predated lxc. Back then you had to patch the kernel to get container primitives. And since virtualization also required patches (eg. Xen), we had to carefully combine both sets of patches, which was poorly supported because nobody else was crazy enough to stack 2 kinds of virtualization!
https://news.ycombinator.com/item?id=2032821
Then it pivoted to docker:
https://news.ycombinator.com/item?id=6634002
When Docker came out it solved so many issues we were having. Thank you for scratching that itch.
I didnt get it at that time haha.
You may have already answered this somewhere, so feel free to tell me what to search for or if you have a link to hand, that works too :)
I went down the rabbit hole of Incus and once I realized it requires the full network aspects to be configured, then I figured using KVM, QEMU, Libvirt was more practical. Then stumbled into nspawn and it was basically ready to go.
Although, I haven’t done enough research to know if Incus provides certain useful capabilities.
It includes project-level resource separation, advanced networking, clustering, snapshots, and more. Plus, Stéphane Graber is super responsive on the forums and issue tracker. I bounced off of Proxmox, and can't recommend Incus more for hypervisor-type scenarios.
[1] https://wiki.archlinux.org/title/Systemd-nspawn
Docker in its earlier incarnations was based on openvz, which predates lxc. The lxc project was initially a kernel development project: a collection of kernel patches to add container support. The userland tool was just a companion. When it became apparent that these patches would "win" the race to be upstreamed even though openvz was much more mature, we switched our tooling to lxc. The patches matured over time, but the userland tool (what we now know as "lxc") was finicky.
The original open-source release of Docker wrapped lxc for low-level container operations. But it was just too unreliable and we swapped it out for an in-house Go implementation called "libcontainer".
As for "commercializing": Docker was famously bad at commercializing anything, we open sourced so much (maybe too much) and still people complained that we were not open enough.
Developers are a tough market to please, especially when your business revolves around open source software.
Unless you have an option to do private cloud hosting (like all these databases and other platforms do), it’s incredibly difficult to monetize.
In the case of docker, I think it’s in the nature of the product that it’s difficult to monetize. Best you can do is enterprise support contracts, but those aren’t good enough to justify all that VC funding.
Podman has never been commercialized.
Docker's difficulties in commercializing were self-inflicted. Enterprise and open source teams were culturally siloed and as a result the enterprise products were not "native". There was a lack of focus and a "checkbox" approach to building product. There was no singular "dictator" who could enforce a coherent product design and vision, and it showed. etc.
I always assumed it was the same internal tech, now productized. Then of course, it became THE tech, overtaking vagrant and anything else for local development and deployment. The latter was the true money shot. Server runs exactly what you test with locally.
That's because there was never a proper sysadmin on your team to slap some sense in your decisions.
You've got too opinionated[0], too tied on yourself[1] monster which required too much resources what you can handle alone.
And your market is not the developers but the companies where whose developers have a daytime jobs: https://news.ycombinator.com/item?id=34377674
BTW just for giggles opened hub.docker.com with Network Tools opened:
368 requests, 9.62MB, Finish: 4.50s.
For a thing which should be a static cached copy with everything bundled. Sure, it's a progress from the previous iterations when it could hang for 30 seconds before actually loading... but this is still ridiculous.
[0] https://news.ycombinator.com/item?id=46739153
[1] Eg: the syntax to point the image to a specific registry is something... bad. So bad I just abandoned running my own registry, it wasn't worth it.
PS there is still (AFAIK) no support for a transaction-like image building in Compose. So instead of something like this:
people need to write these one-liner abominations: EDIT: to add on [1]: https://news.ycombinator.com/item?id=35312087>>[0] 'Bridge' was and still is an established network term for joining two broadcast domains into one. Why the hell you decided to name your NAT'ed network layer a 'bridge'?
Like they told you 9 months ago, the terminology is exactly right. They're called "bridge" networks because they use Linux `bridge` devices to bridge `tap` adapters of all containers attached to the same Docker network.
There's nothing exotic about this terminology at all and complaining about it betrays your own lack of knowledge.
> PS there is still (AFAIK) no support for a transaction-like image building in Compose. So instead of something like this:
Do you mean Dockerfiles? RUN command has had heredoc support for 5 years now.
https://docs.docker.com/reference/dockerfile/#run
Oh, well. I guess you ride to your work on wheels, right? Because while you sit in the car/bus/train it's the wheels what actually rides on the road/track, right?
And if you bothered to look at that comment then why do you ignore what 'macvlan' - which is an actual bridge both in terms of a Linux device and the functionality - is not called 'bridge'?
Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
You see, if they just called it as it should, ig:
then they would be both named correctly and be the technically correct - and 99% of Docker users couldn't tell a difference - because they are not net or sys administrators.But the last 1% is the people who actually makes the workspaces for these developers where they actually do run their fancy-shmancy containers. And for that 1% it surely pays naming this as they are.
And there is also that dumb decision to silently rewrite 127.0.0.1 to Google DNS without a question. Especially when 98% of Docker users are incapable to diagnose it by themselves.
So it's not only the naming of a network functions.
But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
> RUN command has had heredoc support for 5 years now
So ten years late? Adoption is off the limits, as you can see - that snippet is from the official Docker image.
And if I need to run multiple commands and not just one RUN statement? Like the people are asking for years? And you so knowledgeable person of course do know the reason for that, right?
Because renaming things that map 1:1 onto the underlying concepts is a terrible idea that makes learning and communication harder by introducing ambiguity and indirection. "bridge" now means multiple things and not only is it needlessly obfuscated, it's wrong (explained later). I _hate_ projects that do this.
> Because you don't understand the problem. More so, you ignore the actual question there about the broadcast domains.
A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
Rhetorical question, maybe it'll help you find a source of misunderstanding: Why do you expect it to join the broadcast domains of your containers and your LAN? And which LAN, exactly?
> No, if even if you use a bridge internally when your container is on a isolated network which is invisible to anyone on your network and the container access the network behind your host through the NAT - this can't be called a bridge.
Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN. Therefore you incorrectly expect and demand that term "bridge" accurately describe the logical connection between your container and your LAN.
Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
A Docker 'bridge' network is not necessarily routed to LAN either so your suggestion of calling them "hostnatnetwork" is wrong. Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
Disabling NAT for a specific brige network (com.docker.network.bridge.enable_ip_masquerade): https://docs.docker.com/engine/network/drivers/bridge/
Disabling auto-configuration of all firewall rules: https://docs.docker.com/engine/network/packet-filtering-fire...
> But hey, I'm just knowledge lacking guy without experience at all, just yesterday saw a computer a first time in my life, sure, sure.
No I didn't say that, I think you suffer from the curse of having a lot of previous experience in an adjacent area where you built a strong mental model of how things ought to work, but now you're trying to forcefully apply this same mental model onto Docker.
All of this networking terminology is consistent and applied correctly but for some reason it's not clicking with your previous knowledge and you're trying to fit a square peg through a round hole. Take a step back, investigate.
> So ten years late?
Don't shift the goalposts. You claimed it still doesn't exist.
The docker mental model isn’t crazy different. Design choices made it unnecessarily less compatible with existing mental models. This probably contributed to the feeling of it being revolutionary, but impeded its adoption with paying customers.
In my experience adoption was primarily driven by developers and sacrificing clarity would hurt adoption. This perfect familiarity was never achievable anyhow. Docker was a completely new paradigm in software development, similar to how modern cars aren't just horse carriages without the horse. It transformed everything from development, testing, deployment, updates.
Experienced sysadmins often tried to approach it with "VMs but with less overhead" mentality, which couldn't possibly work. Coworkers would ask me how they can install SSH daemons in containers to update packages and view logs, or they would refuse to adopt Dockerfiles, sometimes setting up the container manually and exporting the image. They would wonder how to install many different services in 1 container and handle crashes, etc. It was a huge shift.
> Design choices made it unnecessarily less compatible with existing mental models.
Again, I disagree. Whose mental model do you prioritize? How invested are you in making Docker work just like whatever elaborate system those enterprise admins used before?
Let's not forget that by creating new abstractions, you're penalizing users who already possess foundational knowledge of the primitives. If you do this then I'm probably going to get a whiff that you care more about enterprise sales than developer experience (duh) which is a huge red flag for me.
Sorry, but you do insist on naming 'docker' bridge a bridge because it's underlying tech is a bridge. Make up your mind.
> A Docker 'bridge' network driver creates a broadcast domain between containers attached to that network.
And noone absolutely ever cared about what it's a bridge down there. It could be using gronklew and maybe 20 people on the planet would care to learn what it is.
You are quite bent on it, but nobody cares how the containers talk between each other.
> Why do you expect it to join the broadcast domains of your containers and your LAN?
I expect a setting named 'bridge' in some $thing to actually implement a bridge.
Just like you pressing an AC button in your car expect a very specific thing to turn on/off and not 'Artifically flavoured windshield wipers served in a Container'.
Why do I want it is on me. Maybe I need it. Maybe I control my network and freely give out addresses to anything. Maybe I don't run on some cheap VPS somewere with only 1 IP I can't choose nor change. Bazillion of options. None matters because this is not about 'why do you want it'.
> And which LAN, exactly?
The path of a least automagik.
Only one ethernet interface? Then connect to that ethernet interface. Bitch in the logs what automagik was used and this can lead to the problems.
Many interfaces? Just shutdown (least pain for newbies) or start without attaching (harder to troubleshoot for newbies). Bitch in the logs in either case.
> Your misunderstanding seems to stem from the flawed assumption/expectation that every container is automatically attached to your LAN
'Your misunderstanding seems to stem what there are some people assume what they are driving in like c-a-a-r. When even 5y.o. knows it's the wheels what drives you around.'
> Docker networks are entities independent of your LAN, they're isolated networks that primarily serve to connect containers amongst themselves.
Yes, exactly. Don't call it 'bridge' ~20 years ago and we don't have this talk here.
> Auto-configuration of NAT between the internal bridge network and your LAN is just a convenience feature which you can disable, it's not an inherent property of bridge networks.
It's not even a property of a bridge network. For someone claming knowledge superiority this is a quite lame statement.
> links
Really.
> previous experience in an adjacent area where
Ah, yes, irrelevant 'adjacent area' of computer networks. I'm out of words.[0]
> All of this networking terminology is consistent and applied correctly
No. Go drive your wheels home.
> Don't shift the goalposts. You claimed it still doesn't exist.
I'm just mentioned RUN as an example. You decided what multiline is the goalpost.
And yes, if even the official Docker images doesn't use RUN heredocs then... there is something wrong with them. Like being implemented too late.
[0] but perfectly explains your previous 'flawed assumption'
Just like nobody ever cared that the container isn't bridged to their LAN in the first place - in fact the isolation is a feature.
But you should care, it's only the most common building block of network isolation for containers/VMs on Linux.
---
I'm not going to get into it further. I tried to explain nicely but you seem hell bent on complaining and not learning anything new. At this point the appropriate response is: RTFM, it's all in there, their docs are great.
If you don't have a clue about how things work under the hood and if you have no interest in learning anything, then stop complaining about imaginary technical inaccuracies.
If you decide to edit your tone, I'm happy to discuss more here re: filesystem transactionality tradeoffs, since that's something I've struggled with in both directions WRT containers.
I think you are the one who built an alternate layering system? On mobile, don't like to search rn.
If yes the basics are in the first comment- just not limited to the one RUN statement ofc. And if I do remember how the layers are made then making it transaction-like would be a great dedup saver eg for adding a static rarely changed artifacts - there could be essentially a ~one layer for how many you want images.
Implementation wouldn't be so difficult - you just stop committing until TRAN END; but the main is too provide a mental model for a not very experiencied dev how this supports his work and makes it more efficient on the storage and execution.
bullshit, you've been cunty right out of the gate since your first reply. I'm just surprised anyone else in the thread had as much patience as they've shown here.
you seem exactly like the type of person who is incapable of realizing they're a complete asshole and wonders why people choose to avoid them.
Now, considering you don't even know me: go in the bathroom, look atvthe mirror and repeat your comment.
Come back when the enlightment hit you.
lol What on god's green earth does this have to do with commercializing Docker? Do you really think better lighthouse scores on hub.docker.com was what was holding them back?
> what was holding them back?
Did you miss 'up to 30 secs' in the previous comment? It's not about lighthouse scores. It's the whole feeling of a hodge-podged project which is run by people who are - maybe - passionate but ultimately lacking in the basics.
The ultimate buyers of their product was the system operators and the system owners - not the developers per se. They failed hard everything that space. Having a dog slow the main content site didn't added anything positive.
Right, which is why bringing upva meaningless stat like how many assets their web site downloads feels like satire.
yes, and no. Probably both. Docker would not become such a default standard if it wasnt for all the open sourcing.
None of these things can be transposed today, so I guess I can't answer...
For what it's worth, my current project (dagger) is written in Go.
Another explicit goal was to facilitate contribution. We thought Go syntax was "vanilla" enough to attract a broader set of contributors than a more niche, opinionated language like Erlang, Haskell, Clojure etc. I think that also worked out well - Docker was known for attracting many contributors, many of them making their very first open source contributions. We even ran special meetups to help people become contributors, that was fun.
Thank you.
Actually there are some longer unedited versions I might release.
https://docs.docker.com/reference/dockerfile/
And multi stage builds are just a directed graph short hand for having separate intermediate dockerfiles. There’s nothing special about them either.
If I had to pick one thing that is tricky for someone new to docker to understand, it’s how the filesystem state is cached using the previous state, the env, and the raw command text.
Docker is much more than memorising a handful of Dockerfile directives.
Docker syntax is one of those things that just immediately falls out of my head when I close the editor. I don't care about it at all.
Even better: there are many, many tools that build OCI images without using the Docker syntax. I haven't hand-written a Dockerfile in at least six years.
LLMs have not loved the needle here in my sphere, it was already bouncing off of the zero on the gauge.
edit: this isn't to say the syntax is good or bad (though there are several questionable parts of it) but extremely simple, and concise enough that you can just look up literally any command with great ease. It's surely a large part of its success, for better or for worse.
When I was learning Docker I can't tell you how many people would describe it to me by analogy to a standardized shipping container, and start going off about shipping container dimensions and supersized cargo ships and trucks and railcars... and finish by smiling beatifically, like they had just explained the whole thing in a masterstroke. Let's... uh... not do that. :-D
In a problem set, the student might learn a bit about dockerfiles or docker-compose.yml... but only, I think, so they can try building and running a container of their own!
I remember my first time trying to use a container was about 2015 on a beach cafe's wifi in Bocas del Toro, Panama. You can imagine how it went...
After a brief hiatus though I was back to it and since then it's been a core part of many a local dev stack and even production stack (compose) I've set up. I recall the magic moment for me being when I first realised you could mount src to use all your local native dev tools while completely virtualising everything else, including and especially whatever combo of runtimes and dependencies you need.
It was just a lightbulb moment, like it entirely solves 'works on my machine', setting new people up on a project, it just seems like how software engineering ought to work and once you have experienced it it's difficult to go back.
I'm curious, as its creator are you still using it, or as someone who can clearly look ahead to experimental ways of doing things are you on to the next thing whatever that may be? If so, what is it? :-)
Yes, I still use Docker. Although my way of using it has changed over time, like many others I assume.
My new project, Dagger, although it doesn't directly compete with Docker, does implement its own container runtime and orchestrator (for CI pipelines), and we recently added compatibility with Dockerfiles and Docker Compose files... So increasingly I find myself using Dagger instead of Docker, in the context of my CI pipelines.
On the other hand, I use docker compose on almost every personal project - like a minecraft server for my kids, home automation, AI hacks, etc. These days I just let my AI agent set it up for me, though :)
Yeah I've really been enjoying using AI to write docker/compose config and related ad hoc scripts. It was all pretty understandable anyway, but now you can just set up whatever you can think of in literally minutes, even if you just want a particular configuration as a one off. It's a great era for devops!
I take it vito talked you into adopting Garden? </injoke>
“You’re overpaying for CI” (https://www.youtube.com/watch?v=tsH3eTo2GLI) from KubeCon 2024 seems especially pertinent these days with AI.
Was one of my favorite talks that year and really made Dagger click for me.
Hope to see you this year in SLC!
Even if you don't need to? Even if there's no benefit?
I was handed a vibe coded project recently, a radio station playout system. For some reason, it's spread across 8 different Docker containers. One container has just Icecast in it, for example.
It's a private system that runs a single station on a dedicated server that we have full control over, but because of all the containers, it's harder to do common tasks like check a log or edit a config file.
OTOH it's a godsend for setting up a local dev machine or something like that, where you need to run different vendor provided packages. but I'm tired of all the mindless over-engineering (which unfortunately the LLMs trained on).
For one, did you get rich beyond all comprehension and did you start a band with your wealth?
Second, do you still program ?
I was lucky enough to get in this industry when Docker was new. Sure beats trying to figure out why X happens on machine A and doesn’t on machine B.
I did not start a band, but I did build a nice house, help my family etc.
I still program everyday, working on my second startup, https://dagger.io (please forgive the plug). We have a very active discord if you want to come say hi.
I guess my follow up question is once you’re financially independent, what continues to motivate you?
Is it fun?
Do you feel like you’re creating tools that the industry needs.
I imagine if I ever get rich I’d work on music and small games.
Video games that you buy once and get to play forever, self host, etc. With a full open source commitment within 5 years or so. I’d probably create a game studio in a lower cost of living country though.
We actually use the same Discord server for community and private team communications. So we are literally there all the time.
Hit on a message's permalink (which is the "X hours ago" link), and in there you'll find the action to Favorite it.
When self-hosting Dagger, you can run it on Kubernetes. Beyond that, we don't use Kubernetes at all ourselves.
Shortly before this (the weekend after you secured YC funding, IIRC), I['m almost certain I] met you in downtown Oakland at Saagar & Tiffany's /r/bayarea meetup [0] – won't go into further details, other than I was one of their co-conspirators =P
My point of writing this is dualfold:
1) Thank you for sharing your design – I literally just installed my first Docker module (last week) and am floored at all the problems Docker might help me solve (I am not employed in tech)
2) My recollections of you, this decade+ later, was of a humble young man (we're the same age) that didn't know what else to do with his newfound techfame (and money) than keep'on keepin'on.
After watching your Docker Debut video, I'm fairly certain it WAS you: we chatted about dirtbikes (and just what the hell you were going to do now that you "had money" – then such a foreign concept to us both)... if this wasn't you it still seems to echo in your mannerisms/tact/approach. May your humbleness and cleverness prevail!
[0] It might have been at Gustafson's NYE party (I was the guy remorselessly conjuringbitches; "Volcano"); we had a mutual friend at MalwareBytes?
I’ve been following dagger and it’s been really interesting to see the evolution of the design, and new set of abstractions. There’s also nix which has seen popularity, with their approach of building layered oci containers.
> If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task!
How are you feeling about WASI today?
However Moby was a poor choice for a skunkworks project, it involved mostly communications and project management, which requires consistency and scale. Overall it was poorly executed because I failed to build enough support internally ("Solomon's at it again, let him do his thing"). Here I am trying to explain it on Twitter after the keynote, using a picture of a hand-drawn diagram, because nobody in marketing or product was assigned to help. https://x.com/solomonstre/status/855918630915133440
I think the goal of Moby was fundamentally correct: we were trying to replicate the Red Hat model, with well-scoped upstream components (linux -> runc, containerd..) integrated into a commercial product (Docker -> RHEL), with a faster-moving community project in between (Fedora -> Moby).
This was a way for us to clarify that Docker was a product, with a community of users, and Moby was an open source project, with a community of contributors. This clarity was very much needed, because different communities were projecting very different expectations on the brand "Docker", and these expectations conflicted with each other.
In a way, Moby aspired to be what CNCF ended up becoming - the "big tent" where the broader community meets to hammer out the general shape of the container stack. It seems obvious now that a single company can't be the sole authority over such a vast scope. But in the first years of Docker, we really did have that level of influence, because we literally invented the market, and brought the community together in meetups and an ever-growing Dockercon. It's only when Google and Red Hat pushed Kubernetes and CNCF, that our influence over the stack started to wane. Maybe it was for the better, I don't know if we would have been up to the task as designers and engineers. But at least there would have been a consistent design, with a Linus Torvalds figure willing and able to say "no" to stupid and redundant features. For a while I played that role - but then I was replaced by SIGs and committees.
(And as a result that, with ansible, continues to be my preferred way to deploy while the rest of the world passed me by…)
I also feel lucky that people are still using my work, that it endures without me, and that it generates (mostly) positive reactions. There was a time where Docker had a lot of haters, which was hard on me. But with time I've noticed that the negativity gets mostly washed away, and what remains is positive interactions like the ones in this whole thread. So now Docker is this cool experience I share with lots of people around the world, and when we meet we get to talk about this shared interest. I love it.
People are underestimating now big of a tech shift it was back in the day.
What was your imagined use case when you first came up with it?
Did you think Docker as it is used in the industry right now is where you imagined it would be used?
Any use cases for it that the industry missed but you think would have made things even better?
One design principle that was very important to me, was portability and avoiding fragmentation. I wanted every image name to have the same meaning in every engine, so that when writing a Dockerfile, you could have confidence that it would actually behave the same on every docker installation. In this we disagreed with other platforms (like Red Hat) who sought to "embrace and extend" Docker by changing the meaning of short names. We always fought this to avoid fragmentation of the platform. This did not make sense from Red Hat's perspective, since they were the platform - who cared about fragmenting Docker? That led to some negativity which is probably what you're remembering.
At the same time early Docker users wanted the ability to host their own images without being forced to use our central registry. Using Go's system solved both problems at once.
I personally never use the short names and have really come to like your solution. I host my own registry and I can glance at a build or compose file and immediately know if I’m depending on any external resources that I shouldn’t be.
I originally disliked it, but it’s been my favorite namespace solution for a long time now.
I was there when dotCloud announced Docker in 2013 and it was truly pragmatic. I had previously burned my hands on LXC, Juju Charms and Vagrant to containerize the apps, but it was all a house of cards. I remember helping a friend ship his whole Vagrant Box to the interview panel for a job!
It’s incredible to think about how much boilerplate Docker removed right at it's launch. Thank you.
That's for sure, especially for local dev.