After 6 years, I removed Docker from all my home servers.
apt purge -y docker-ce
Why?
This was triggered by a recurring incident I faced where the Docker daemon was using 100% CPU on multiple cores that made the host effectively unusable.
This had happened a few times before, and was likely due to a script that had got out of hand starting up too many containers. I’d never really got to the bottom of it, as I had to run a command to kill off all the containers and restart the daemon. This time, the daemon wouldn’t restart without a kill -9, so I figured enough was enough.
Anyway, I didn’t necessarily blame Docker for it, but it did add force to an argument I’d heard before:
Why does Docker need a daemon at all?
Podman, Skopeo, and Buildah
These three tools are an effort mostly pushed by RedHat that do everything I need Docker to do. They don’t require a daemon or access to a group with root privileges.
Podman
Podman replaces the Docker command for most of its sub-commands (run, push, pull etc). Because it doesn’t need a daemon, and uses user namespacing to simulate root in the container, there’s no need to attach to a socket with root privileges, which was a long-standing concern with Docker.
Buildah
Buildah builds OCI images. Confusingly, podman build can also be used to build Docker images also, but it’s incredibly slow and used up a lot of disk space by using the vfs storage driver by default. buildah bud (‘build using Dockerfile’) was much faster for me, and uses the overlay storage driver.
The user namespacing allowing rootless builds was the other killer feature that made me want to move. I wrote a piece about trying to get rootless builds going last year, and now it comes out of the box with /etc/subuid and /etc/subgid set up for you, on Ubuntu at least.
Skopeo
Skopeo is a tool that allows you to work with Docker and OCI images by pushing, pulling, and copying images.
The code for these three are open source and available here:
Podman
Buildah
Skopeo
Steps to Move
Installing these tools on Ubuntu was a lot easier than it was 6 months ago.
I did seem to have to install runc independently of those instructions. Not sure why it wasn’t a pre-existing dependency.
First, I replaced all instances of docker in my cron and CI jobs with podman. That was relatively easy as it’s all in my Ansible scripts, and anything else was a quick search through my GitHub repos.
Once that was bedded in, I could see if anything else was calling docker by using sysdig to catch any references to it:
sysdig | grep -w docker
This may slow down your system considerably if you’re performance-sensitive.
Once happy that nothing was trying to run docker, I could run:
apt remove -y docker-ce
I didn’t actually purge in case there was some config I needed.
Once everything was deemed stable, the final cleanup could take place:
- Remove any left-over sources in
/etc/apt/*that point to Docker apt repos - Remove the docker group from the system with
delgroup docker - Remove any left-over files in
etc/docker/*,/etc/default/dockerand/var/lib/docker
A few people asked what I did about Docker Compose, but I don’t use it, so that wasn’t an issue for me.
Edit: there exists a podman-compose project,
but it’s not considered mature.
Differences?
So far, and aside from the ‘no daemon’ and ‘no sudo access required’, I haven’t noticed many differences.
Builds are local to my user (in ~/.local/containers) rather than global (in /var/lib/docker), in keeping with the general philosophy of these tools as user-oriented rather than daemon-oriented. But since my home servers have only one user using Docker, that wasn’t much of an issue.
The other big difference I noticed was that podman pull downloads get all layers in parallel, in contrast to Docker’s. I don’t know if this causes problems if too many images are being pulled at once, but that wasn’t a concern for me.
If you like this, you might like one of my books:
Learn Bash the Hard Way
Learn Git the Hard Way
Learn Terraform the Hard Way
zwischenzugs 
Podman uses buildah under the hood to build images.
Podman uses buildah to build images. Doesn’t do it on its own.
You said that some of the containers were consuming 100% of CPU cores, but you said nothing about trying to limit the resources. If your containers consume too much, perhaps you did not configure them of you docker right?
I’ve switched to podman and buildah too. Now our CI system is happy to build all the stuff faster and without painfull workarounds.
Docker just has too many bugs lasting forever.
I didn’t even change docker calls for that as podman has podman-docker wrapper which allows you to call docker command to execute actual podman.
There was some refactoring in dockerfiles as buildah had behaven more restrictive way.
Good job, now finish the deal by moving to Red Hat for your servers :) If you don’t want to pay, go with Fedora and CentOS. They both have great integration with Red Hat products, including Ansible which will help you automate your servers.
Could you please explain little bit more, how you the docker deployment setup on RHEL? Docker-ce or EE? How to auto update and usage of docker compose? Very interested on best Practice
Thanks for providing those alternatives, what do you think of docker mentioning the CPU issue and a fix : https://success.docker.com/article/how-to-fix-slow-ucp-or-dtr-login-with-high-cpu
I think also if you upgraded , this issue won’t be there
Unless you want docker instances to run at specific time cron shouldnt be use if it is ob restart of system you should tell docker to start a container in bot that is why docker requires a daemon also for control software to run independant of any user as a service it also makes it a tad bit more secure during shutdowns
Set up live reload, the daemon and containers are then independent. The daemon can be restarted without affecting containers.
I’ve rarely seen the daemon driving up cpu usage but you can use tools like this to automatically shoot and restart it. Likely a daemon using all the cpu also isn’t being responsive so this script will restart it https://github.com/kubernetes/kubernetes/blob/master/cluster/gce/gci/health-monitor.sh
Obviously youve done the legwork to switch off docker but these are easier options for people that continue to use docker. It is the most reliable solution today
I get where you’re coming from with this but can’t help wondering why the modern webdev solutions to problematic software components is always to replace them with multiple new solutions that add even more complexity to an already overly complex stack.
Perhaps for some people it is less complex? I find the approach of one tool to do it all to be more complex than a set of tools where each focuses on doing one thing (commonly called the Unix approach). I can understand each simple tool in its entirety knowing it has a small bounded scope. I can then compose those tools is ways the original designers may not have thought of.
I like building blocks. Some like all in one. Luckily we have both now and thanks to the author of this article, people who prefer the first approach now have a guide to getting started.
We got this issue with high CPU usage by docker daemon, we ended up manually upgrade to docker ce 18.9 version and the issue is gone
I am an editor at http://www.InfoQ.cn , May I translate your post : Goodbye Docker: Purging is Such Sweet Sorrow into Chinese for appropriate credit.
Yes, but please post link back here.
InfoQ publish link：https://www.infoq.cn/article/OLMbGbRDw-IKekPDqkDc
Advertising your Docker book below a post where you are talking about how you moved away from it.
p.s. Such sweet thunder
Well, moving away from it _at home_. $WORK still using it… for now.
Stay far away from IBM (Red Hat) ecosystem- e.g. podman, bulidah, etc. Nice attempt to shill this crap
What say that and not provide a link or at least a reason about your claims?
Obvious troll is obvious…
Which version of Ubuntu did you set this up on? I tried this on Ubuntu 18.04 and was hit by this issue https://github.com/containers/buildah/issues/1709, and I don’t know fuse-overlayfs will be available in 18.04 :P
Insightful write up, thank you!
Yes, it was 19.04. fuse-overlayfs and runc need to be installed first, I have found out after this article went out.
Thanks for the confirmation!
thank you for this article this is very good
thank you fort his amazing article this is very interesting
Great blogg I enjoyed reading