Make Your Own Bespoke Docker Image

Recently, a few ideas around Docker converged in my mind and led me to make my own distribution, and to enable others to do the same. Hear me out.

• Docker is a software packaging tool (a point hammered home to me in a talk by Nic Ferrier).
• Docker separates build from deployment.
• Docker allows for stateless builds (and ShutIt on top of that allows for complex stateless builds)
• Linux distributions are essentially packaging systems.
• It’s really annoying when you get “distribution clash” between Docker builds, eg when someone writes a Dockerfile for Debian and then you want to integrate it with one written for Centos. This happens.
• Docker has a layered filesystem, so bloat only matters if you don’t re-use layers.
• Linux from Scratch exists (and rocks).

Putting all the above together I realised that with the advent of Docker there was less of a need to bother with traditional packaging systems at all. Why not build from source?

“Because it’s bloody hard, bloody time-consuming, and bloody painful” is the standard answer. But since you can reproduce builds at will deterministically and deploy later elsewhere, much of that difficulty can be offloaded to someone else. Add in Docker’s layered filesystem, and these other issues become easier:

• Bug/build issue reproduction (complete audited steps).
• More efficient space usage (build in the layers/applications you need and no more, share common layers down the dependency hierarchy).
• Tweaking your distro from a given layer is relatively easy (just add a ShutIt module).
• Collaboration on improvements much more effective (we’re all talking about the same thing).
• Auditing builds (for escrow purposes, or even security purposes).
• A common reference point for applications built in Docker.
• Dockerfiles/ShutIt can help produce self-documenting dependencies and build steps for porting elsewhere if desired.
• You can have more bleeding edge versions of software if you want (or configure and try older ones if preferred)
• Automated testing is trivial (just run the build, use ShutIt’s test phase).
• Patching is trivial (docker pull, or just hack, commit, and tag the image).

The ShutIt Docker Distro

So I built (and present) the ShutIt Distro. Based on the giant shoulders of Linux From Scratch (and Beyond Linux From Scratch) and the broad shoulders of Docker it allows you to effectively create a bespoke distribution to target the application you want, mixing different applications with a few clicks, dependencies managed and all from source.

There’s a lot of images already available here, eg:

OSQuery

rpm

node

emscripten

all built from source and with a full bash history available for reference.

Here is the sequence of commands to get from debian:jessie (docker pull imiell/sd_base) to the sd_base filesystem artifact lfs.tar.xz (see below). Empty lines indicate just hitting return to accept defaults.

Here is the sequence of commands to get from imiell/sd_base to a complete working OSQuery image.

And here’s an example of how to run the OSQuery image:

$ docker run -t -i imiell/sd_osquery
bash-4.3# osqueryi
[...]
osquery> select name, path from processes;                  
+----------+-------------------------+
| name     | path                    |
+----------+-------------------------+
| bash     |                         |
| osqueryi | /usr/local/bin/osqueryi |
+----------+-------------------------+
osquery>

Dependencies

Since the environment is so predictable and restricted, certain aspects of the dependency management process become simpler.

ShutIt allows you to produce dependency graphs of your modules with the sc (show config) sub-command. Here is a gist example.

Here’s a (big – get ready with the zoom!) image of all the dependencies for the whole of the ShutIt-Distro module “universe” as it stands today:

and a much smaller one just for the modules needed for the OSQuery module:

and NodeJS‘s much simpler one (bear in mind that the base image has a base toolchain et al in it:

Build Your Own Image From Source

Want to cherry pick your own image, with tools and editors to your fancy? Instructions here, raw cut and paste for Ubuntu here You’ll be presented with a sparse web interface like this:

On the left are the modules in strict dependency order. At the bottom are the more complex modules. If we take OSQuery as an example, clicking on that will cause all the dependent modules to be emboldened (this can take a few seconds if there are lots of deps):

You can add any other modules you like to your image. So if you’re an emacs user, you can tick that box and it’ll be added also. This way you can configure your own personalized image. Then scroll back up and click on “Begin Build” to kick off the build. The commands issues will be on the third pane, the log on the fourth.

When the build is complete, it can be downloaded as a tar file by clicking on “Export Container”, which you can use to load in like so:

cat downloaded.tar | docker load

which will import the image ready to run. Using the web interface is not required, but it’s obviously easier than using the command line and configs, and is a good intro to the ShutIt framework.

How the Base Toolchain Image is Produced

Initially I tried to build the entire toolchain within debian:jessie, but got myself into hot water pretty quickly. Turns out building a toolchain from scratch is quite hard. Thankfully, Automated Linux From Scratch exists, so I just used that.

That delivers an artifact called lfs.tar.xz to the filesystem which can be picked up to make a base image with this Dockerfile:

FROM scratch
ADD lfs.tar.xz /
CMD ["/bin/bash"]

The base image is then delivered to the Docker Hub here. This forms the basis for the ShutIt Distro. Here is a gist for the commands to do this yourself. Note: it takes a while!

Lessons Learned

• Sourceforge is in DR mode a lot. In fact, primary sites (eg gnu.org the other day for a couple of hours) generally are down surprisingly often.
• You’re a complete idiot if you deviate from Linux From Scratch one iota. I did this a few times and was beaten back into line several times with brutal severity.
• NodeJS and Ruby have surprisingly few dependencies needed to build from source.
• apt-file is an incredibly handy tool.

Help Wanted

If you’ve read this far and are still interested, do get in touch.

There’s plenty to grep TODO :)