What is ‘Busting the Cache’?
If you’ve ever spent any time building Docker images, you will know that Docker caches layers as they are built, and as long as those lines don’t change, Docker treats the outputted layer is identical
There’s a problem here. If you go to the network to pick up an artefact, for example with:
RUN curl https://myartefactserver.local/myjar.jar > myjar.jar
then Docker will treat that command as cache-able, even if the artefact has changed.
Solution 1: –no-cache
The sledgehammer solution to this is to add a
--no-cache flag to your build. This removes the caching behaviour, meaning your build will run fully every time, no matter whether the lines of your Dockerfile change or not.
Problem solved? Well… not really. If your build is installing a bunch of other more stable artefacts, like this:
FROM ubuntu RUN apt-get update -y && apt-get install -y many packages you want to install # ... # more commands # ... RUN curl https://myartefactserver.local/myjar.jar > myjar.jar CMD ./run.sh
Then every time you want to do a build, the cycle time is slow as you wait for the image to fully rebuild. This can get very tedious.
Solution 2: Manually Change the Line
You can get round this problem by dropping the
--no-cache flag and manually changing the line every time you build. Open up your editor, and change the line like this:
RUN [command] # sdfjasdgjhadfa
Then the build will But this can get tedious.
Solution 3: Automate the Line Change
But this can get tedious too. So here’s a one-liner that you can put in an alias, or your makefile to ensure the cache is busted at the right point.
First change the line to this:
RUN [command] # bustcache:
and then change your build command to:
perl -p -i -e "s/(.bustcache:).*/\1 $RANDOM/" Dockerfile && docker build -t tag .
perl command will ensure that the line is changed to a random number generated by the shell.
There’s a 1/100,000 chance that the number will repeat itself in two runs, but I’m going to ignore that…
If you like this, you might like one of my books:
Learn Bash the Hard Way
Learn Git the Hard Way
Learn Terraform the Hard Way
If you enjoyed this, then please consider buying me a coffee to encourage me to do more.
3 thoughts on “Surgically Busting the Docker Cache”
Replace $RANDOM with $(uuid) and you don’t have to worry about the chance the text will occur twice.
…assuming `uuid` is available on the building system…
I often use build-args to implment cache bust:
then build the image using `docker build –build-arg CACHEBUST=$(date +%s)`