Docker build not using layer cache

I found the answer by the way :) The Dockerfile itself was ok except one problem I could then find via docker history Which shows the real shell command which are executed by docker.

The Problem was that ARG BUILD_VERSION lead to docker adds to every run command the environment variable like /bin/sh -c "ARG=123 ./sbt ...". This leads to a different call signature and a different hash every time the arg changes and therefore the run command is not applied from cache. To fix this issue just move the ARG down to the first RUN command which needs it.

FROM openjdk:8 as workspaceWORKDIR /buildCOPY ./sbt ./sbtCOPY ./sbt-dist ./sbt-distCOPY ./build.sbt ./build.sbtCOPY ./project/build.properties ./project/build.propertiesCOPY ./project/plugins.sbt ./project/plugins.sbtRUN ./sbt -sbt-dir ./sbt-dir -ivy ./ivy updateCOPY ./ ./# Embedded postgres need to be run as non-root userRUN useradd -ms /bin/bash runnerRUN chown -R runner /buildUSER runnerRUN ./sbt -sbt-dir ./sbt-dir -ivy ./ivy clean testARG BUILD_VERSIONRUN ./sbt -sbt-dir ./sbt-dir -ivy ./ivy docker:stage -Ddocker.image.version="${BUILD_VERSION}"

As the docker build output shows, the building stopped using the cache at the stage:

4:32:35 PMStep 4/22 : ADD ./sbt ./sbt4:32:36 PM---> 7a9e21819cea

This is happening because there have been changes in the ./sbt folder on the host and thus docker will rerun the ADD instruction. From there on, docker will not use the cache anymore as a base layer was invalidated causing the layers after it to also be invalidated.

The change might be in the contents of the directories, but rather something trivial such as a timestamp.

As a best practice, to get advantage of the build caching mechanism, you should delay instructions that change often to the bottom of the Dockerfile. In general, adding source to a docker image is one of these steps that changes the very frequently between builds.