No space on device with Jenkins and Docker - how to cleanup properly

Docker < 1.13

For Docker older than 1.13 you can do following for cleaning up some space on your device:

docker ps -a | grep -i 'exited' | awk '{print $1}' | xargs docker rm > /dev/null 2>&1 &docker images -a | grep "<none>" | awk '{print $3}' | xargs docker rmi > /dev/null 2>&1 &

Alternatively you can try running following docker command:

docker rmi $(docker images --filter "dangling=true" -q --no-trunc)

It will clean old orphan containers and will remove images tagged with <none>. I use these two formulas on one of my CI servers and it works fine. Before that I was facing similar to your issue (no space left on device).

Cleaning orphan volumes

docker volume rm $(docker volume ls -qf dangling=true)docker volume ls -qf dangling=true | xargs -r docker volume rm

Docker >= 1.13

Docker 1.13 introduces docker system prune command (https://docs.docker.com/engine/reference/commandline/system_prune/). Alternatively you can run:

• docker image prune
• docker volume prune
• docker container prune

You can run those commands as a part of your Jenkins pipeline. In one of the projects I work on we run cleanup after building new Docker images during the release process. Try it as well to fix "Cannot connect to the Docker daemon. Is the docker daemon running on this host?" problem.

After using the script provided by burnettk below it seemed that while some space was freed after time running more builds I was back at the same place, no space on my EBS volume. It simply does not make sense that I would have to add more storage and pay AWS even more on my monthly bill.

In doing some investigation I discovered that for EACH build there were approximately 7 images created (docker images -a) consisting of about 1.4GB each, ie 9GB/build. The first 2 are tagged with the build # and latest while the rest are tagged .

It's really not important that all these images are stored on this server as the purpose is for build and anyway they are pushed to ECR. So I've added the following into my script so that only the latest docker image is kept:

docker rmi $(docker images | sed 1,3d | awk '{print $3}')

Lastly, I have also adjusted my docker build command by adding the --rm argument so that it will remove intermediate containers after building.

docker build --rm

Hope this is helpful!

in order to get past the "cannot connect to the docker docker daemon" issue, figure out what users are in the docker group

grep 'docker' /etc/group

and then run the docker cleanup commands (you'll want to turn it into a script you run on cron or something) as one of those users. or get sudo access with another user and use sudo:

sudo docker rmi [image_name_here]

here's the contents of an example cleanup script (/usr/local/bin/clean_up_docker_stuff_on_ci_agent or similar):

#!/bin/bash# stop containers that have been running for more than a day (may not be valid in your context if you intend run things for a long time)docker ps -a | egrep " days" | awk '{print $1}' | grep -v CONTAINER | xargs docker stop# remove all exited containersdocker ps -a | egrep "Exited|Created" | awk '{print $1}' | grep -v CONTAINER | xargs docker rm# remove old imagesdocker images | egrep 'weeks|months' | awk '{print $1 ":" $2}' | xargs docker rmi -fdocker images | egrep 'weeks|months' | grep '<none>' | awk '{ print $3 }' | xargs docker rmi -f# kill stray volumesdocker volume ls -qf dangling=true | xargs -r docker volume rm

As Szymon Stepniak mentions in his answer, if you're using docker >= 1.13, there are simpler options.

cron example (20 after every hour):

20 * * * * /usr/local/bin/clean_up_docker_stuff_on_ci_agent > /dev/null 2>&1