docker - driver "devicemapper" failed to remove root filesystem after process in container killed

I had the same problem and the solution was a real surprise.

So here is the error om docker rm:

$ docker rm 08d51aad0e74Error response from daemon: driver "devicemapper" failed to remove root filesystem for 08d51aad0e74060f54bba36268386fe991eff74570e7ee29b7c4d74047d809aa: remove /var/lib/docker/devicemapper/mnt/670cdbd30a3627ae4801044d32a423284b540c5057002dd010186c69b6cc7eea: device or resource busy

Then I did the following (basically go through all processes and look for docker in mountinfo):

$  grep docker /proc/*/mountinfo | grep 958722d105f8586978361409c9d70aff17c0af3a1970cb3c2fb7908fe5a310ac/proc/20416/mountinfo:629 574 253:15 / /var/lib/docker/devicemapper/mnt/958722d105f8586978361409c9d70aff17c0af3a1970cb3c2fb7908fe5a310ac rw,relatime shared:288 - xfs /dev/mapper/docker-253:5-786536-958722d105f8586978361409c9d70aff17c0af3a1970cb3c2fb7908fe5a310ac rw,nouuid,attr2,inode64,logbsize=64k,sunit=128,swidth=128,noquota

This got be the PID of the offending process keeping it busy - 20416 (the item after /proc/)

So I did a ps -p and to my surprise find:

[devops@dp01app5030 SeGrid]$ ps -p 20416  PID TTY          TIME CMD20416 ?        00:00:19 ntpd

A true WTF moment. So I pair problem solved with Google and found this: Then found this https://github.com/docker/for-linux/issues/124

Turns out I had to restart ntp daemon and that fixed the issue!!!