Docker is unable to start due to `dm_task_set_cookie failed`

Fixed I did echo 'y' | sudo dmsetup udevcomplete_all and it was able to start

To clarify on the original question/answer here since many others have had this problem (or jump to the solution below):

The general issue here is that earlier versions of Docker could have devicemapper leak semaphores/cookies outside of a container's mount space. For me, this happened after doing multiple rapid calls to docker build, run, rm, stop, etc.

This could cause the system's semaphore arrays to fill up without being cleared, which would prevent all docker commands from working (and may impact other system functions relying on those shared semaphores).

You could solve this by increasing the number of semaphore arrays, or by clearing some of the old arrays as I do below.

The accepted answer uses clearing all semaphores with dmsetup udevcomplete_all, but you may not want to do this if you have other containers or processes relying on those cookies/semaphores.

Instead, check your semaphore arrays -- these were full on my machine (128 arrays):

$ ipcs -u------ Semaphore Status --------used arrays = 128allocated semaphores = 1136------ Messages Status --------allocated queues = 0used headers = 0used space = 0 bytes------ Shared Memory Status --------segments allocated 2pages allocated 20057pages resident  16214pages swapped   2Swap performance: 0 attempts     0 successes

These semaphores correspond to dm cookies, as shown with ipcs and dmsetup udevcookies (showing only recent cookies; look for the matching semid):

$ ipcs -s -tsemid    owner      last-op                    last-changed              540278908 root        Sat Jun 13 10:03:57 2020   Sat Jun 13 10:03:57 2020  548634749 root        Sat Jun 13 10:09:22 2020   Sat Jun 13 10:09:22 2020  555876478 root        Sat Jun 13 10:14:05 2020   Sat Jun 13 10:14:05 2020  572096639 root        Sat Jun 13 11:43:42 2020   Sat Jun 13 11:43:42 2020 ...$ dmsetup udevcookiesCookie       Semid      Value      Last semop time           Last change time0xd4d819c    540278908  1          Sat Jun 13 10:03:57 2020  Sat Jun 13 10:03:57 20200xd4d8d60    548634749  1          Sat Jun 13 10:09:22 2020  Sat Jun 13 10:09:22 20200xd4d8eca    555876478  1          Sat Jun 13 10:14:05 2020  Sat Jun 13 10:14:05 20200xd4d7263    572096639  1          Sat Jun 13 11:43:42 2020  Sat Jun 13 11:43:42 2020...

To avoid impacting the older semaphores/cookies, I wanted to release only cookies for certain dates using msetup udevreleasecookie <cookie ID>:

# filter the list of cookies, get the IDs, and feed it to udevreleasecookie:$ REGEX_STR="Jun 13"    # change to whatever filter you need$ dmsetup udevcookies | grep -E "$REGEX_STR" | awk '{ print $1 }' | xargs -n1 dmsetup udevreleasecookie# check that the semaphores are gone$ ipcs -s -t | grep -E "root" | grep -E "$REGEX_STR"    ...$

This allowed me to run docker commands again.