What does a kubectl delete do that a Crashloop Backoff Restart doesn't do?

• CrashLoopBackOff
  • Restarts the same pod and containers in the pod
  • If the pods keep crashing or failing health checks the time that it takes to restart the pod keeps increasing with every restart.
• kubectl delete
  • Deletes the pod
  • If the pods are managed by a higher abstraction: DaemonSet, Deployment, StatefulSet, etc. a new pod gets created. Note that in a StatefulSet the ordinal numbers remain constant so the pod will have the same name, but with other abstractions, your pod name will change.

My first thought is mounting a file, etc. I've seen where some issues are resolved after a Delete, even though a Crashloop was in effect.

Yes, when you delete, technically volumes are unmounted and then re-mounted on the new Pod. When a CrashLoopBackOff the containers are restarted.

From the docs:

Whilst a Pod is running, the kubelet is able to restart containers to handle some kind of faults. Within a Pod, Kubernetes tracks different container states and handles

✌️

The main difference between the two is that a crashloop backoff restarts the containers, but deleting a pod restarts the whole pod.

There are a few actions that happen on a pod start that don't happen on container start. From a troubleshooting standpoint, the ones to care about are as follows:

• Volume Mounts
• Loading in values from Configmaps / Secrets

Volume mounts aren't a problem very often, as the pod just retries to mount them until it works. It's rare to see an issue with volume mounts that is only fixed by deleting the pod.

However, configmaps and secrets can be a huge problem. A pod only loads in the values from configmaps and secrets on startup. Once a pod is up, it will forever ignore any changes to the configmaps and secrets that it is using.

Therefore, if you change a configmap or secret used by a pod, you will see a difference between deleting the pod, and the crashloop backoff restart.