Jenkins X builds fail with "The node was low on resource: [DiskPressure]." Jenkins X builds fail with "The node was low on resource: [DiskPressure]." kubernetes kubernetes

Jenkins X builds fail with "The node was low on resource: [DiskPressure]."


Let's approach this from the Kubernetes side. There are few things you could do to fix this:

  1. As mentioned by @Vasily check what is causing disk pressure on nodes. You may also need to check logs from:
    • kubeclt logs: kube-scheduler events logs
    • journalctl -u kubelet: kubelet logs
    • /var/log/kube-scheduler.log

More about why those logs below.

  1. Check your Eviction Thresholds. Adjust Kubelet and Kube-Scheduler configuration if needed. See what is happening with both of them (logs mentioned earlier might be useful now). More info can be found here

  2. Check if you got a correctly running Horizontal Pod Autoscaler: kubectl get hpaYou can use standard kubectl commands to setup and manage your HPA.

  3. Finally, the volume related errors that you receive indicates that we might have problem with PVC and/or PV. Make sure you have your volume in the same zone as node. If you want to mount the volume to a specific container make sure it is not exclusively attached to another one. More info can be found here and here

I did not test it myself because more info is needed in order to reproduce the whole scenario but I hope that above suggestion will be useful.

Please let me know if that helped.