kubernetes and debugging it in general

My experience using kubernetes on aws while getting unhelpful errors as

Error from server: dial unix /var/run/docker.sock: no such file or directory

was resolved by choosing a more hefty aws cluster instance type ... here are relevant env vars

export MASTER_SIZE=t2.mediumexport NODE_SIZE=t2.mediumexport NUM_NODES=2   #  if not defined aws will auto guess

... also remove mention of resource limiting settings under tag resources until after it runs OK

Following commands are essential ... just leave off mention of namespace if you are not using

kubectl describe svc --namespace=xxx

kubectl get pods --namespace=xxx

kubectl describe pods --namespace=xxx

kubectl describe nodes

Also nice is ability to perform a live edit of a deployment ... first see your deployments .. issue

kubectl get deployments  --namespace=ruptureofthemundaneplane 

... output

NAME                   DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGEloudspeed-deployment   1         1         1            1           1hmongo-deployment       1         1         1            1           1h

so now we know name of deployment to do live edit issue

kubectl edit deployment/mongo-deployment

which will open an edit session in your terminal using default editor where you can change settings at will

I find when troubleshooting a database deployment its handy to also launch an image using below Dockerfile ... this allows you login using exec as per

kubectl exec -ti $(kubectl get pods --namespace=${PROJECT_ID}|grep ${GKE_NODEDEPLOYMENT}|cut -d' ' -f1) --namespace=${PROJECT_ID} -c ${GKE_NGINX} -- bash

where you are free to do interactive database login session (once you install needed client code or put same into below Dockerfile) ... here is matching Dockerfile for this troubleshooting deployment container

FROM ubuntu:16.04ENV TERM linuxENV DEBIAN_FRONTEND noninteractiveRUN apt-get updateRUN apt-get install -y wget  curlCOPY .bashrc /root/# ENTRYPOINT ["/bin/bash"]CMD ["/bin/bash"]## docker build --tag stens_ubuntu .## docker run -d  stens_ubuntu  sleep infinity## docker ps## # ... find CONTAINER ID from above and put into something like this## docker exec -ti 3cea1993ed28 bash##