Letting only one elasticsearch pod come up on a node in Kubernetes
As Egor suggested, you need podAntiAffinity:
apiVersion: apps/v1kind: Deploymentmetadata: name: redis-cachespec: selector: matchLabels: app: store replicas: 3 template: metadata: labels: app: store spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - store topologyKey: "kubernetes.io/hostname"
So, with your current label, it might look like this:
spec: affinity: nodeAffinity: # node affinity stuff here podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: "test.service.es-master" operator: In values: - "true" topologyKey: "kubernetes.io/hostname"
Ensure that you put this in the correct place in your yaml, or else it won't work.
Firstly, both in your initial manifest and even in the updated manifest you are using topologyKey
for nodeAffinity
which will give you an error while trying to deploy those manifest using kubectl create
or kubectl apply
because there is no api key called topologyKey
for nodeAffinity
Ref doc
Secondly, you are using a key called test.service.es-master
for your nodeAffinity are you sure your "node" has those labels? please confirm by this command kubectl get nodes --show-labels
Lastly, Augmenting to @Laszlo answer and your @bitswazsky comment on it to simplify it, you can use the below code:
Here I have used a node label (as key) called role
to identify the node, you can add that to your existing clusters' node by executing this command kubectl label nodes <node-name> role=platform
selector: matchLabels: component: nginx template: metadata: labels: component: nginx spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: role operator: In values: - platform podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: component operator: In values: - nginx topologyKey: kubernetes.io/hostname
This works for me with Kubernetes 1.11.5:
apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: nginxspec: replicas: 3 selector: matchLabels: test.service.es-master: "true" template: metadata: labels: test.service.es-master: "true" spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: test.service.es-master operator: In values: - "true" topologyKey: kubernetes.io/hostname nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: test.service.es-master operator: In values: - "true" containers: - image: nginx:1.7.10 name: nginx
I don't know why you chose the same key/value for the pod deployment selector label, as for the node selector. They are confusing as a minimum...