Apache Ignite on Kubernetes not joining cluster
I figured what the issue is here. Apparently it has to do with the way I configured the service object in kubernetes. I am not sure if this is a bug or a feature but it looks like an Ignite node can only scale to node and not across nodes.What I mean by this is the service object should be unique to a node. If you share the service object across nodes (microservices) expecting the cluster to spread across multiple nodes it will hang. (I am not sure if this is an anti-pattern)What worked was keeping the service object unique to the node and then scaling the node if required.
I think if this is the case then we should probably keep the ignite nodes as a separate cluster and not embedded within the micro-services.
as per your Kubernetes Deployment, you likely have defined a readinessProbe on your spec.template.spec.container
This will prevent the Pod to be registered as Endpoints under the Kubernetes Service, and each Ignite embedded node will start its own cluster of 1 node :-/
give it a try without the readinessProbe and see if your Ignite nodes join the same cluster.