Hazelcast failes liveness probe in OpenShift when loading a large map Hazelcast failes liveness probe in OpenShift when loading a large map kubernetes kubernetes

Hazelcast failes liveness probe in OpenShift when loading a large map


You can increase the time for POD to check the health of POD if you are loading data initially.

https://github.com/hazelcast/charts/blob/d3b8d118da400255effc81e67a13c1863fee5f41/stable/hazelcast/values.yaml#L120

Above is helm example line showing the readiness and liveness configuration.

You can change the initialDelaySeconds to wait and after that seconds only Hazelcast will start checking the health.

Accordingly you can also adjust the other probes configuration like failureThreshold: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

if initialDelaySeconds is not helpful, still you can increase the configuration for the readiness & liveness accordingly and increase the timeout whenever map loader loading the data.

Also only in liveness your POD or container get restart if Readiness failing K8s will mark your POD as not ready to accept traffic. So while loading data you can increase the liveness configuration so it never failed and set to max possible.

The kubelet uses liveness probes to know when to restart a container.For example, liveness probes could catch a deadlock, where anapplication is running, but unable to make progress. Restarting acontainer in such a state can help to make the application moreavailable despite bugs.

The kubelet uses readiness probes to know when a container is ready tostart accepting traffic. A Pod is considered ready when all of itscontainers are ready. One use of this signal is to control which Podsare used as backends for Services. When a Pod is not ready, it isremoved from Service load balancers.

You can also set the restartPolicy to Never so the container will not restart.