EventStore on Kubernetes: Connection refused
We encountered the same issue running EventStoreDB on a Kubernetes cluster with Istio sidecar injection enabled.
According to Istio's documentation on protocol selection, Istio will look at the name of the port
you defined on your Service
, and this will decide which protocol Istio is trying to intercept. If the format is not respected, Istio will try to guess the protocol (works for HTTP, HTTPS and gRPC).
In your case, your ports' name
started with http-
(http-int
and http-ext
). Therefore, Istio will not try to detect the protocol used, but instead will assume that the protocol is http
(HTTP/1.1).
However, EventStoreDB's API is a gRPC endpoint. Therefore, you have two options:
- Rename the port to start with
grpc-
. In that case any Istio proxy will know that this port is exposing gRPC - Name the port somethig else directly (like
api
oreventstoredb
for example), to let Istio detect the protocol used.
Note that EventStoreDB exposes a admin web interface on the same port, which is HTTP. If you are accessing it through port-forwarding, then you have no Istio sidecar in the way, so the port's name will not influence the traffic. But if you try to expose the admin interface through an Istio Ingress Gateway (which I wouldn't recommend since you would be exposing your database to the Internet), then you might have issues accessing the admin interface. In that case, the second solution to let Istio detect the traffic is probably a more flexible solution.
A last option would be to expose 2 ports on the Service
, one for http
and the other for grpc
, and have them both redirect to the same port on the Pod
, but I'm actually not sure if this is allowed by Kubernetes.