Kubernetes ingress rules for external service (with ExternalName type)
Explanation of what's going on:
kube-proxy is responsible for implementing a form of virtual IP for Services of type other than ExternalName
The ingress controller is trying to resolve the external-service which has CNAME internaldns.com and when your kube-dns/coredns (10.96.0.10) tries to do a lookup, it can only find CNAME record but no A record, therefore, your ingress is failing to resolve the DNS name.
When looking up the host my-service.prod.svc.cluster.local, the cluster DNS Service returns a CNAME record with the value my.database.example.com
Moreover there is a clear warning on the website about ExternalNames:
You may have trouble using ExternalName for some common protocols, including HTTP and HTTPS. If you use ExternalName then the hostname used by clients inside your cluster is different from the name that the ExternalName references.
TL;DR: ingress is trying to resolve a DNS using kubernetes DNS(kube-dns/coredns) which doesn't have any A record, hence fails to associate DNS to IP!
If ingress was made to lookup a different DNS server (other than kubernetes DNS) which has A record entry for internaldns.com then this problem may not have happened but I am not 100% sure if that's possible .
Solution: - Create a Headless service without selector and then manually create an endpoint using the same name as of the service. Follow the example here
Note:
- in the above solutions you will need the static IP of the external service.
- I would use ExternalNames only when my Pod directly wants to talk to a 3rd party service over the internet, which is to say the service is hosted outside my local network. I would follow such an approach because if I can access something locally via IP why would deter performance by talking to a Nameserver to resolve DNS name!