Headers based routing with Kubernetes
Kubernetes Ingress works on OSI Layer7, so it can take into account HTTP headers, but it only forwards traffic to Kubernetes Services, not to Pods.
Unfortunately, Kubernetes Services in turn, can't deliver traffic to specific pods depending on HTTP headers, because Service is basically set of iptables rules that deliver traffic to pod, only analyzing data on OSI Layer4 (IP address, tcp/udp, port number).
For example, let's look at kube-dns service iptables rules:
# kube-dns service-A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU # random load balancing traffic between pods-A KUBE-SVC-TCOU7JCQXEZGVUNU -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-AALXN3QQZ3U27IAI-A KUBE-SVC-TCOU7JCQXEZGVUNU -j KUBE-SEP-WTX2W5TQOZGP42TM# dns pod 1-A KUBE-SEP-AALXN3QQZ3U27IAI -p udp -m udp -j DNAT --to-destination 10.244.0.16:53# dns pod 2-A KUBE-SEP-WTX2W5TQOZGP42TM -p udp -m udp -j DNAT --to-destination 10.244.0.17:53I can imagine only one realistic way to deliver traffic to specific pod based on HTTP Headers.
Configure custom Ingress controller which can set the headers for each Client session and use known DNS names of pods as a back-end destination points. I can't recommend particular solution, so in worst case it could be created using some examples.
Kubernetes StatefulSet creates Pods with predictable names like statefulset-name-0, statefulset-name-1, etc. Corresponding Headless Service (ClusterIP: None) creates DNS names for each Pod.
For example, for StatefulSet nginx-ss with three replicas, three pods would be created and Service nginx-ss would create three DNS A records for Pods:
nginx-ss-0 1/1 Running 10.244.3.72 nginx-ss-1 1/1 Running 10.244.3.73 nginx-ss-2 1/1 Running 10.244.1.165 nginx-ss-0.nginx-ss.default.svc.cluster.local. 5 IN A 10.244.3.72nginx-ss-1.nginx-ss.default.svc.cluster.local. 5 IN A 10.244.3.73nginx-ss-2.nginx-ss.default.svc.cluster.local. 5 IN A 10.244.1.165
Assuming that 'pod_location' is inserted into HTML header by app that is running on that pod, the Ingress (and Ingress Controller) can be used to achieve header based routing.
For example, Traefik v2.0 has the new Custom Resource Definition (CRD) called IngressRoute that extends the Ingress spec and adds support for features such as Header based routing.
In the following example, I have two services: one exposing an Nginx deployment and other one exposing an Apache deployment. With the IngressRoute CRD, the match for the router will be the header X-Route:
apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: headers spec: entrypoints: - web - websecure routes: - match: Headers(`X-ROUTE`,`Apache`) kind: Rule services: - name: apache port: 80 - match: Headers(`X-ROUTE`,`nginx`) kind: Rule services: - name: nginx port: 80Full example
With the X-ROUTE: Apache header:
curl http://46.101.68.190/ -H 'X-ROUTE: Apache' html><body><h1>It works!</h1></body></html>With the X-ROUTE: nginx header:
> curl http://46.101.68.190/ -H 'X-ROUTE: nginx'<!DOCTYPE html><html><head><title>Welcome to nginx!</title>...and so on...And Traefik provides additional info with configuration examples for their middleware .
If you want to make sure that connections from a particular client are passed to the same Pod each time, you can select the session affinity based on client’s IP addresses by setting service.spec.sessionAffinity to “ClientIP” (the default is “None”) in the service definition YAML.
You can also set the maximum session sticky time by setting service.spec.sessionAffinityConfig.clientIP.timeoutSeconds appropriately. (the default value is 10800, which works out to be 3 hours)