Colocate pods in Kubernetes

If you want to have strict separation of the workloads over the two zones - I'd suggest using nodeSelector on a node's zone.

A similar result is possible with pod affinity but it's more complex and to get the clear split you describe. You'd need to use the requiredDuringScheduling / execution rules which are usually best avoided unless you really need them.

Just like you used anti-affinity rules to avoid provisioning both apps in the same zone, you can use affinity to provision app-0 only in the zone where db-0 exists, technically that would mean that you could drop anti affinity from app completely as if you tell it to schedule only in the zone of db, and db is defined to spread zones with anti-affinity, you inherit the distribution from the database part.

PodAntiAffinity to spread to Zones

For your database

metadata:  labels:    app: dbspec:  affinity:    podAntiAffinity:      requiredDuringSchedulingIgnoredDuringExecution:      - labelSelector:          matchExpressions:          - key: app            operator: In            values:            - db        topologyKey: "kubernetes.io/hostname"

For your web-app

metadata:  labels:    app: web-appspec:  affinity:    podAntiAffinity:      requiredDuringSchedulingIgnoredDuringExecution:      - labelSelector:          matchExpressions:          - key: app            operator: In            values:            - web-app        topologyKey: "kubernetes.io/hostname"

PodAffinity to co-locate pods on nodes

In addition, you add podAffinity to your web-app (not database) to co-locate it on nodes with your database.

With podAffinity added for your web-app:

metadata:  labels:    app: web-appspec:  affinity:    podAntiAffinity:      requiredDuringSchedulingIgnoredDuringExecution:      - labelSelector:          matchExpressions:          - key: app            operator: In            values:            - web-app        topologyKey: "kubernetes.io/hostname"    podAffinity:      requiredDuringSchedulingIgnoredDuringExecution:      - labelSelector:          matchExpressions:          - key: app            operator: In            values:            - db        topologyKey: "kubernetes.io/hostname"

Result

With both PodAntiAffinity and PodAffinity you will get web-app-X co-located with db-X.

web-app-1     web-app-2db-1          db-2

See the Kubernetes documentation has an example to co-locate cache with app on nodes about PodAffinity and ´PodAntiAffinity`.

Stable Network identity for StatefulSet

To address one specific instance of a StatefulSet, create an Headless Service with ClusterIP: None for each db replica. This allow your web-apps to connect to a specific instance.

Access the closest db instance

Now your web-apps can connect to db-0 and db-1 via the headless services. Let your web-apps connect to both initially, and use the one with shortest response time - that one is most likely the one on the same node.