Support public and private networks on Mesos or Kubernetes?

On Kubernetes, there is not a fully supported way to do this. I think this is not supported by docker either (https://github.com/docker/docker/issues/1824)

As a work around, you could sort of do it this way:

• have one interface be the "default network interface" for pods. It is the one you configure Kubelet and docker to use. Most of your pods use this one. They get a PodIP.

• For "special" pods that need access to the other interface, or to both, use the "hostNet: true" parameter on those pods, and Kubernetes will not put the pod in a network container. These pods

  • can bind to either interface.
  • will not get a "podIP", but use the IPs of whichever interface they use.
  • you will have to manage port conflicts yourself. You may want to use DaemonSet for these pods.
  • you won't get any NetworkPolicy protection between pods with hostNet.
  • all the pods on the same node with hostNet will be able to talk to each other on localhost, so you get less isolation.

This workaround is good if only one or a few applications need "non-default" networking, and those apps are "system applications", managed by the same team that manages the cluster, rather than by a "less trusted" application team. Or if you have a small organization with only a few people running the Kubernetes cluster.