Integrate Kubernetes with Yarn Integrate Kubernetes with Yarn kubernetes kubernetes

Integrate Kubernetes with Yarn


The hortonworks solution seems odd to me. Kubernetes manages docker containers on physical machines, Yarn schedule jobs a cluster (built by a set of linux boxes). The nature way of integration should be letting yarn running on docker containers managed by Kubernetes. Why the opposite?

IMO, the right solution is not integrate kubernete to your existing yarn cluster. Instead, you should rebuild your cluster with micro-service oriented architecture, which means every service (yarn is one of them) is running in docker containers managed by kubernetes. Long term, this will be a more manageable, maintenance-save approach for a big data cluster.