Making spark use /etc/hosts file for binding in YARN cluster mode
+1 for the question.
Spark uses Akka for communication.
So it's more of an Akka question than Spark.
If you need to bind your network interface to a different address - use akka.remote.netty.tcp.bind-hostname and akka.remote.netty.tcp.bind-port settings.
This is currently an issue in spark, the only way to get spark to bind to the proper interface is to use custom nameservers.
Spark essentially does a hostname lookup and uses the IP address that it finds to bind with Akka. Workaround is to create a custom bind zone and run a nameserver.