Apache SPARK:-Nullpointer Exception on broadcast variables (YARN Cluster mode)
This is because your broadcast variable is in class level. And since when the class is initialized in the worker node it will not see the value you assigned in the main method. It will only see a null since the broadcast variable is not initialized to anything. The Solution i found was to pass the broadcast variable to the method when calling the method. This is also the case for Accumulators