Spark (Java) to Elasticsearch
Spark has executors and driver process. Executor runs in different node apart from driver node. Spark computes the rdd graph in various stages depending up on the transformations. And these stages have tasks that is executed on executors. So you need to pass the dependent jars to both executors and driver if you are using the library methods to compute rdd.
You should pass the dependent jars in --jars
options in spark-submit
spark-submit --jars $JARS \ --driver-class-path $JARS_COLON_SEP \ --class $CLASS_NAME $APP_JAR
In your case it would be
spark-submit --jars elasticsearch-hadoop-2.3.2.jar \ --master local[4]\ --driver-class-path elasticsearch-hadoop-2.3.2.jar \ --class "SimpleApp" target/simple-project-1.0.jar