ElasticSearch to Spark RDD
You should use the following codes to initialize:
val sparkConf = new SparkConf().setAppName("Test").setMaster("local").set("spark.serializer", classOf[KryoSerializer].getName)val sc = new SparkContext(sparkConf)val conf = new JobConf()conf.set("es.nodes", "localhost:9200")conf.set("es.resource", "bank/account")conf.set("es.query", "?q=firstname:Daniel")
you can try
val spark = new SparkConf() .set("spark.serializer","org.apache.spark.serializer.KryoSerializer") .set("es.nodes",localhost) .set("es.port","9200") .appName("ES") .master("local[*]")val data = spark.read .format("org.elasticsearch.spark.sql") .option("es.query", "?q=firstname:Daniel")") .load("bank/account").rdddata.first()data.collect()