How to submit multiple jars to workers through sparkSession?
As explained in spark submit add multiple jars in classpath and Passing additional jars to Spark via spark-submit you should use comma separated list:
SparkSession spark = SparkSession.builder() .appName("MySQL Connection") .master("spark://ip:7077") .config("spark.jars", "/path/mysql.jar,/path/to/another.jar") .getOrCreate();
I know how to send multiple jars if used SparkConfig().setJars() but not of my use since I need to use SparkSession.
SparkConf
is still applicable for SparkSession
:
SparkConf conf;... SparkSession.builder().config(conf).getOrCreate();