Spark and Elastic causing jackson overlap
You can add jars to the classpath programatically using addjar, inside file spark-defaults.conf or with spark-submit flags.
Also, We have 2 properties from runtime-environment docs here to mitigate conflicts
spark.driver.userClassPathFirstdefault false (Experimental) Whether to give user-added jars precedence over Spark's own jars when loading classes in the driver. This feature can be used to mitigate conflicts between Spark's dependencies and user dependencies. Note : It is currently an experimental feature. This is used in cluster mode only.
spark.executor.userClassPathFirstdefault false (Experimental) Same functionality as
spark.driver.userClassPathFirst, but applied to executor instances.
Properties set directly on the SparkConf take highest precedence, then flags passed to spark-submit or spark-shell, then options in the spark-defaults.conf file