How to use Avro on HDInsight Spark/Jupyter?

You just need to follow the below article

For HDInsight 3.3 and HDInsight 3.4

You will add below cell in your notebook

%%configure { "packages":["com.databricks:spark-avro_2.10:0.1"] }

For HDInsight 3.5

You will add below cell in your notebook

%%configure{ "conf": {"spark.jars.packages": "com.databricks:spark-avro_2.10:0.1" }}

For HDInsight 3.6

You will add below cell in your notebook

%%configure{ "conf": {"spark.jars.packages": "com.databricks:spark-avro_2.11:4.0.0" }}

CodeHunter