How to save Spark RDD to local filesystem
I think you should try "file:///local path" instead of "/local path".
The following code works fine:
val outputFilePath = "file:////home/opsdev/SDG/output/" DF.repartition(1) .write.mode(SaveMode.Append) .option("sep", "|") .option("header", "true") .option("escape", "\"") .option("quoteAll", "true") .csv(outputFilePath)And dont forget to run in local mode.