Spark exception handling for json
Unfortunately you are out of luck here. DataFrameReader.json which is used under the hood is pretty much all-or-nothing. If your input contains malformed lines you have to filter these manually. A basic solution could look like this:
import scala.util.parsing.json._val df = sqlContext.read.json( sc.textFile("file").filter(JSON.parseFull(_).isDefined))Since above validation is rather expensive you may prefer to drop jsonFile / read.json completely and to use parsed JSON lines directly.