Spark Task not serializable (Case Classes) Spark Task not serializable (Case Classes) hadoop hadoop

Spark Task not serializable (Case Classes)


Most likely the function "doSomething" is defined on your class which isn't serilizable. Instead move the "doSomething" function to a companion object (e.g. make it static).


It was the dateFormatter, I placed it inside the partition loop and it works now.

usersRDD.foreachPartition(part => {    val id = userRow.id    val dateFormatter = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss")    val date1 = dateFormatter.parseDateTime(userRow.date1)})