ConcurrentModificationException when using Spark collectionAccumulator

I doubt if having synchronized block really helps. CustomeAccumulators or all other accumulator are not thread-safe. They do not really have to since the DAGScheduler.updateAccumulators method that the spark driver uses to update the values of accumulators after a task completes (successfully or with a failure) is only executed on a single thread that runs scheduling loop. Besides that, they are write-only data structures for workers that have their own local accumulator reference whereas accessing the value of an accumulator is only allowed by the driver. And when you say that it works in local mode because it is single JVM but in cluster mode, they are different JVM and java instance, PRC calls are being triggered to enable the communication.

How your MyRecord object looks like and if you just end your line with .value rather having an iterator over it will help. Just try.

myAccumulator.value

It makes sense to read the accumulator only after some action on the RDD has been called (collect or count).

Also you don't need to synchronize on the accumulator since an independent copy of it will be allocated per partition.