Error from python worker:... SyntaxError: invalid syntax Error from python worker:... SyntaxError: invalid syntax hadoop hadoop

Error from python worker:... SyntaxError: invalid syntax


I had a similar issue with version 3.7, I solved it by adding the following to the Environment section (Eclipse -> Preferences -> PyDev -> Interpreters -> Python interpreter -> Environment):

PYSPARK_PYTHON /usr/local/bin/python3.7PYSPARK_DRIVER_PYTHON python3

Eclipse PyDev preferences