Airflow SparkSubmitOperator - How to spark-submit in another server
To answer your first question, yes it is a good practice.
For how you can use SparkSubmitOperator, please refer to my answer on https://stackoverflow.com/a/53344713/5691525
- Yes, you need spark-binaries on airflow machine.
- -
- Yes
- No -> You still need a connection to tell Airflow where have you installed your spark binary files. Similar to https://stackoverflow.com/a/50541640/5691525
- Should work