Setting YARN queue in PySpark

You can use below argument in you spark-submit command.

--queue queue_name

You can set this property in your code. spark.yarn.queue

Hope this will help.

Thanks

hadoop apache-spark pyspark hadoop-yarn

Try to use spark.yarn.queue rather than queue.

conf = pyspark.SparkConf().set("spark.yarn.queue", "your_queue_name")sc

CodeHunter