Spark on Kubernetes from jupyterhub not working with IRSA
Identified the issue was the aws region associated with spark pods.Included was region variable on the base spark docker image and the issue got resolved
Identified the issue was the aws region associated with spark pods.Included was region variable on the base spark docker image and the issue got resolved