unable to mount volume to spark.kubernetes.executor
this is an example from nfs-example
spark.kubernetes.driver.volumes.nfs.images.options.server=example.comspark.kubernetes.driver.volumes.nfs.images.options.path=/data
I think you need to declare the path that you want to mount under options.path
and the spark.kubernetes.driver.volumes.[VolumeType].[VolumeName].mount.path
is the mount path in your container
For example:
If I want to mount /home/lemon/data
on the node of k8s to the path /data
the docker container with VolumeName exepv
, then
conf.set("spark.kubernetes.executor.volumes.hostPath.exepv.mount.path","/data")conf.set("spark.kubernetes.executor.volumes.hostPath.exepv.options.path", "/home/lemon/data")
after this, you can access the path /data
in your executor container