unable to mount volume to spark.kubernetes.executor unable to mount volume to spark.kubernetes.executor kubernetes kubernetes

unable to mount volume to spark.kubernetes.executor


this is an example from nfs-example

spark.kubernetes.driver.volumes.nfs.images.options.server=example.comspark.kubernetes.driver.volumes.nfs.images.options.path=/data

I think you need to declare the path that you want to mount under options.path and the spark.kubernetes.driver.volumes.[VolumeType].[VolumeName].mount.path is the mount path in your container

For example:

If I want to mount /home/lemon/data on the node of k8s to the path /data the docker container with VolumeName exepv, then

conf.set("spark.kubernetes.executor.volumes.hostPath.exepv.mount.path","/data")conf.set("spark.kubernetes.executor.volumes.hostPath.exepv.options.path", "/home/lemon/data")

after this, you can access the path /data in your executor container