How to mount S3 bucket on Kubernetes container/pods?
We recently opensourced a project that looks to automate this steps for you: https://github.com/IBM/dataset-lifecycle-framework
Basically you can create a dataset:
apiVersion: com.ie.ibm.hpsys/v1alpha1kind: Datasetmetadata: name: example-datasetspec: local: type: "COS" accessKeyID: "iQkv3FABR0eywcEeyJAQ" secretAccessKey: "MIK3FPER+YQgb2ug26osxP/c8htr/05TVNJYuwmy" endpoint: "http://192.168.39.245:31772" bucket: "my-bucket-d4078283-dc35-4f12-a1a3-6f32571b0d62" region: "" #it can be emptyAnd then you will get a pvc you can mount in your pods
in general, you just don't do that. You should instead interact directly with S3 API to retrieve/store what you need (probably via some tools like aws cli).
As you run in AWS, you can have IAM configured in a way that your nodes can access particular data authorized on "infrastructure" level, or you can provide S3 access tokens via secrets/confogmaps/env etc.
S3 is not a filesystem, so don't expect it to behave like one (even if there are FUSE clients that emulate FS for your needs, this is rarely the right solution)