Copy files between s3 and kubernetes container Copy files between s3 and kubernetes container kubernetes kubernetes

Copy files between s3 and kubernetes container


Define an initContainer that mounts a PersistentVolume and uses the aws CLI or some other tool to retrieve files from your S3 bucket and copy them to the PV. Then mount this PV in your normal container as well and reference the files you retrieved accordingly.

Bonus points if you configure your initContainer to accept your AWS Access & Secret Access Keys as well as the S3 Bucket via Environment Variable and you pass these in on runtime.

Alternatively retrieve the files as part of a CI/CD pipeline before deploying the containers to Kubernetes. And volume mount them in accordingly using file or block storage.

EDIT: Looks like an AWS S3 Operator exists too. However, the project is not active nor is it officially created by AWS. I would not recommend using it. Especially when you can codify this process as part of a pipeline or your Deployment manifest.

EDIT 2: I also feel compelled to mention that containers are intended to be stateless and this might be why you're asking this question. While kubectl cp exists, it is by and large a non-standard way of getting files into Kubernetes. Normally you would do this through the use of a PersistentVolume that is provided by a StorageClass. With Docker, any changes made to the OS after execution are not persisted; the layer is discarded when the container exits. So rather than copying files into a container like you would a VM, try and use volume mounts to ensure they're there from the start.