script to download file from Amazon S3 bucket
I write this bash script to download file from s3 (I download compressed file, you can change contentType to download other types of file)
#!/bin/sh outputFile="Your_PATH"amzFile="AMAZON_FILE_PATH"bucket="YOUR_BUCKET"resource="/${bucket}/${amzFile}"contentType="application/x-compressed-tar"dateValue=`date -R`stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}"s3Key="YOUR_S3_KEY"s3Secret="YOUR_S3SECRET"signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`curl -H "Host: ${bucket}.s3.amazonaws.com" \ -H "Date: ${dateValue}" \ -H "Content-Type: ${contentType}" \ -H "Authorization: AWS ${s3Key}:${signature}" \ https://${bucket}.s3.amazonaws.com/${amzFile} -o $outputFile
Avoid signing the request yourself, a lot can go wrong or be hard to do. For example, you should check that the date is set to GMT or use x-amz-date headers.
Another approach is to use the AWS Command Line Interface and so use $ aws s3 cp
or $ aws s3 sync
.
As of August 2019 I found this to work. Has added region, and format of URL has changed.
#!/bin/sh outputFile="/PATH/TO/LOCALLY/SAVED/FILE"amzFile="BUCKETPATH/TO/FILE"region="YOUR-REGION"bucket="SOME-BUCKET"resource="/${bucket}/${amzFile}"contentType="binary/octet-stream"dateValue=`TZ=GMT date -R`# You can leave our "TZ=GMT" if your system is already GMT (but don't have to)stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}"s3Key="ACCESS_KEY_ID"s3Secret="SECRET_ACCESS_KEY"signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`curl -H "Host: s3-${region}.amazonaws.com" \ -H "Date: ${dateValue}" \ -H "Content-Type: ${contentType}" \ -H "Authorization: AWS ${s3Key}:${signature}" \ https://s3-${region}.amazonaws.com/${bucket}/${amzFile} -o $outputFile