script to download file from Amazon S3 bucket script to download file from Amazon S3 bucket curl curl

script to download file from Amazon S3 bucket


I write this bash script to download file from s3 (I download compressed file, you can change contentType to download other types of file)

#!/bin/sh outputFile="Your_PATH"amzFile="AMAZON_FILE_PATH"bucket="YOUR_BUCKET"resource="/${bucket}/${amzFile}"contentType="application/x-compressed-tar"dateValue=`date -R`stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}"s3Key="YOUR_S3_KEY"s3Secret="YOUR_S3SECRET"signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`curl  -H "Host: ${bucket}.s3.amazonaws.com" \     -H "Date: ${dateValue}" \     -H "Content-Type: ${contentType}" \     -H "Authorization: AWS ${s3Key}:${signature}" \     https://${bucket}.s3.amazonaws.com/${amzFile} -o $outputFile


Avoid signing the request yourself, a lot can go wrong or be hard to do. For example, you should check that the date is set to GMT or use x-amz-date headers.

Another approach is to use the AWS Command Line Interface and so use $ aws s3 cp or $ aws s3 sync.


As of August 2019 I found this to work. Has added region, and format of URL has changed.

#!/bin/sh outputFile="/PATH/TO/LOCALLY/SAVED/FILE"amzFile="BUCKETPATH/TO/FILE"region="YOUR-REGION"bucket="SOME-BUCKET"resource="/${bucket}/${amzFile}"contentType="binary/octet-stream"dateValue=`TZ=GMT date -R`# You can leave our "TZ=GMT" if your system is already GMT (but don't have to)stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}"s3Key="ACCESS_KEY_ID"s3Secret="SECRET_ACCESS_KEY"signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64`curl -H "Host: s3-${region}.amazonaws.com" \     -H "Date: ${dateValue}" \     -H "Content-Type: ${contentType}" \     -H "Authorization: AWS ${s3Key}:${signature}" \     https://s3-${region}.amazonaws.com/${bucket}/${amzFile} -o $outputFile