Architectural and design question about uploading photos from iPhone app and S3

I've discussed this issue on the AWS forums before. As I say there, the proper solution for accessing AWS from a mobile device is to use the AWS Identity and Access Management service to generate temporary, limited-privilege access keys for each user. The service is great, but it's still in beta for now and it's not part of the mobile SDK yet. I have a feeling once this thing is released for good, you'll see it out on the mobile SDK immediately afterwards.

Until then, generate presigned URLs for your users, or proxy through your own server like some others have suggested. The presigned URL will allow your users to temporarily GET or PUT to an S3 object in one of your buckets without actually having your credentials (they are hashed into the signature). You can read about the details here.

EDIT: I've implemented a proper solution for this problem, using the preview beta of IAM. It's available on GitHub, and you can read about it here.

Upload to your server and then post to S3. From an architecture standpoint you will want to do this from your server. There are many things that could go wrong during the data transfer you can handle better on the server and if you want to store any data about the image you are sending to S3 you are probably going to have a server side call anyway.

Plus, your Secret Access Key is stored in a more secure environment. Your own.

If you are worried about scalability and you are going to be doing a considerable number of S3 transfers I would consider hosting your server on and EC2 instance. Transferring data between the two is free (given you store you data in following data centers).

There is no Data Transfer charge for data transferred between Amazon EC2 and Amazon S3 within the same Region or for data transferred between the Amazon EC2 Northern Virginia Region and the Amazon S3 US Standard Region." Amazon Simple Storage Service (Amazon S3)

There are many post here on SO Amazon - EC2 cost? (example) about the pros and cons of using EC2.

You can upload directly from your iPhone to S3 using the REST API, and having the server be responsible for generating the part of the Authorization header value that requires the secret key. This way, you don't risk exposing the access key to anyone with a jailbroken iPhone, while you don't put the burden of uploading the file on the server. The exact details of the request to make can be found under "Example Object PUT" of "Signing and Authenticating REST Requests". I strongly recommend reading that document before proceeding any further.

The following code, written in Python, generates the part of the Authorization header value that is derived from your S3 secret access key. You should substitute your own secret access key and bucket name in virtual host form for _S3_SECRET and _S3_BUCKET_NAME below, respectively:

import base64from datetime import datetimeimport hmacimport sha# Replace these values._S3_SECRET = "my-s3-secret"_S3_BUCKET_NAME = "my-bucket-name"def get_upload_header_values(content_type, filename):   now = datetime.utcnow()  date_string = now.strftime("%a, %d %b %Y %H:%M:%S +0000")  full_pathname = '/%s/%s' % (_S3_BUCKET_NAME, filename)  string_to_sign = "PUT\n\n%s\n%s\n%s" % (      content_type, date_string, full_pathname)  h = hmac.new(_S3_SECRET, string_to_sign, sha)  auth_string = base64.encodestring(h.digest()).strip()  return (date_string, auth_string)

Calling this with the filename foo.txt and content-type text/plain yields:

>>> get_upload_header_values('text/plain', 'foo.txt')('Wed, 06 Feb 2013 00:57:45 +0000', 'EUSj3g70aEsEqSyPT/GojZmY8eI=')

Note that if you run this code, the time returned will be different, and so the encoded HMAC digest will be different.

Now the iPhone client just has to issue a PUT request to S3 using the returned date and HMAC digest. Assuming that

• the server returns date_string and auth_string above in some JSON object named serverJson
• your S3 Access Key (not your secret, which is only on the server) is named kS3AccessKey
• your S3 bucket name (set to my-bucket-name above) is named kS3BucketName
• the file contents are marshalled in an NSData object named data
• the filename that was sent to the server is a string named filename
• the content-type that was sent to the server is a string named contentType

Then you can do the following to create the NSURLRequest:

NSString *serverDate = [serverJson objectForKey:@"date"]NSString *serverHmacDigest = [serverJson objectForKey:@"hmacDigest"]// Create the headers.NSMutableDictionary *headers = [[NSMutableDictionary alloc] init];[headers setObject:contentType forKey:@"Content-Type"];NSString *host = [NSString stringWithFormat:@"%@.s3.amazonaws.com", kS3BucketName][headers setObject:host forKey:@"Host"];[headers setObject:serverDate forKey:@"Date"];NSString *authorization = [NSString stringWithFormat:@"AWS %@:%@", kS3AccessKey, serverHmacDigest];[headers setObject:authorization forKey:@"Authorization"];// Create the request.NSMutableURLRequest *request = [[NSMutableURLRequest alloc] init];[request setAllHTTPHeaderFields:headers];[request setHTTPBody:data];[request setHTTPMethod:@"PUT"];NSString *postUrl = [NSString stringWithFormat:@"http://%@.s3.amazonaws.com/%@", kS3BucketName, filename];[request setURL:[NSURL URLWithString:postUrl]];

Next you can issue the request. If you're using the excellent AFNetworking library, then you can wrap request in an AFXMLRequestOperation object using XMLDocumentRequestOperationWithRequest:success:failure:, and then invoking its start method. Don't forget to release headers and request when done.

Note that the client got the value of the Date header from the server. This is because, as Amazon describes under "Time Stamp Requirement":

"A valid time stamp (using either the HTTP Date header or an x-amz-date alternative) is mandatory for authenticated requests. Furthermore, the client time-stamp included with an authenticated request must be within 15 minutes of the Amazon S3 system time when the request is received. If not, the request will fail with the RequestTimeTooSkewed error status code. "

So instead of relying on the client having the correct time in order for the request to succeed, rely on the server, which should be using NTP (and a daemon like ntpd).