Python - Download File Using Requests, Directly to Memory

r.raw (HTTPResponse) is already a file-like object (just pass stream=True):

#!/usr/bin/env pythonimport sysimport requests # $ pip install requestsfrom PIL import Image # $ pip install pillowurl = sys.argv[1]r = requests.get(url, stream=True)r.raw.decode_content = True # Content-Encodingim = Image.open(r.raw) #NOTE: it requires pillow 2.8+print(im.format, im.mode, im.size)

In general if you have a bytestring; you could wrap it as f = io.BytesIO(r.content), to get a file-like object without touching the disk:

#!/usr/bin/env pythonimport ioimport zipfilefrom contextlib import closingimport requests # $ pip install requestsr = requests.get("http://www.pythonchallenge.com/pc/def/channel.zip")with closing(r), zipfile.ZipFile(io.BytesIO(r.content)) as archive:    print({member.filename: archive.read(member) for member in archive.infolist()})

You can't pass r.raw to ZipFile() directly because the former is a non-seekable file.

I would like to see if I can circumvent having to code a file deletion line

tempfile can delete files automatically f = tempfile.SpooledTemporaryFile(); f.write(u.content). Until .fileno() method is called (if some api requires a real file) or maxsize is reached; the data is kept in memory. Even if the data is written on disk; the file is deleted as soon as it closed.

Your answer is u.content. The content is in the memory. Unless you write it to a file, it won’t be stored on disk.

This is what I ended up doing.

import zipfile import requestsimport StringIOu = requests.get("http://www.pythonchallenge.com/pc/def/channel.zip")f = StringIO.StringIO() f.write(u.content)def extract_zip(input_zip):    input_zip = zipfile.ZipFile(input_zip)    return {i: input_zip.read(i) for i in input_zip.namelist()}extracted = extract_zip(f)