Validating Uploaded Files in Django

All the answers are focusing on validating files. This is pretty much impossible.

The Django devs aren't asking you to validate whether files can be executed as cgi files. They are just telling you not to put them in a place where they will be executed.

You should put all Django stuff in a specially Django directory. That Django code directory should not contain static content. Don't put user files in the Django source repository.

If you are using Apache2, check out the basic cgi tutorial: http://httpd.apache.org/docs/2.0/howto/cgi.html

Apache2 might be setup to run any files in the ScriptAlias folder. Don't put user files in the /cgi-bin/ or /usr/local/apache2/cgi-bin/ folders.

Apache2 might be set to server cgi files, depending on the AddHandler cgi-script settings. Don't let the users submit files with extensions like .cgi or .pl.

However, you do need to sanitize user submitted files so they are safe to run on other clients' machines. Submitted HTML is unsafe to other users. It won't hurt your server. Your server will just spit it back at whoever requests it. Get a HTML sanitizer.

Also, SVG may be unsafe. It's had bugs in the past. SVG is an XML document with javascript in it, so it can be malicious.

PDF is ... tricky. You could convert it to an image (if you really had to), or provide an image preview (and let users download at their own risk), but it would be a pain for people trying to use it.

Consider a white-list of files that are OK. A virus embedded in a gif, jpeg or png file will just look like a corrupt picture (or fail to display). If you want to be paranoid, convert them all to a standard format using PIL (hey, you could also check sizes). Sanitized HTML should be OK (stripping out script tags isn't rocket science). If the sanitization is sucking cycles (or you're just cautious), you could put it on a separate server, I guess.

For images you might be able to just use Python Imaging Library (PIL).

Image.open(filepath)

If the file is not an image, an exception will be thrown. I'm pretty new to Python/Django so someone else might have a better way of validating images.

The first thing you want to do with the uploaded content is store it in a directory which is not directly accessible for downloading. If your app exists in ~/www/ consider putting your data in '~/data/`.

The second thing, you need to determine what kind of file the user uploaded, and then create rules for each file type.

You can't trust the file based on the extension, so use something like Fileinfo. Then for each mime type, create a validator. ImageMagick can validate image files. For higher security, you may have to run a virus scanner over files like pdf's and flash files. For html, you may want to consider limit to a subset of tags.

I can't find a Python equivalent of the Fileinfo module, though it's always possible to exec /usr/bin/file -i. Most system that allow uploads then create a content name or id. They then use mod_rewrite to parse the URL, and find the content on disk. Once the content is found, it's returned to the user using sendfile, or something similar. For example, until the content is approved, maybe only the user who uploaded it is allowed to view it.