From virtualenv, pip freeze > requirements.txt give TONES of garbage! How to trim it out?

That is one thing that has bugged me too quite a bit. This happens when you create a virtualenv without the --no-site-packages flag.

There are a couple of things you can do:

1. Create virtualenv with the --no-site-packages flag.
2. When installing apps, dont run pip install <name> directly, instead, add the library to your requirements.txt first, and then install the requirements. This is slower but makes sure your requirements are updated.
3. Manually delete libraries you dont need. A rule of thumb i follow for this is to add whatever is there in my INSTALLED_APPS, and database adapters. Most other required libraries will get installed automatically because of dependencies. I know its silly, but this is what I usually end up doing.

-- Edit --

I've since written a couple of scripts to help manage this. The first runs pip freeze and adds the found library to a provided requirements file, the other, runs pip install, and then adds it to the requirements file.

function pipa() {    # Adds package to requirements file.    # Usage: pipa <package> <path to requirements file>    package_name=$1    requirements_file=$2    if [[ -z $requirements_file ]]    then        requirements_file='./requirements.txt'    fi    package_string=`pip freeze | grep -i $package_name`    current_requirements=`cat $requirements_file`    echo "$current_requirements\n$package_string" | LANG=C sort | uniq > $requirements_file}function pipia() {    # Installs package and adds to requirements file.    # Usage: pipia <package> <path to requirements file>    package_name=$1    requirements_file=$2    if [[ -z $requirements_file ]]    then        requirements_file='./requirements.txt'    fi    pip install $package_name    pipa $package_name $requirements_file}

pipreqs solves the problem. It generates project-level requirement.txt file.

• Install pipreqs: pip install pipreqs
• Generate project-level requirement.txt file: pipreqs /path/to/your/project/requirements file would be saved in /path/to/your/project/requirements.txt

If you care a lot about the cleanliness of your requirements.txt you should not only use the --no-site-packages option as already mentioned but also consider not to pipe the output of pip freeze directly to your requirements.txt. The reason for that is, that when doing a pip freeze not only packages specified by yourself show up, but also dependencies installed by these packages! It isn't necessary to keep them all in your requirements.txt as they will get installed automatically with the package that requires them...So if you add a new package to your virtualenv you probably should just add the line for this package to your requirements.txt...

See this example:

(demo)[~]$ pip freezedistribute==0.6.19wsgiref==0.1.2(demo)[~]$ pip install django-blog-zinniaDownloading/unpacking django-blog-zinnia  Downloading django-blog-zinnia-0.9.tar.gz (523Kb): 523Kb downloaded  Running setup.py egg_info for package django-blog-zinnia    no previously-included directories found matching 'docs/api'    no previously-included directories found matching 'docs/build'    no previously-included directories found matching 'docs/coverage'    no previously-included directories found matching 'zinnia/media/zinnia/css/.sass-cache'Downloading/unpacking BeautifulSoup>=3.2.0 (from django-blog-zinnia)  Downloading BeautifulSoup-3.2.1.tar.gz  Running setup.py egg_info for package BeautifulSoup  # truncated as it installs some more dependenciesSuccessfully installed django-blog-zinnia BeautifulSoup django-mptt django-tagging django-xmlrpc pyparsingCleaning up...(demo)[~]$ pip freezeBeautifulSoup==3.2.1distribute==0.6.19django-blog-zinnia==0.9django-mptt==0.5.2django-tagging==0.3.1django-xmlrpc==0.1.3pyparsing==1.5.6wsgiref==0.1.2

(Though I should probably mentioned that in most cases it will not hurt that you have these dependencies there, just your file will grow and get harder to maintain.)